--- title: 'Input and Output Filters' source: 'https://academia.sh/en/courses/ai-safety-and-evaluation/input-and-output-filters' course: 'AI Safety and Evaluation' language: en updated: '2026-10-21T05:02:28+00:00' license: 'CC BY-SA 4.0' objectives: - 'Split the rule-based filter into an input side and an output side and count the harmful records each blocks together with its false positives.' - 'Show that stacking the rule-based filter and the model-based filter lowers precision and raises the false block rate while leaving recall unchanged.' - 'Write down which number moves and which stays fixed across two runs of the same setup, together with the resolution of the set.' --- # Input and Output Filters Counting the harmful records that rule-based and model-based checks block on the same record set together with the clean records they block by mistake: when the rule-based filter is split into an input side and an output side, one blocks 55 harmful records and the other 44, and their union rises to 74; the model-based filter at a 0.50 threshold blocks fewer harmful records but blocks far fewer clean ones; stacking the two is worse than the rule-based filter alone — recall stays at 0.7957, precision drops from 0.7629 to 0.7551, the false block rate rises from 0.1565 to 0.1633, and the layer blocks one extra clean record without catching a single new harmful one. The previous eight lessons counted the attack surface: five harmful classes, the source distribution of each, and the share the defense caught in that class. Throughout, the defense side was **a single filter** asked a single question — how much of this class does it block. The filter itself was never opened. Yet a filter has two places to stand. One part looks at what can be read **before the text enters the model**, another at what **the response carries out**. They do not see the same record and do not catch the same class. A second question was never asked either: what happens when a second filter is placed on top of the first. This lesson counts both, and the answer to the second shapes the rest of the course. - **DF1.** The record set, the classes and the defense layers are **fictional**; no real model endpoint is called. The seed is **20260218**, the second run uses **20260219**. - **DF2.** **This is not an attack lesson.** All of the code that runs sits on the defense side: filter, count and measure. No working attack, payload or evasion routine is written. - **DF3.** **An attack is not a text but a feature profile.** The `text` field carried by the records equals one of six ordinary support requests and **has no relation to the class**; the class information lives only in the structural feature profile. - **DF4.** The baseline is the **undefended setup**: no filter, harmful records blocked 0, false positives 0. - **DF5.** Splitting the rule-based filter into an input side and an output side is a fiction; the split is made by whether a feature can be read before the text enters the model or only after the response is produced. - **DF6.** The general routine of input validation was measured in the Secure Coding course and is not repeated here; this lesson adds only what is specific to **text going into a language model**. - **DF7.** The resolution of the set is the lower bound of the measurement band: the step of recall is **0.0108**, the step of the false block rate is **0.0068**. ## The Record Set and the Two Numbers of a Defense The first block sets up the course's shared definition. The six sentences in the `TEXT` list are ordinary subscriber requests carrying no instruction, payload or evasion text, and the class they land in is **random**. What determines the class is the `PROFILE` dictionary: which structural feature each class carries with which probability. Clean records carry low shares too, because a well-meant request can be long or written in the instruction mode. ```python # FICTIONAL. The record set, classes, features and defense layers are defined inside # the lesson; no real model endpoint is called. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def pick(u, w): t = 0.0 for i, x in enumerate(w): t += x if u < t: return i return len(w) - 1 def trace_id(m): h = 2166136261 for c in str(m): h = ((h ^ ord(c)) * 16777619) & M32 return h # ---- FICTIONAL records: no attack TEXT is written, an attack is a FEATURE PROFILE CLASSES = ["clean", "indirect_instruction", "hidden_field_request", "privilege_escalation", "source_spoofing", "corrupted_input"] SOURCES = ["user", "retrieved_document", "tool_output"] # Features are structural; none of them is a working payload. FEATURES = ["instruction_mode", "role_change_request", "hidden_field_name", "side_effect_tool_name", "source_stamp_mismatch", "unusual_stamp_distribution", "excessive_length"] # Feature probability per class (fictional). Clean records carry small shares too: # in the real world, well-meant text can carry these features as well. PROFILE = { "clean": [0.06, 0.02, 0.05, 0.03, 0.02, 0.04, 0.05], "indirect_instruction": [0.88, 0.42, 0.10, 0.30, 0.35, 0.08, 0.12], "hidden_field_request": [0.35, 0.15, 0.86, 0.12, 0.10, 0.06, 0.08], "privilege_escalation": [0.30, 0.25, 0.12, 0.84, 0.14, 0.05, 0.07], "source_spoofing": [0.22, 0.55, 0.08, 0.10, 0.82, 0.07, 0.06], "corrupted_input": [0.05, 0.03, 0.04, 0.03, 0.06, 0.80, 0.62], } SOURCE_SHARE = { "clean": [0.72, 0.20, 0.08], "indirect_instruction": [0.05, 0.70, 0.25], "hidden_field_request": [0.55, 0.30, 0.15], "privilege_escalation": [0.40, 0.35, 0.25], "source_spoofing": [0.10, 0.60, 0.30], "corrupted_input": [0.65, 0.20, 0.15], } # Fictional, entirely harmless support texts. They have NO relation to the class; # the class information lives only in the feature profile. TEXT = ["bu ayki fatura tutari gecen aya gore yuksek geldi", "sayac endeksi yanlis girilmis olabilir", "mahallede kesinti ne zaman bitecek", "tarife basamagimi ogrenmek istiyorum", "ariza kaydim icin ekip ne zaman gelir", "adres degisikligi nasil yapilir"] RECORDS = [] for i in range(240): r = rng(SEED + 6151 * i) c = CLASSES[pick(r(), [0.62, 0.10, 0.07, 0.07, 0.07, 0.07])] source = SOURCES[pick(r(), SOURCE_SHARE[c])] feat = {o: (r() < p) for o, p in zip(FEATURES, PROFILE[c])} RECORDS.append({"no": i + 1, "class": c, "source": source, "features": feat, "text": TEXT[int(r() * len(TEXT))], "length": 40 + int(r() * 260)}) def harmful(k): return k["class"] != "clean" # ---- DEFENSE 1: rule filter (structural, explainable) def rule_filter(k, rules=("instruction_mode", "hidden_field_name", "side_effect_tool_name", "source_stamp_mismatch")): return any(k["features"][o] for o in rules) # ---- DEFENSE 2: model-based filter (score + threshold) WEIGHT = {"instruction_mode": 0.42, "role_change_request": 0.26, "hidden_field_name": 0.44, "side_effect_tool_name": 0.40, "source_stamp_mismatch": 0.38, "unusual_stamp_distribution": 0.30, "excessive_length": 0.12} SOURCE_BONUS = {"user": 0.00, "retrieved_document": 0.18, "tool_output": 0.12} def score(k): p = sum(WEIGHT[o] for o in FEATURES if k["features"][o]) + SOURCE_BONUS[k["source"]] r = rng(SEED ^ trace_id(k["no"])) return min(max(p + (r() - 0.5) * 0.20, 0.0), 1.6) def model_filter(k, threshold=0.40): return score(k) >= threshold # ---- MEASURE: what was blocked and what was blocked by mistake are written together def measure(records, decision): tp = fp = tn = fn = 0 for k in records: flagged, actual = decision(k), harmful(k) if flagged and actual: tp += 1 elif flagged and not actual: fp += 1 elif not flagged and actual: fn += 1 else: tn += 1 precision = tp / (tp + fp) if tp + fp else 0.0 recall = tp / (tp + fn) if tp + fn else 0.0 return {"tp": tp, "fp": fp, "fn": fn, "tn": tn, "precision": round(precision, 4), "recall": round(recall, 4), "false_block_rate": round(fp / (fp + tn), 4) if fp + tn else 0.0} ``` The `measure` function puts the course's rule into code: for every decision, the **harmful records blocked** (`tp`) and the **clean records blocked by mistake** (`fp`) come back side by side. The third column, `fn`, is the **missed harmful record**, that is, the **false negative**; throughout the course it is called the missed harmful record. A defense that reports precision alone or recall alone counts as unmeasured in this course. ## Where the Filter Stands The rule-based filter of the shared definition reads four features. Two of them are known before the record enters the model: whether the text is written in the instruction mode (`instruction_mode`) and whether the source stamp carried by the record matches the place it came from (`source_stamp_mismatch`). The other two become visible only while the response is produced: the hidden field name the response carries (`hidden_field_name`) and the requested side-effecting tool call (`side_effect_tool_name`). The first pair is the **input filter**, the second is the **output filter**. ```python INPUT_RULES = ("instruction_mode", "source_stamp_mismatch") OUTPUT_RULES = ("hidden_field_name", "side_effect_tool_name") def input_filter(k): """Structural features readable before the text reaches the model.""" return rule_filter(k, INPUT_RULES) def output_filter(k): """Field and call names the response will carry out.""" return rule_filter(k, OUTPUT_RULES) def show(name, s): print(f"{name:<20}{s['tp']:>4}{s['fp']:>4}{s['fn']:>4}{s['precision']:>10.4f}" f"{s['recall']:>12.4f}{s['false_block_rate']:>12.4f}") print("records:", len(RECORDS), " harmful:", sum(1 for k in RECORDS if harmful(k)), " clean:", sum(1 for k in RECORDS if not harmful(k))) print("source:", {a: sum(1 for k in RECORDS if k["source"] == a) for a in SOURCES}) print() print(f"{'setup':<20}{'tp':>4}{'fp':>4}{'fn':>4}{'precision':>10}" f"{'recall':>12}{'false block':>12}") show("undefended baseline", measure(RECORDS, lambda k: False)) show("input filter", measure(RECORDS, input_filter)) show("output filter", measure(RECORDS, output_filter)) show("input + output", measure(RECORDS, lambda k: input_filter(k) or output_filter(k))) show("rule filter", measure(RECORDS, rule_filter)) output_only = [k for k in RECORDS if harmful(k) and output_filter(k) and not input_filter(k)] input_only = [k for k in RECORDS if harmful(k) and input_filter(k) and not output_filter(k)] missed = [k for k in RECORDS if harmful(k) and not input_filter(k) and not output_filter(k)] print() print(f"{'class':<20}{'input only':>13}{'output only':>14}{'missed':>7}") for s in CLASSES[1:]: print(f"{s:<20}" + "".join(f"{sum(1 for k in u if k['class'] == s):>{g}}" for u, g in ((input_only, 13), (output_only, 14), (missed, 7)))) print(f"{'total':<20}{len(input_only):>13}{len(output_only):>14}{len(missed):>7}") ``` ``` records: 240 harmful: 93 clean: 147 source: {'user': 137, 'retrieved_document': 72, 'tool_output': 31} setup tp fp fn precision recall false block undefended baseline 0 0 93 0.0000 0.0000 0.0000 input filter 55 13 38 0.8088 0.5914 0.0884 output filter 44 12 49 0.7857 0.4731 0.0816 input + output 74 23 19 0.7629 0.7957 0.1565 rule filter 74 23 19 0.7629 0.7957 0.1565 class input only output only missed indirect_instruction 11 1 0 hidden_field_request 2 8 1 privilege_escalation 0 6 1 source_spoofing 15 1 3 corrupted_input 2 3 14 total 30 19 19 ``` ## Caught and Missed The baseline row is the same in every lesson: with no defense, harmful records blocked **0**, false positives **0**, missed harmful records **93**. Every defense places a cost on top of this zero. The input filter alone blocks **55 harmful records** and blocks **13 clean** records: recall 0.5914, false block rate 0.0884. The output filter blocks **44 harmful records** and blocks **12 clean** records: recall 0.4731. Read on their own, the two numbers make the input side look far ahead. The second table shows that this is misleading: the **19 harmful records** blocked by the output filter are records the input filter **never sees**, and 8 of them belong to the hidden field request class and 6 to privilege escalation. The structure of these two classes is exactly this — the input side of the record looks like an ordinary request, and what sets it apart is the field name the response will carry. Symmetrically, of the 30 records the input filter catches alone, 15 are source spoofing and 11 are indirect instruction. The union row confirms that the two sides do different jobs: adding 55 and 44 gives not 99 but **74**, which means that on 25 records both caught the same thing. Recall rises from 0.5914 to 0.7957 — a difference of nineteen times the resolution. The cost is in the same table: the false block rate rises from 0.0884 to 0.1565, and the blocked clean records from 13 to 23. The four-rule filter and the input-plus-output union give the **identical row**; the split does not distort the set. The last column names the blindness of the defense. Of the 19 harmful records that the two filters miss together, **14 belong to the corrupted input class**. The profile of that class does not carry any of the four rules with high probability; its distinguishing features are the stamp distribution and excessive length, and the rule set does not look at them. When a class is almost entirely missed, the average recall of 0.7957 hides it. ## Layer on Layer The first fix that comes to mind for the missed class is a second filter. The model-based filter gives a weighted score to all seven features, adds a bonus by source, and decides with a threshold; it also reads the stamp distribution that the rule-based filter ignores. ```python def score_second(k): """Same model, second run: only the noise seed changes.""" p = sum(WEIGHT[o] for o in FEATURES if k["features"][o]) + SOURCE_BONUS[k["source"]] r = rng(20260219 ^ trace_id(k["no"])) return min(max(p + (r() - 0.5) * 0.20, 0.0), 1.6) print(f"{'setup':<20}{'tp':>4}{'fp':>4}{'fn':>4}{'precision':>10}" f"{'recall':>12}{'false block':>12}") show("rule filter", measure(RECORDS, rule_filter)) show("model filter 0.50", measure(RECORDS, lambda k: model_filter(k, 0.50))) show("rule OR model", measure(RECORDS, lambda k: rule_filter(k) or model_filter(k, 0.50))) added = [k for k in RECORDS if not rule_filter(k) and model_filter(k, 0.50)] print("\nrecords the layer added to the rule filter:", len(added), " class:", [k["class"] for k in added]) print("new harmful records the layer added:", sum(1 for k in added if harmful(k))) print("\nsecond run, same records, noise seed 20260219:") show("rule filter", measure(RECORDS, rule_filter)) show("model filter 0.50", measure(RECORDS, lambda k: score_second(k) >= 0.50)) print("records whose decision changed between runs:", sum(1 for k in RECORDS if model_filter(k, 0.50) != (score_second(k) >= 0.50))) print("recall step:", round(1 / 93, 4), " false block step:", round(1 / 147, 4)) ``` ``` setup tp fp fn precision recall false block rule filter 74 23 19 0.7629 0.7957 0.1565 model filter 0.50 66 8 27 0.8919 0.7097 0.0544 rule OR model 74 24 19 0.7551 0.7957 0.1633 records the layer added to the rule filter: 1 class: ['clean'] new harmful records the layer added: 0 second run, same records, noise seed 20260219: rule filter 74 23 19 0.7629 0.7957 0.1565 model filter 0.50 69 8 24 0.8961 0.7419 0.0544 records whose decision changed between runs: 11 recall step: 0.0108 false block step: 0.0068 ``` ## Defense Breaks Things Too The third row is the proof of this course's second claim and it has to be written by name: **stacking the rule-based filter and the model-based filter is worse than the rule-based filter alone.** Precision drops from 0.7629 to **0.7551**, the false block rate rises from 0.1565 to **0.1633**, and recall **stays at 0.7957**. The number of records the layer adds to the rule-based filter is **1**, and that record belongs to the **clean** class. The sentence is this: the layer **did not catch a single new harmful record, and blocked one extra clean record.** The numbers have to be read against the resolution. The change of 0.0068 in the false block rate is exactly one step, a single clean record. The difference in recall is **zero**, and this is not an unknown tied to the resolution but a real zero: the added set has a single element and it is not harmful. Why it comes out this way is written in the second row. At a 0.50 threshold the model-based filter blocks **fewer** harmful records (66 against 74) but blocks **far fewer** clean records (8 against 23), and its precision is 0.8919. This is not a defense worse than the rule-based filter; it is a defense that stands at **a different balance**. Combining the two with OR does not take the good sides of the two defenses; it takes the **union of the sets they flag**; of the 74 records flagged by the defense with the higher precision, 73 are already inside the set flagged by the rule-based filter. The union only adds up the false positives. Adding a layer for its high precision means not asking **on which set** the precision was measured. The last block separates uncertainty. The rule-based filter gives the **identical** row in both runs; it reads structural features, produces no score, and so is independent of the run. The model-based filter, on the other hand, moves: on the same 240 records the decision changes for 11 records, harmful records blocked rise from 66 to 69, and recall rises from 0.7097 to 0.7419. The difference is 0.0322, three times the resolution — not within the band but a **real** movement. The 0.0042 difference in precision and the zero difference in the false block rate, by contrast, cannot be measured. The recall that a defense reports from a single run lies, on this set, inside a band of at least three steps. ## Summary - The undefended baseline is the same in every lesson: harmful records blocked 0, false positives 0, missed harmful records 93. - The rule-based filter splits into an input side (55 harmful records, 13 false positives) and an output side (44 harmful records, 12 false positives); their union gives 74 harmful records and 23 false positives, because they catch different classes. - Of the 19 harmful records that the two miss together, 14 belong to the corrupted input class; the average recall of 0.7957 hides this blindness. - Stacking the rule-based filter and the model-based filter lowers precision from 0.7629 to 0.7551, raises the false block rate from 0.1565 to 0.1633 and leaves recall unchanged: the layer blocks one extra clean record without catching a single new harmful one. - The rule-based filter gives the same number in both runs; the recall of the model-based filter moves between 0.7097 and 0.7419, which is three resolution steps and cannot be reported from a single run. ## Next Step All the numbers of this lesson were read from a single decision: block or let through. A filter that calls a record harmful does not say **which kind**, and the measure is squeezed into a single pair of numbers, yet the last column showed that almost an entire class is missed. If a defense also produces a class name, it becomes possible to ask which class a single average rate hides. The next lesson turns the filter into a **content moderation** step and prints precision and recall separately for each class.