Input and Output Filters

The previous eight lessons counted the attack surface: five harmful classes, the source distribution of each, and the share the defense caught in that class. Throughout, the defense side was a single filter asked a single question — how much of this class does it block. The filter itself was never opened.

Yet a filter has two places to stand. One part looks at what can be read before the text enters the model, another at what the response carries out. They do not see the same record and do not catch the same class. A second question was never asked either: what happens when a second filter is placed on top of the first. This lesson counts both, and the answer to the second shapes the rest of the course.

• DF1. The record set, the classes and the defense layers are fictional; no real model endpoint is called. The seed is 20260218, the second run uses 20260219.
• DF2. This is not an attack lesson. All of the code that runs sits on the defense side: filter, count and measure. No working attack, payload or evasion routine is written.
• DF3. An attack is not a text but a feature profile. The text field carried by the records equals one of six ordinary support requests and has no relation to the class; the class information lives only in the structural feature profile.
• DF4. The baseline is the undefended setup: no filter, harmful records blocked 0, false positives 0.
• DF5. Splitting the rule-based filter into an input side and an output side is a fiction; the split is made by whether a feature can be read before the text enters the model or only after the response is produced.
• DF6. The general routine of input validation was measured in the Secure Coding course and is not repeated here; this lesson adds only what is specific to text going into a language model.
• DF7. The resolution of the set is the lower bound of the measurement band: the step of recall is 0.0108, the step of the false block rate is 0.0068.

The Record Set and the Two Numbers of a Defense

The first block sets up the course’s shared definition. The six sentences in the TEXT list are ordinary subscriber requests carrying no instruction, payload or evasion text, and the class they land in is random. What determines the class is the PROFILE dictionary: which structural feature each class carries with which probability. Clean records carry low shares too, because a well-meant request can be long or written in the instruction mode.

# FICTIONAL. The record set, classes, features and defense layers are defined inside
# the lesson; no real model endpoint is called.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL records: no attack TEXT is written, an attack is a FEATURE PROFILE
CLASSES = ["clean", "indirect_instruction", "hidden_field_request", "privilege_escalation",
           "source_spoofing", "corrupted_input"]
SOURCES = ["user", "retrieved_document", "tool_output"]
# Features are structural; none of them is a working payload.
FEATURES = ["instruction_mode", "role_change_request", "hidden_field_name", "side_effect_tool_name",
            "source_stamp_mismatch", "unusual_stamp_distribution", "excessive_length"]
# Feature probability per class (fictional). Clean records carry small shares too:
# in the real world, well-meant text can carry these features as well.
PROFILE = {
    "clean":                [0.06, 0.02, 0.05, 0.03, 0.02, 0.04, 0.05],
    "indirect_instruction": [0.88, 0.42, 0.10, 0.30, 0.35, 0.08, 0.12],
    "hidden_field_request": [0.35, 0.15, 0.86, 0.12, 0.10, 0.06, 0.08],
    "privilege_escalation": [0.30, 0.25, 0.12, 0.84, 0.14, 0.05, 0.07],
    "source_spoofing":      [0.22, 0.55, 0.08, 0.10, 0.82, 0.07, 0.06],
    "corrupted_input":      [0.05, 0.03, 0.04, 0.03, 0.06, 0.80, 0.62],
}
SOURCE_SHARE = {
    "clean":                [0.72, 0.20, 0.08],
    "indirect_instruction": [0.05, 0.70, 0.25],
    "hidden_field_request": [0.55, 0.30, 0.15],
    "privilege_escalation": [0.40, 0.35, 0.25],
    "source_spoofing":      [0.10, 0.60, 0.30],
    "corrupted_input":      [0.65, 0.20, 0.15],
}
# Fictional, entirely harmless support texts. They have NO relation to the class;
# the class information lives only in the feature profile.
TEXT = ["bu ayki fatura tutari gecen aya gore yuksek geldi",
        "sayac endeksi yanlis girilmis olabilir",
        "mahallede kesinti ne zaman bitecek",
        "tarife basamagimi ogrenmek istiyorum",
        "ariza kaydim icin ekip ne zaman gelir",
        "adres degisikligi nasil yapilir"]

RECORDS = []
for i in range(240):
    r = rng(SEED + 6151 * i)
    c = CLASSES[pick(r(), [0.62, 0.10, 0.07, 0.07, 0.07, 0.07])]
    source = SOURCES[pick(r(), SOURCE_SHARE[c])]
    feat = {o: (r() < p) for o, p in zip(FEATURES, PROFILE[c])}
    RECORDS.append({"no": i + 1, "class": c, "source": source, "features": feat,
                    "text": TEXT[int(r() * len(TEXT))],
                    "length": 40 + int(r() * 260)})


def harmful(k):
    return k["class"] != "clean"


# ---- DEFENSE 1: rule filter (structural, explainable)
def rule_filter(k, rules=("instruction_mode", "hidden_field_name", "side_effect_tool_name",
                          "source_stamp_mismatch")):
    return any(k["features"][o] for o in rules)


# ---- DEFENSE 2: model-based filter (score + threshold)
WEIGHT = {"instruction_mode": 0.42, "role_change_request": 0.26, "hidden_field_name": 0.44,
          "side_effect_tool_name": 0.40, "source_stamp_mismatch": 0.38,
          "unusual_stamp_distribution": 0.30, "excessive_length": 0.12}
SOURCE_BONUS = {"user": 0.00, "retrieved_document": 0.18, "tool_output": 0.12}


def score(k):
    p = sum(WEIGHT[o] for o in FEATURES if k["features"][o]) + SOURCE_BONUS[k["source"]]
    r = rng(SEED ^ trace_id(k["no"]))
    return min(max(p + (r() - 0.5) * 0.20, 0.0), 1.6)


def model_filter(k, threshold=0.40):
    return score(k) >= threshold


# ---- MEASURE: what was blocked and what was blocked by mistake are written together
def measure(records, decision):
    tp = fp = tn = fn = 0
    for k in records:
        flagged, actual = decision(k), harmful(k)
        if flagged and actual:
            tp += 1
        elif flagged and not actual:
            fp += 1
        elif not flagged and actual:
            fn += 1
        else:
            tn += 1
    precision = tp / (tp + fp) if tp + fp else 0.0
    recall = tp / (tp + fn) if tp + fn else 0.0
    return {"tp": tp, "fp": fp, "fn": fn, "tn": tn,
            "precision": round(precision, 4), "recall": round(recall, 4),
            "false_block_rate": round(fp / (fp + tn), 4) if fp + tn else 0.0}

The measure function puts the course’s rule into code: for every decision, the harmful records blocked (tp) and the clean records blocked by mistake (fp) come back side by side. The third column, fn, is the missed harmful record, that is, the false negative; throughout the course it is called the missed harmful record. A defense that reports precision alone or recall alone counts as unmeasured in this course.

Where the Filter Stands

The rule-based filter of the shared definition reads four features. Two of them are known before the record enters the model: whether the text is written in the instruction mode (instruction_mode) and whether the source stamp carried by the record matches the place it came from (source_stamp_mismatch). The other two become visible only while the response is produced: the hidden field name the response carries (hidden_field_name) and the requested side-effecting tool call (side_effect_tool_name). The first pair is the input filter, the second is the output filter.

INPUT_RULES = ("instruction_mode", "source_stamp_mismatch")
OUTPUT_RULES = ("hidden_field_name", "side_effect_tool_name")


def input_filter(k):
    """Structural features readable before the text reaches the model."""
    return rule_filter(k, INPUT_RULES)


def output_filter(k):
    """Field and call names the response will carry out."""
    return rule_filter(k, OUTPUT_RULES)


def show(name, s):
    print(f"{name:<20}{s['tp']:>4}{s['fp']:>4}{s['fn']:>4}{s['precision']:>10.4f}"
          f"{s['recall']:>12.4f}{s['false_block_rate']:>12.4f}")


print("records:", len(RECORDS), " harmful:", sum(1 for k in RECORDS if harmful(k)),
      " clean:", sum(1 for k in RECORDS if not harmful(k)))
print("source:", {a: sum(1 for k in RECORDS if k["source"] == a) for a in SOURCES})
print()
print(f"{'setup':<20}{'tp':>4}{'fp':>4}{'fn':>4}{'precision':>10}"
      f"{'recall':>12}{'false block':>12}")
show("undefended baseline", measure(RECORDS, lambda k: False))
show("input filter", measure(RECORDS, input_filter))
show("output filter", measure(RECORDS, output_filter))
show("input + output", measure(RECORDS, lambda k: input_filter(k) or output_filter(k)))
show("rule filter", measure(RECORDS, rule_filter))

output_only = [k for k in RECORDS if harmful(k) and output_filter(k) and not input_filter(k)]
input_only = [k for k in RECORDS if harmful(k) and input_filter(k) and not output_filter(k)]
missed = [k for k in RECORDS if harmful(k) and not input_filter(k) and not output_filter(k)]
print()
print(f"{'class':<20}{'input only':>13}{'output only':>14}{'missed':>7}")
for s in CLASSES[1:]:
    print(f"{s:<20}" + "".join(f"{sum(1 for k in u if k['class'] == s):>{g}}"
                               for u, g in ((input_only, 13), (output_only, 14), (missed, 7))))
print(f"{'total':<20}{len(input_only):>13}{len(output_only):>14}{len(missed):>7}")

records: 240  harmful: 93  clean: 147
source: {'user': 137, 'retrieved_document': 72, 'tool_output': 31}

setup                 tp  fp  fn precision      recall false block
undefended baseline    0   0  93    0.0000      0.0000      0.0000
input filter          55  13  38    0.8088      0.5914      0.0884
output filter         44  12  49    0.7857      0.4731      0.0816
input + output        74  23  19    0.7629      0.7957      0.1565
rule filter           74  23  19    0.7629      0.7957      0.1565

class                  input only   output only missed
indirect_instruction           11             1      0
hidden_field_request            2             8      1
privilege_escalation            0             6      1
source_spoofing                15             1      3
corrupted_input                 2             3     14
total                          30            19     19

Caught and Missed

The baseline row is the same in every lesson: with no defense, harmful records blocked 0, false positives 0, missed harmful records 93. Every defense places a cost on top of this zero.

The input filter alone blocks 55 harmful records and blocks 13 clean records: recall 0.5914, false block rate 0.0884. The output filter blocks 44 harmful records and blocks 12 clean records: recall 0.4731. Read on their own, the two numbers make the input side look far ahead. The second table shows that this is misleading: the 19 harmful records blocked by the output filter are records the input filter never sees, and 8 of them belong to the hidden field request class and 6 to privilege escalation. The structure of these two classes is exactly this — the input side of the record looks like an ordinary request, and what sets it apart is the field name the response will carry. Symmetrically, of the 30 records the input filter catches alone, 15 are source spoofing and 11 are indirect instruction.

The union row confirms that the two sides do different jobs: adding 55 and 44 gives not 99 but 74, which means that on 25 records both caught the same thing. Recall rises from 0.5914 to 0.7957 — a difference of nineteen times the resolution. The cost is in the same table: the false block rate rises from 0.0884 to 0.1565, and the blocked clean records from 13 to 23. The four-rule filter and the input-plus-output union give the identical row; the split does not distort the set.

The last column names the blindness of the defense. Of the 19 harmful records that the two filters miss together, 14 belong to the corrupted input class. The profile of that class does not carry any of the four rules with high probability; its distinguishing features are the stamp distribution and excessive length, and the rule set does not look at them. When a class is almost entirely missed, the average recall of 0.7957 hides it.

Layer on Layer

The first fix that comes to mind for the missed class is a second filter. The model-based filter gives a weighted score to all seven features, adds a bonus by source, and decides with a threshold; it also reads the stamp distribution that the rule-based filter ignores.

def score_second(k):
    """Same model, second run: only the noise seed changes."""
    p = sum(WEIGHT[o] for o in FEATURES if k["features"][o]) + SOURCE_BONUS[k["source"]]
    r = rng(20260219 ^ trace_id(k["no"]))
    return min(max(p + (r() - 0.5) * 0.20, 0.0), 1.6)


print(f"{'setup':<20}{'tp':>4}{'fp':>4}{'fn':>4}{'precision':>10}"
      f"{'recall':>12}{'false block':>12}")
show("rule filter", measure(RECORDS, rule_filter))
show("model filter 0.50", measure(RECORDS, lambda k: model_filter(k, 0.50)))
show("rule OR model", measure(RECORDS, lambda k: rule_filter(k) or model_filter(k, 0.50)))

added = [k for k in RECORDS if not rule_filter(k) and model_filter(k, 0.50)]
print("\nrecords the layer added to the rule filter:", len(added),
      " class:", [k["class"] for k in added])
print("new harmful records the layer added:", sum(1 for k in added if harmful(k)))

print("\nsecond run, same records, noise seed 20260219:")
show("rule filter", measure(RECORDS, rule_filter))
show("model filter 0.50", measure(RECORDS, lambda k: score_second(k) >= 0.50))
print("records whose decision changed between runs:",
      sum(1 for k in RECORDS if model_filter(k, 0.50) != (score_second(k) >= 0.50)))
print("recall step:", round(1 / 93, 4), " false block step:", round(1 / 147, 4))

setup                 tp  fp  fn precision      recall false block
rule filter           74  23  19    0.7629      0.7957      0.1565
model filter 0.50     66   8  27    0.8919      0.7097      0.0544
rule OR model         74  24  19    0.7551      0.7957      0.1633

records the layer added to the rule filter: 1  class: ['clean']
new harmful records the layer added: 0

second run, same records, noise seed 20260219:
rule filter           74  23  19    0.7629      0.7957      0.1565
model filter 0.50     69   8  24    0.8961      0.7419      0.0544
records whose decision changed between runs: 11
recall step: 0.0108  false block step: 0.0068

Defense Breaks Things Too

The third row is the proof of this course’s second claim and it has to be written by name: stacking the rule-based filter and the model-based filter is worse than the rule-based filter alone. Precision drops from 0.7629 to 0.7551, the false block rate rises from 0.1565 to 0.1633, and recall stays at 0.7957. The number of records the layer adds to the rule-based filter is 1, and that record belongs to the clean class. The sentence is this: the layer did not catch a single new harmful record, and blocked one extra clean record.

The numbers have to be read against the resolution. The change of 0.0068 in the false block rate is exactly one step, a single clean record. The difference in recall is zero, and this is not an unknown tied to the resolution but a real zero: the added set has a single element and it is not harmful.

Why it comes out this way is written in the second row. At a 0.50 threshold the model-based filter blocks fewer harmful records (66 against 74) but blocks far fewer clean records (8 against 23), and its precision is 0.8919. This is not a defense worse than the rule-based filter; it is a defense that stands at a different balance. Combining the two with OR does not take the good sides of the two defenses; it takes the union of the sets they flag; of the 74 records flagged by the defense with the higher precision, 73 are already inside the set flagged by the rule-based filter. The union only adds up the false positives. Adding a layer for its high precision means not asking on which set the precision was measured.

The last block separates uncertainty. The rule-based filter gives the identical row in both runs; it reads structural features, produces no score, and so is independent of the run. The model-based filter, on the other hand, moves: on the same 240 records the decision changes for 11 records, harmful records blocked rise from 66 to 69, and recall rises from 0.7097 to 0.7419. The difference is 0.0322, three times the resolution — not within the band but a real movement. The 0.0042 difference in precision and the zero difference in the false block rate, by contrast, cannot be measured. The recall that a defense reports from a single run lies, on this set, inside a band of at least three steps.

Summary

• The undefended baseline is the same in every lesson: harmful records blocked 0, false positives 0, missed harmful records 93.
• The rule-based filter splits into an input side (55 harmful records, 13 false positives) and an output side (44 harmful records, 12 false positives); their union gives 74 harmful records and 23 false positives, because they catch different classes.
• Of the 19 harmful records that the two miss together, 14 belong to the corrupted input class; the average recall of 0.7957 hides this blindness.
• Stacking the rule-based filter and the model-based filter lowers precision from 0.7629 to 0.7551, raises the false block rate from 0.1565 to 0.1633 and leaves recall unchanged: the layer blocks one extra clean record without catching a single new harmful one.
• The rule-based filter gives the same number in both runs; the recall of the model-based filter moves between 0.7097 and 0.7419, which is three resolution steps and cannot be reported from a single run.

Next Step

All the numbers of this lesson were read from a single decision: block or let through. A filter that calls a record harmful does not say which kind, and the measure is squeezed into a single pair of numbers, yet the last column showed that almost an entire class is missed. If a defense also produces a class name, it becomes possible to ask which class a single average rate hides. The next lesson turns the filter into a content moderation step and prints precision and recall separately for each class.