--- title: 'Personal Data Masking' source: 'https://academia.sh/en/courses/ai-safety-and-evaluation/personal-data-masking' course: 'AI Safety and Evaluation' language: en updated: '2026-10-21T05:02:28+00:00' license: 'CC BY-SA 4.0' objectives: - 'Count together the number of records that masking cleans in the data going to the model and the number of tasks that cannot be solved because of the field it deleted.' - 'Compare the three masking policies on the axes of escaped hidden field requests and broken tasks.' - 'Show why reading a defense that does not block with a filter measure is misleading.' --- # Personal Data Masking Reading the two numbers of reduction in the data that goes to the model together: flag-based masking removes a field from 32 of 240 records, and 14 of them belong to the hidden field request class and 9 are entirely clean records; the same policy makes 23 tasks that require the field unsolvable, and 7 of those 23 tasks belong to clean subscribers; 3 records of the hidden field request class do not carry the flag and so escape masking, the narrow policy raises the escaped count to 14 while lowering the broken tasks to 8, and the policy that masks every record misses no request but breaks 169 tasks, 104 of which belong to clean subscribers. Because masking blocks no record it has no false alarm, and reading it like a filter is misleading. In two lessons the defense decided on the whole of the record: let through, block, or put a class name. In all three the record was counted as an indivisible whole, and the measure was always read from the same pair — the harmful record blocked and the clean record blocked by mistake. Yet the part of a record that should go to the model can be separated from the part that should not. Masking does exactly this: the record passes, and certain fields inside it do not. Because no record is blocked, it has **no false alarm**, and that makes it look at first glance like a defense with no cost. The cost is somewhere else. The removed field is not only the thing a harmful request is after; it is also the thing some tasks **need in order to be solved**. This lesson puts those two numbers side by side: how many records masking cleans and how many tasks it deprives of the field they require. - **DF15.** The record set, the classes, the secret field classes, the task definition and the masking are **fictional**; no real personal data is processed and no real model endpoint is called. The seed is **20260218**, the second draw uses **20260219**. - **DF16.** **This is not an attack lesson.** All of the code that runs sits on the defense side: field removal and counting. No working attack, payload or evasion routine is written. - **DF17.** **An attack is not a text but a feature profile.** The `text` field of the records equals one of six ordinary support requests and **has no relation to the class**. - **DF18.** The general routine of personal data, privacy and masking was measured in the Detection, Response and Compliance and Model Evaluation, Interpretability and Ethics courses and is not repeated here. This lesson gives only the number of **reduction in the data that goes to the model**. - **DF19.** The baseline is the **setup without masking**: no field is removed, cleaned records 0, broken tasks 0. - **DF20.** The task definition is fictional: of the six support texts, four need one of the secret field classes to be solved and two do not. When the field is removed, the task counts as **unsolvable**. - **DF21.** The counts are independent of the run; masking produces no score, it looks at a structural flag. The step of the task rate is **0.0042**, the step of the clean record share is **0.0068**, the step of the hidden field request class is **0.0588**. ## The Defense That Removes the Field The first block sets up the course's shared definition and adds the fourth defense: the `SECRET_FIELD` dictionary names the four secret field classes, and `redact` reports that the field was removed in the records that carry a hidden field name. Masking returns no decision, it reports a **transformation**; the record itself goes on its way. ```python # FICTIONAL. The record set, classes, secret field classes and redaction are defined inside # the lesson; no real personal data is processed and no real model endpoint is called. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def pick(u, w): t = 0.0 for i, x in enumerate(w): t += x if u < t: return i return len(w) - 1 # ---- FICTIONAL records: no attack TEXT is written, an attack is a FEATURE PROFILE CLASSES = ["clean", "indirect_instruction", "hidden_field_request", "privilege_escalation", "source_spoofing", "corrupted_input"] SOURCES = ["user", "retrieved_document", "tool_output"] # Features are structural; none of them is a working payload. FEATURES = ["instruction_mode", "role_change_request", "hidden_field_name", "side_effect_tool_name", "source_stamp_mismatch", "unusual_stamp_distribution", "excessive_length"] # Feature probability per class (fictional). Clean records carry small shares too: # in the real world, well-meant text can carry these features as well. PROFILE = { "clean": [0.06, 0.02, 0.05, 0.03, 0.02, 0.04, 0.05], "indirect_instruction": [0.88, 0.42, 0.10, 0.30, 0.35, 0.08, 0.12], "hidden_field_request": [0.35, 0.15, 0.86, 0.12, 0.10, 0.06, 0.08], "privilege_escalation": [0.30, 0.25, 0.12, 0.84, 0.14, 0.05, 0.07], "source_spoofing": [0.22, 0.55, 0.08, 0.10, 0.82, 0.07, 0.06], "corrupted_input": [0.05, 0.03, 0.04, 0.03, 0.06, 0.80, 0.62], } SOURCE_SHARE = { "clean": [0.72, 0.20, 0.08], "indirect_instruction": [0.05, 0.70, 0.25], "hidden_field_request": [0.55, 0.30, 0.15], "privilege_escalation": [0.40, 0.35, 0.25], "source_spoofing": [0.10, 0.60, 0.30], "corrupted_input": [0.65, 0.20, 0.15], } # Fictional, entirely harmless support texts. They have NO relation to the class; # the class information lives only in the feature profile. TEXT = ["bu ayki fatura tutari gecen aya gore yuksek geldi", "sayac endeksi yanlis girilmis olabilir", "mahallede kesinti ne zaman bitecek", "tarife basamagimi ogrenmek istiyorum", "ariza kaydim icin ekip ne zaman gelir", "adres degisikligi nasil yapilir"] RECORDS = [] for i in range(240): r = rng(SEED + 6151 * i) c = CLASSES[pick(r(), [0.62, 0.10, 0.07, 0.07, 0.07, 0.07])] source = SOURCES[pick(r(), SOURCE_SHARE[c])] feat = {o: (r() < p) for o, p in zip(FEATURES, PROFILE[c])} RECORDS.append({"no": i + 1, "class": c, "source": source, "features": feat, "text": TEXT[int(r() * len(TEXT))], "length": 40 + int(r() * 260)}) def harmful(k): return k["class"] != "clean" # ---- DEFENSE 1: rule filter (structural, explainable) def rule_filter(k, rules=("instruction_mode", "hidden_field_name", "side_effect_tool_name", "source_stamp_mismatch")): return any(k["features"][o] for o in rules) # ---- DEFENSE 4: personal data masking (count only) SECRET_FIELD = {"subscriber_id": "identity", "contact_info": "contact", "address": "location", "access_string": "access"} def redact(k): """Fictional: in records that carry a hidden field name, the field is removed.""" return {"redacted": k["features"]["hidden_field_name"]} # ---- MEASURE: what was blocked and what was blocked by mistake are written together def measure(records, decision): tp = fp = tn = fn = 0 for k in records: flagged, actual = decision(k), harmful(k) if flagged and actual: tp += 1 elif flagged and not actual: fp += 1 elif not flagged and actual: fn += 1 else: tn += 1 precision = tp / (tp + fp) if tp + fp else 0.0 recall = tp / (tp + fn) if tp + fn else 0.0 return {"tp": tp, "fp": fp, "fn": fn, "tn": tn, "precision": round(precision, 4), "recall": round(recall, 4), "false_block_rate": round(fp / (fp + tn), 4) if fp + tn else 0.0} ``` ## Cleaned Record and Broken Task To measure the cost, the record has to have a **job** as well. The second block ties the six support texts to tasks and writes which secret field class each task needs: the questions about the invoice amount and the meter reading ask for the subscriber identity, the outage question asks for the address, the crew for the fault record asks for the contact information; the tariff step and the address change procedure ask for no field, because both are general explanations. Then three masking policies run side by side. ```python # FICTIONAL task definition: each support text is a task and some of them need a field from a # secret field class to be solved. TASK_FIELD = {TEXT[0]: "subscriber_id", TEXT[1]: "subscriber_id", TEXT[2]: "address", TEXT[3]: None, TEXT[4]: "contact_info", TEXT[5]: None} def policy(name): """Three redaction policies. None blocks; all remove a field.""" if name == "none": return lambda k: False if name == "flag": return lambda k: redact(k)["redacted"] if name == "narrow": return lambda k: redact(k)["redacted"] and k["source"] != "user" return lambda k: True def count(records, f): m = [k for k in records if f(k)] broken = [k for k in m if TASK_FIELD[k["text"]]] escaped = [k for k in records if k["class"] == "hidden_field_request" and not f(k)] return (len(m), sum(1 for k in m if not harmful(k)), len(broken), sum(1 for k in broken if not harmful(k)), len(escaped)) required = sum(1 for k in RECORDS if TASK_FIELD[k["text"]]) print("tasks that need a field:", required, "/", len(RECORDS), " hidden field request class:", sum(1 for k in RECORDS if k["class"] == "hidden_field_request")) print() print(f"{'policy':<10}{'masked':>11}{'of clean':>10}{'broken task':>15}" f"{'of clean':>10}{'escaped request':>16}") for name in ("none", "flag", "narrow", "all"): print(f"{name:<10}" + "".join(f"{v:>{g}}" for v, g in zip(count(RECORDS, policy(name)), (11, 10, 15, 10, 16)))) print() mask = [k for k in RECORDS if redact(k)["redacted"]] print("classes of the records the flag policy masked:", {s: sum(1 for k in mask if k["class"] == s) for s in CLASSES if sum(1 for k in mask if k["class"] == s)}) print("redaction read as a filter:", measure(RECORDS, lambda k: redact(k)["redacted"])) print("field class requested by the broken tasks:", {a: sum(1 for k in mask if TASK_FIELD[k["text"]] == a) for a in SECRET_FIELD if sum(1 for k in mask if TASK_FIELD[k["text"]] == a)}) print("field class no task requests:", [a for a in SECRET_FIELD if a not in TASK_FIELD.values()]) ``` ``` tasks that need a field: 169 / 240 hidden field request class: 17 policy masked of clean broken task of clean escaped request none 0 0 0 0 17 flag 32 9 23 7 3 narrow 11 2 8 2 14 all 240 147 169 104 0 classes of the records the flag policy masked: {'clean': 9, 'indirect_instruction': 4, 'hidden_field_request': 14, 'privilege_escalation': 2, 'source_spoofing': 2, 'corrupted_input': 1} redaction read as a filter: {'tp': 23, 'fp': 9, 'fn': 70, 'tn': 138, 'precision': 0.7188, 'recall': 0.2473, 'false_block_rate': 0.0612} field class requested by the broken tasks: {'subscriber_id': 10, 'contact_info': 6, 'address': 7} field class no task requests: ['access_string'] ``` ## Reading the Two Numbers Together The baseline row is as expected: with no masking the cleaned records are 0, the broken tasks are 0, and **all 17 records** of the hidden field request class go to the model together with the field they ask for. The flag policy removes a field from **32** of the 240 records. The distribution of those 32 shows how well the defense hits its target: **14** really belong to the hidden field request class, 9 to other harmful classes and **9 are entirely clean** records. The second half of the same row writes the cost. In **23** of the 32 masked records the task required the field, and because that field was deleted the task becomes unsolvable; **7** of those 23 tasks belong to **clean subscribers**. This is the cost of a defense that does not block the record: nobody was stopped, but twenty-three requests went unanswered. The third column pair writes the residue of the defense. **3 records** of the hidden field request class escape masking, because those records do not carry the hidden field name flag. The profile of the class carries this feature with a share of 0.86, not entirely. The scope of a defense cannot be larger than the scope of the flag that triggers it — and that flag is not the same thing as the class itself here. The narrow policy is the opposite end: only the records that carry the flag and come from a source other than the user are masked. Masked records fall from 32 to **11**, broken tasks from 23 to **8**, the broken tasks of clean subscribers from 7 to 2. The gain is real, but the escaped requests rise from 3 to **14**: fourteen of the seventeen records of the class now go to the model with their field. What shrinks together with the cost is the scope. The last row shows the choice at the extreme. Masking every record brings the escaped requests down to **zero** — the first place in this course where the residue of a defense comes out zero. Its cost is in the same row: **169 tasks** are broken, that is, all of the tasks that need a field, and **104 of them belong to clean subscribers**. 104 of the clean records, that is, 0.7075 of the 147, receive no answer. Zero residue is attainable as a defense target, and it is usually a target that should not be attained. The last two lines show that the cost is not spread equally over the field classes. Of the 23 broken tasks, **10** asked for the subscriber identity, 7 for the address and 6 for the contact information. The fourth secret field class, the access string, is asked for by **no task at all**. This means masking is not a single on-off switch: a field class that no task needs can be masked **at no cost**, while in the class that the most tasks need, every masking decision is the most expensive. The policy that masks every record never sees this distinction and pays for all 169 tasks. To see the distinction, it must be written down which task requires which field — this is not a security decision but a product decision, and when it is not written down, the masking policy becomes unmeasurable. The last output line is a warning. If we take masking for a filter and measure it with `measure`, we get a precision of 0.7188 and a recall of 0.2473. These numbers are **meaningless**: masking stops no record, so the 23 in the `tp` column are not "harmful records blocked" and the 9 in the `fp` column are not a false alarm. Reading a defense that does not block with a blocking measure credits it with both a success it does not deserve and a cost it does not pay. The two numbers of this defense are the **cleaned records** and the **broken tasks**. ## Independence from the Draw Masking produces no score, has no threshold and asks no model; it looks at a structural flag. Its decision is the same in every run for the same record. This does not mean the numbers are independent of the set as well: the third block draws the same definition again with a second seed. ```python def draw(seed): """Same definition, second draw: only the seed changes.""" k2 = [] for i in range(240): r = rng(seed + 6151 * i) c = CLASSES[pick(r(), [0.62, 0.10, 0.07, 0.07, 0.07, 0.07])] source = SOURCES[pick(r(), SOURCE_SHARE[c])] feat = {o: (r() < p) for o, p in zip(FEATURES, PROFILE[c])} k2.append({"no": i + 1, "class": c, "source": source, "features": feat, "text": TEXT[int(r() * len(TEXT))], "length": 40 + int(r() * 260)}) return k2 print(f"{'draw':<10}{'harmful':>8}{'masked':>11}{'broken':>9}" f"{'broken/masked':>15}{'escaped request':>16}{'step':>8}") for seed in (20260218, 20260219): kk = draw(seed) m, mc, br, brc, esc = count(kk, policy("flag")) n = sum(1 for k in kk if k["class"] == "hidden_field_request") print(f"{seed:<10}{sum(1 for k in kk if harmful(k)):>8}{m:>11}{br:>9}" f"{br / m:>15.4f}{str(esc) + '/' + str(n):>16}{1 / m:>8.4f}") print() print("task rate step:", round(1 / 240, 4), " clean record step:", round(1 / 147, 4)) ``` ``` draw harmful masked broken broken/masked escaped request step 20260218 93 32 23 0.7188 3/17 0.0312 20260219 83 25 17 0.6800 1/11 0.0400 task rate step: 0.0042 clean record step: 0.0068 ``` ## Summary - Masking blocks no record; its two numbers are the **cleaned records** and the **broken tasks**, and reading it with a filter measure is misleading. - The flag policy removes a field from 32 records — 14 of them from the hidden field request class, 9 entirely clean — and makes the 23 tasks that require the field unsolvable; 7 of those 23 belong to clean subscribers. - 3 records of the hidden field request class escape masking because they do not carry the flag: the scope of a defense cannot be larger than the scope of the flag that triggers it. - The narrow policy lowers the broken tasks from 23 to 8 and raises the escaped requests from 3 to 14; the policy that masks every record brings the escaped requests to zero and breaks 169 tasks, 104 of them belonging to clean subscribers. - 10 of the 23 broken tasks ask for the subscriber identity; no task asks for the access string, so that field class can be masked at no cost. Masking is not a single switch but a decision per field class. - In the two draws the cleaned records are 32 and 25 and the broken tasks 23 and 17; the ratio stays between 0.7188 and 0.6800 and the order does not change. ## Next Step In three lessons the defense always worked **on the text**. The rule filter read the structural features of the text, the content moderation step gave it a class name, masking removed a field from inside it. The assumption common to all three was that checking the text entering the model is enough. Yet the harm of a record is not in the text but in **the action the text triggers**: the tool called, the record written, the message sent. How much authority a request that got past the filter holds in its hands, how many harmful records narrowing that authority would stop and how many tasks it would leave unfinished was never asked. The next lesson takes the defense off the text and puts it on the **action**.