Permissions and Sandboxing

In three lessons the defense worked on the text. The input and output filters read a record and flagged it, content moderation separated it per class, masking deleted the field going to the model. What all three touched was the same: the text entering or leaving the model. What the agent does after reading that text was never touched.

This lesson moves the defense from the text to the action. Even if a record passes the filter, it is harmless if the capability it wants to touch was never granted. The only thing measured is this: as permissions narrow, how many harmful records stop, how many clean records stop by mistake, and how many tasks can still be completed.

• DF41. The code is a simulator; no real tool, file system or runtime is called. The record set, the classes and the features are fictional, the seed is 20260218.
• DF42. An attack is not a text but a feature profile. The six sentences the records carry are ordinary support requests and have no relation to the class; the class lives only in the structural flags. Nowhere in the code is there a working payload, template or instruction string.
• DF43. The baseline is the undefended setup: no filter, marking or gate; the harmful stopped is 0, the false alarm is 0.
• DF44. The rule of the course: a defense that does not write the share it stopped and the share it stopped by mistake together counts as unmeasured.
• DF45. The permission surface in this lesson is a pair: the capability the tool declares (read, external access, write, execution) and the scope it can touch (the source class). The capability a record asks for is derived only from the flags that exist; no new text is added to the set.
• DF46. The sandbox is the boundary this pair draws, and the gate stands in front of the executor: a request that does not pass never reaches the tool. The gate only stops; there is no path that leaves the boundary.
• DF47. The completed task is the share of clean records that are not stopped: how many of the 147 clean records can finish their job.
• DF48. Resolution: the step of recall is $1/93 = 0.0108$, the step of the false block rate is $1/147 = 0.0068$. There is no draw in the decision of the gate, so the measure is independent of the run; the same set gives the same number in every run.

Shared Definition

The record set, the classes, the defense layers and the measure are the same throughout the course. The set holds 240 fictional support records; each record has a class, a source and seven structural features. The measure function measure returns every decision with five numbers and never separates what is stopped from what is stopped by mistake.

SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL records: no attack TEXT is written, an attack is a FEATURE PROFILE
CLASSES = ["clean", "indirect_instruction", "hidden_field_request", "privilege_escalation",
           "source_spoofing", "corrupted_input"]
SOURCES = ["user", "retrieved_document", "tool_output"]
# Features are structural; none of them is a working payload.
FEATURES = ["instruction_mode", "role_change_request", "hidden_field_name", "side_effect_tool_name",
            "source_stamp_mismatch", "unusual_stamp_distribution", "excessive_length"]
# Feature probability per class (fictional). Clean records carry small shares too:
# well-meant text can carry these features as well.
PROFILE = {
    "clean":                [0.06, 0.02, 0.05, 0.03, 0.02, 0.04, 0.05],
    "indirect_instruction": [0.88, 0.42, 0.10, 0.30, 0.35, 0.08, 0.12],
    "hidden_field_request": [0.35, 0.15, 0.86, 0.12, 0.10, 0.06, 0.08],
    "privilege_escalation": [0.30, 0.25, 0.12, 0.84, 0.14, 0.05, 0.07],
    "source_spoofing":      [0.22, 0.55, 0.08, 0.10, 0.82, 0.07, 0.06],
    "corrupted_input":      [0.05, 0.03, 0.04, 0.03, 0.06, 0.80, 0.62],
}
SOURCE_SHARE = {
    "clean":                [0.72, 0.20, 0.08],
    "indirect_instruction": [0.05, 0.70, 0.25],
    "hidden_field_request": [0.55, 0.30, 0.15],
    "privilege_escalation": [0.40, 0.35, 0.25],
    "source_spoofing":      [0.10, 0.60, 0.30],
    "corrupted_input":      [0.65, 0.20, 0.15],
}
# Fictional, entirely harmless support texts. They have NO relation to the class;
# the class information lives only in the feature profile.
TEXT = ["bu ayki fatura tutari gecen aya gore yuksek geldi",
        "sayac endeksi yanlis girilmis olabilir",
        "mahallede kesinti ne zaman bitecek",
        "tarife basamagimi ogrenmek istiyorum",
        "ariza kaydim icin ekip ne zaman gelir",
        "adres degisikligi nasil yapilir"]

RECORDS = []
for i in range(240):
    r = rng(SEED + 6151 * i)
    c = CLASSES[pick(r(), [0.62, 0.10, 0.07, 0.07, 0.07, 0.07])]
    source = SOURCES[pick(r(), SOURCE_SHARE[c])]
    feat = {o: (r() < p) for o, p in zip(FEATURES, PROFILE[c])}
    RECORDS.append({"no": i + 1, "class": c, "source": source, "features": feat,
                    "text": TEXT[int(r() * len(TEXT))],
                    "length": 40 + int(r() * 260)})


def harmful(k):
    return k["class"] != "clean"


# ---- DEFENSE 1: rule filter (structural, explainable)
def rule_filter(k, rules=("instruction_mode", "hidden_field_name", "side_effect_tool_name",
                          "source_stamp_mismatch")):
    return any(k["features"][o] for o in rules)


# ---- MEASURE: what was blocked and what was blocked by mistake are written together
def measure(records, decision):
    tp = fp = tn = fn = 0
    for k in records:
        flagged, actual = decision(k), harmful(k)
        if flagged and actual:
            tp += 1
        elif flagged and not actual:
            fp += 1
        elif not flagged and actual:
            fn += 1
        else:
            tn += 1
    precision = tp / (tp + fp) if tp + fp else 0.0
    recall = tp / (tp + fn) if tp + fn else 0.0
    return {"tp": tp, "fp": fp, "fn": fn, "tn": tn,
            "precision": round(precision, 4), "recall": round(recall, 4),
            "false_block_rate": round(fp / (fp + tn), 4) if fp + tn else 0.0}


print("records:", len(RECORDS), "- harmful:", sum(harmful(k) for k in RECORDS),
      "- clean:", sum(not harmful(k) for k in RECORDS))
print("source:", {s: sum(k["source"] == s for k in RECORDS) for s in SOURCES})

records: 240 - harmful: 93 - clean: 147
source: {'user': 137, 'retrieved_document': 72, 'tool_output': 31}

The Permission Surface and the Boundary of the Sandbox

The principle of least privilege was established in the Introduction to Cybersecurity course, sandboxes and allowlists were measured in the Endpoint and System Hardening course, and file permissions in the Introduction to Linux course. The AI Agents course measured the same gate in the agent loop: at full permission 0.9583 of the tasks were completed, in the allowlisted set 0.8333, and the gate stopped 20 calls. None of these procedures is repeated; the numbers are input. The question here is different: how many harmful records does the same gate stop as a defense, and what does it cost.

Note that the gate only stops. No path that crosses the boundary is defined in the code; the sandbox function decides whether a request can pass, and a request that does not pass never reaches the tool. The five permission sets narrow step by step: each row is a subset of the one before.

# ---- Specific to this lesson: the CAPABILITY a record wants to touch is read from existing flags
def requested_capability(k):
    """No new text or payload is added; the capability is derived only from structural flags."""
    if k["features"]["side_effect_tool_name"]:
        return "execution" if k["features"]["instruction_mode"] else "write"
    if k["source"] != "user":
        return "external_access"
    return "read"


ALL_CAPABILITIES = ("read", "external_access", "write", "execution")
ALL_SCOPES = ("user", "retrieved_document", "tool_output")
PERMISSIONS = {
    "full access":     (ALL_CAPABILITIES, ALL_SCOPES),
    "no execution":    (("read", "external_access", "write"), ALL_SCOPES),
    "no side effects": (("read", "external_access"), ALL_SCOPES),
    "allowlisted":     (("read", "external_access"), ("user", "retrieved_document")),
    "read-only":       (("read",), ("user",)),
}


def sandbox(k, p):
    """The gate stands in front of the executor and ONLY stops; there is no path
    that leaves the boundary. A request that does not pass never reaches the tool."""
    capability, scope = p
    return requested_capability(k) not in capability or k["source"] not in scope


print("requested capability:", {g: sum(requested_capability(k) == g for k in RECORDS)
                                for g in ALL_CAPABILITIES})
print("\npermission set   block   false  precision      recall  false-block   task   marginal")
prev_tp = prev_fp = 0
for name, p in PERMISSIONS.items():
    s = measure(RECORDS, lambda k, p=p: sandbox(k, p))
    d_tp, d_fp = s["tp"] - prev_tp, s["fp"] - prev_fp
    marginal = f"{d_tp / (d_tp + d_fp):.4f}" if d_tp + d_fp else "-"
    print(f"{name:<15} {s['tp']:>6} {s['fp']:>7} {s['precision']:>10.4f} {s['recall']:>11.4f}"
          f" {s['false_block_rate']:>12.4f} {s['tn'] / 147:>6.4f} {marginal:>10}")
    prev_tp, prev_fp = s["tp"], s["fp"]

print("\nshare stopped per class (allowlisted set)")
for s in CLASSES[1:]:
    g = [k for k in RECORDS if k["class"] == s]
    d = sum(sandbox(k, PERMISSIONS["allowlisted"]) for k in g)
    print(f"  {s:<20} {d:>2}/{len(g):<3} {d / len(g):.4f}")


def allowlisted(k):
    return sandbox(k, PERMISSIONS["allowlisted"])


print("\nlayer on layer            block   false  precision      recall  false-block")
for name, decision in (("rule filter", rule_filter),
                       ("permission gate", allowlisted),
                       ("rule OR permission", lambda k: rule_filter(k) or allowlisted(k))):
    s = measure(RECORDS, decision)
    print(f"{name:<24} {s['tp']:>6} {s['fp']:>7} {s['precision']:>10.4f}"
          f" {s['recall']:>11.4f} {s['false_block_rate']:>12.4f}")
added = [k for k in RECORDS if allowlisted(k) and not rule_filter(k)]
print("added by permission only:", len(added), "records,",
      sum(harmful(k) for k in added), "harmful")

requested capability: {'read': 129, 'external_access': 84, 'write': 15, 'execution': 12}

permission set   block   false  precision      recall  false-block   task   marginal
full access          0       0     0.0000      0.0000       0.0000 1.0000          -
no execution        12       0     1.0000      0.1290       0.0000 1.0000     1.0000
no side effects     24       3     0.8889      0.2581       0.0204 0.9796     0.8000
allowlisted         34      17     0.6667      0.3656       0.1156 0.8844     0.4167
read-only           62      49     0.5586      0.6667       0.3333 0.6667     0.4667

share stopped per class (allowlisted set)
  indirect_instruction 14/24  0.5833
  hidden_field_request  1/17  0.0588
  privilege_escalation 10/10  1.0000
  source_spoofing       6/23  0.2609
  corrupted_input       3/19  0.1579

layer on layer            block   false  precision      recall  false-block
rule filter                  74      23     0.7629      0.7957       0.1565
permission gate              34      17     0.6667      0.3656       0.1156
rule OR permission           78      37     0.6783      0.8387       0.2517
added by permission only: 18 records, 4 harmful

The Marginal Value of Narrowing

The first row is the baseline: the gate stops nothing, all 93 harmful records pass and all 147 clean records finish their job. Every row after that stops something and pays a price.

The second row is the cheapest gain of this lesson. Closing the execution capability stops 12 harmful records, blocks no clean record, leaves the precision at 1.0000 and the completed task at 1.0000. The reason is in the structure: the execution capability requires two flags to be present together, and that is not where clean traffic goes. This is the measurable core of the principle of least privilege — closing a capability that clean traffic does not visit is a free defense.

The last column says where that ends. Marginal precision is the ratio of the extra harmful records a narrowing step stops to the total extra records stopped in that step. The sequence runs 1.0000, 0.8000, 0.4167, 0.4667. At the third step the ratio falls below 0.5000: moving to the allowlist stops 10 more harmful records while blocking 14 more clean records. The fourth step is heavier still — 32 clean records for 28 harmful. This is the evidence this lesson contributes to the second claim of the course: past a certain point, continuing to narrow stops more tasks than harmful records. The number is still growing, because recall rises from 0.3656 to 0.6667; but in the same step the completed task drops from 0.8844 to 0.6667. Reading recall alone would count this step as an improvement.

Resolution tells whether the differences are measured. The last jump of 0.3011 in recall is twenty-eight times the step of 0.0108; the drop of 0.2177 in the completed task is thirty-two times the step of 0.0068. Both are far above the band. On the other hand, the difference in false blocks between no execution and full access is exactly zero, and that is not rounding but the count itself.

Where the Gate Is Blind

The last block opens what an average hides. The allowlisted set gives a recall of 0.3656 in general, but the distribution per class is not flat at all: 1.0000 of the privilege_escalation class is stopped, 0.5833 of the indirect_instruction class, and only 0.0588 of the hidden_field_request class.

The reason is what the gate measures. The permission gate looks at what a record wants to do, not what it says. A record carrying a hidden field request mostly asks only for the read capability; read stays open in every set, because when it is closed no task remains. The permission gate is blind to that class by its structure, and this blindness is not closed by narrowing.

Two conclusions follow. First, the permission layer does not take the place of a filter: the two layers look at different classes, and the recall of one does not fill the blind spot of the other. Second, reporting a defense with a single average rate is misleading — the number 0.3656 contains both 1.0000 and 0.0588 and does not say which attack is stopped by which layer.

Layer on Layer

The first answer that comes to mind for the blind spot is to stack two layers: let the rule filter look at the text and the permission gate at the action, and a record that is caught by either is enough. The last table gives the number, and the result is the opposite of what was expected.

The rule filter alone stops 74 harmful records and blocks 23 clean records. Putting the permission gate on top raises recall from 0.7957 to 0.8387, that is, four more harmful records. In the same step the false block rate rises from 0.1565 to 0.2517 and the precision falls from 0.7629 to 0.6783. The last line says why: of the 18 records that the permission gate adds to the filter, only 4 are harmful and 14 are clean. The marginal precision of the layer is $4/18 = 0.2222$.

Four harmful records are four times the step of 0.0108 and are a measured gain; fourteen clean records are fourteen times the step of 0.0068 and are a measured loss. Both are real; which one weighs more is read not from the number but from how expensive a blocked task is. The one thing the number says for certain is this: stacking two defenses does not enlarge recall for free. Reporting this table with the recall column alone would have made the gain of the added layer look larger than it is.

Summary

• The permission surface is the pair of the capability a tool declares and the scope it can touch; the sandbox is the boundary this pair draws, and the gate stands in front of the executor and only stops.
• On the undefended baseline the harmful stopped is 0 and the false alarm is 0; closing the execution capability stops 12 harmful records and blocks no clean record, with precision 1.0000 and completed task 1.0000. The allowlisted set stops 34 harmful and lowers the task to 0.8844, read-only stops 62 and lowers it to 0.6667: while recall rises to 0.6667, a third of the tasks disappear.
• Marginal precision falls from 1.0000 to 0.4167: the last two narrowing steps block more than one clean record for every harmful record they stop.
• The permission gate is class-selective: it stops 1.0000 of the privilege_escalation class and only 0.0588 of the hidden_field_request class; a single average rate hides this distinction.
• Putting the permission gate on top of the rule filter raises recall from 0.7957 to 0.8387 while it moves the false block rate from 0.1565 to 0.2517; of the 18 added records only 4 are harmful.
• There is no draw in the decision of the gate, the measure is independent of the run; the recall step is 0.0108, the false block step is 0.0068.

Next Step

Every defense in this lesson worked in the same way: it stopped a record. The cost of stopping is also written in the table — the four narrowing steps blocked 49 clean records in total. Yet stopping is not the only way to neutralize a harmful record. The harm of a record often comes not from its content but from that content being read as an instruction; if the text that comes from a retrieved document is counted as data, it becomes harmless without being blocked at all. The next lesson measures this defense that does not block and counts two things: how many of the 93 harmful records marking neutralizes without stopping any record, which classes remain, and what the false alarm counterpart of this is.