Jailbreak Techniques

The previous lesson measured the rule-based filter: it looks at four flags, blocks 74 of 93 harmful records, and wrongly blocks 23 clean ones. Its decision is sharp — a flag is present or absent. A filter that produces a score puts a threshold in between, and because the threshold is a setting, it is in the hands of whoever operates the defense. Where it is placed looks like a configuration detail, yet it determines how much of the surface stays open.

Guardrail bypass is not treated here as a procedure. No bypass method, template, or example is written; the only things written are the class name and the share the defense catches in that class. The question: as the threshold rises, how do precision and recall move in opposite directions, which class is the first to collapse, and above which threshold does the region most open to bypass begin?

• AS9. The record set, classes, and defense layers are fictional; the previous lesson’s definition continues unchanged. Seed 20260218.
• AS10. This is not an attack lesson. All code that runs is on the defense side: scoring, threshold, filter, and count. No working bypass method, template, or instruction sequence is written.
• AS11. An attack is not a text but an attribute profile; the text of the records is entirely harmless support requests and has no relation to the class.
• AS12. The model-based filter is a simulator: it produces a weighted sum, a source-dependent addition, and a narrow share of uncertainty. No real classifier is trained.
• AS13. The swept thresholds are five points; intermediate values were not measured, and the curve is not assumed to be flat between them.
• AS14. Per-class recall is divided by that class’s record count; the privilege escalation class has 10 records, and there one record is worth 0.1000.
• AS15. In the second run the set is unchanged; only the uncertainty share of the score is redrawn with the second seed. What is measured is the run variability of the model.
• AS16. The smallest recall step is 1/93 = 0.0108 and the false block rate step is 1/147 = 0.0068.

The Threshold Is Not a Defense, It Is a Trade-Off Point

The model-based filter gives every record a score: a weighted sum of the flags, an addition that depends on the source, and a narrow uncertainty share. The score is not a decision; the decision arises with the threshold. When the threshold is low the filter flags many records, when it is high it flags few. This setting determines not how good the defense is but where it stands, and it moves two numbers in opposite directions at once.

# FICTIONAL. The record set, classes, attributes and defense layers are defined inside
# the lesson; no real model endpoint is called. No attack TEXT is written.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL records: no attack TEXT is written, an attack is a FEATURE PROFILE
CLASSES = ["clean", "indirect_instruction", "hidden_field_request", "privilege_escalation",
           "source_spoofing", "corrupted_input"]
SOURCES = ["user", "retrieved_document", "tool_output"]
# Features are structural; none of them is a working payload.
FEATURES = ["instruction_mode", "role_change_request", "hidden_field_name", "side_effect_tool_name",
            "source_stamp_mismatch", "unusual_stamp_distribution", "excessive_length"]
# Feature probability per class (fictional). Clean records carry small shares too:
# in the real world, well-meant text can carry these features as well.
PROFILE = {
    "clean":                [0.06, 0.02, 0.05, 0.03, 0.02, 0.04, 0.05],
    "indirect_instruction": [0.88, 0.42, 0.10, 0.30, 0.35, 0.08, 0.12],
    "hidden_field_request": [0.35, 0.15, 0.86, 0.12, 0.10, 0.06, 0.08],
    "privilege_escalation": [0.30, 0.25, 0.12, 0.84, 0.14, 0.05, 0.07],
    "source_spoofing":      [0.22, 0.55, 0.08, 0.10, 0.82, 0.07, 0.06],
    "corrupted_input":      [0.05, 0.03, 0.04, 0.03, 0.06, 0.80, 0.62],
}
SOURCE_SHARE = {
    "clean":                [0.72, 0.20, 0.08],
    "indirect_instruction": [0.05, 0.70, 0.25],
    "hidden_field_request": [0.55, 0.30, 0.15],
    "privilege_escalation": [0.40, 0.35, 0.25],
    "source_spoofing":      [0.10, 0.60, 0.30],
    "corrupted_input":      [0.65, 0.20, 0.15],
}
# Fictional, entirely harmless support texts. They have NO relation to the class;
# the class information lives only in the feature profile.
TEXT = ["bu ayki fatura tutari gecen aya gore yuksek geldi",
        "sayac endeksi yanlis girilmis olabilir",
        "mahallede kesinti ne zaman bitecek",
        "tarife basamagimi ogrenmek istiyorum",
        "ariza kaydim icin ekip ne zaman gelir",
        "adres degisikligi nasil yapilir"]

RECORDS = []
for i in range(240):
    r = rng(SEED + 6151 * i)
    c = CLASSES[pick(r(), [0.62, 0.10, 0.07, 0.07, 0.07, 0.07])]
    source = SOURCES[pick(r(), SOURCE_SHARE[c])]
    feat = {o: (r() < p) for o, p in zip(FEATURES, PROFILE[c])}
    RECORDS.append({"no": i + 1, "class": c, "source": source, "features": feat,
                    "text": TEXT[int(r() * len(TEXT))],
                    "length": 40 + int(r() * 260)})


def harmful(k):
    return k["class"] != "clean"


# ---- DEFENSE 1: rule filter (structural, explainable)
def rule_filter(k, rules=("instruction_mode", "hidden_field_name", "side_effect_tool_name",
                          "source_stamp_mismatch")):
    return any(k["features"][o] for o in rules)


# ---- DEFENSE 2: model-based filter (score + threshold)
WEIGHT = {"instruction_mode": 0.42, "role_change_request": 0.26, "hidden_field_name": 0.44,
          "side_effect_tool_name": 0.40, "source_stamp_mismatch": 0.38,
          "unusual_stamp_distribution": 0.30, "excessive_length": 0.12}
SOURCE_BONUS = {"user": 0.00, "retrieved_document": 0.18, "tool_output": 0.12}


def score(k):
    p = sum(WEIGHT[o] for o in FEATURES if k["features"][o]) + SOURCE_BONUS[k["source"]]
    r = rng(SEED ^ trace_id(k["no"]))
    return min(max(p + (r() - 0.5) * 0.20, 0.0), 1.6)


def model_filter(k, threshold=0.40):
    return score(k) >= threshold


# ---- MEASURE: what was blocked and what was blocked by mistake are written together
def measure(records, decision):
    tp = fp = tn = fn = 0
    for k in records:
        flagged, actual = decision(k), harmful(k)
        if flagged and actual:
            tp += 1
        elif flagged and not actual:
            fp += 1
        elif not flagged and actual:
            fn += 1
        else:
            tn += 1
    precision = tp / (tp + fp) if tp + fp else 0.0
    recall = tp / (tp + fn) if tp + fn else 0.0
    return {"tp": tp, "fp": fp, "fn": fn, "tn": tn,
            "precision": round(precision, 4), "recall": round(recall, 4),
            "false_block_rate": round(fp / (fp + tn), 4) if fp + tn else 0.0}

The Sweep and the Rate of Change

The second block sweeps five thresholds, then counts at each step what is traded for what: when the threshold rises one step, how many more harmful records escape and how many clean records are released in return.

THRESHOLDS = (0.30, 0.40, 0.50, 0.60, 0.80)
print("threshold      flagged     blocked   false alarm  escaped  precision      recall  false block")
for th in THRESHOLDS:
    m = measure(RECORDS, lambda k, th=th: model_filter(k, th))
    print(f"{th:<9.2f} {m['tp'] + m['fp']:>12} {m['tp']:>11} {m['fp']:>13} {m['fn']:>8}"
          f" {m['precision']:>10.4f} {m['recall']:>11.4f} {m['false_block_rate']:>12.4f}")

print("\neffect of raising the threshold")
print("step                extra escapes       clean released ratio false alarms at upper threshold")
for lo, hi in zip(THRESHOLDS, THRESHOLDS[1:]):
    m_lo = measure(RECORDS, lambda k, th=lo: model_filter(k, th))
    m_hi = measure(RECORDS, lambda k, th=hi: model_filter(k, th))
    d_escaped, d_freed = m_hi["fn"] - m_lo["fn"], m_lo["fp"] - m_hi["fp"]
    print(f"{lo:.2f} -> {hi:.2f} {d_escaped:>20} {d_freed:>20} {d_escaped / d_freed:>5.2f} {m_hi['fp']:>31}")

print("\nsize of the differences relative to the resolution of the set")
print("step         recall difference steps false block difference steps")
for lo, hi in zip(THRESHOLDS, THRESHOLDS[1:]):
    m_lo = measure(RECORDS, lambda k, th=lo: model_filter(k, th))
    m_hi = measure(RECORDS, lambda k, th=hi: model_filter(k, th))
    d_recall = m_lo["recall"] - m_hi["recall"]
    d_false = m_lo["false_block_rate"] - m_hi["false_block_rate"]
    print(f"{lo:.2f} -> {hi:.2f} {d_recall:>17.4f} {d_recall / (1 / 93):>5.1f} {d_false:>22.4f}"
          f" {d_false / (1 / 147):>5.1f}")

threshold      flagged     blocked   false alarm  escaped  precision      recall  false block
0.30               111          84            27        9     0.7568      0.9032       0.1837
0.40                97          77            20       16     0.7938      0.8280       0.1361
0.50                74          66             8       27     0.8919      0.7097       0.0544
0.60                64          59             5       34     0.9219      0.6344       0.0340
0.80                47          46             1       47     0.9787      0.4946       0.0068

effect of raising the threshold
step                extra escapes       clean released ratio false alarms at upper threshold
0.30 -> 0.40                    7                    7  1.00                              20
0.40 -> 0.50                   11                   12  0.92                               8
0.50 -> 0.60                    7                    3  2.33                               5
0.60 -> 0.80                   13                    4  3.25                               1

size of the differences relative to the resolution of the set
step         recall difference steps false block difference steps
0.30 -> 0.40            0.0752   7.0                 0.0476   7.0
0.40 -> 0.50            0.1183  11.0                 0.0817  12.0
0.50 -> 0.60            0.0753   7.0                 0.0204   3.0
0.60 -> 0.80            0.1398  13.0                 0.0272   4.0

The sweep table does not show a one-directional improvement. As the threshold rises from 0.30 to 0.80, precision climbs from 0.7568 to 0.9787, the false block rate drops from 0.1837 to 0.0068, and the 111 flagged records fall to 47. The same movement takes recall from 0.9032 to 0.4946: at the highest threshold the defense misses 47 of the 93 harmful records. A setup that reports only precision looks nearly perfect at 0.80, and it stays that way as long as the escaped column is not written.

The second table turns the choice of threshold from a preference into a calculation. From 0.30 to 0.50 the trade is one to one: 7 clean records for 7 harmful ones, then 12 for 11. Above 0.50 the ratio breaks: in the 0.50 → 0.60 step, 3 clean records cost 7 harmful ones (2.33); in the 0.60 → 0.80 step, 4 clean records cost 13 harmful ones (3.25).

This is the region most open to bypass. A defense that raises the threshold above 0.50 to reduce false alarm complaints releases two to three times as many harmful records as the clean traffic it gains, and does so without any indicator getting worse: precision rises, the false block rate falls, the flagged records decrease. Here the bypass is opened not by a method but by the setting of whoever operates the defense.

The last column writes why this region is the most expensive. At threshold 0.60 the false alarms are 5 records, 0.0340 of 147 clean ones; going from there to 0.80 gains 4 clean records and releases 13 harmful ones. The place where loosening is most expensive is where false alarms are already the fewest: when the clean traffic to be gained runs out, each step only enlarges the escaped harmful records.

The third table says whether these differences are measurable. The smallest of the recall differences is 0.0752, seven times the step of the set (0.0108); the smallest of the false block differences is 0.0204, three times the step (0.0068). No step of the sweep falls below the resolution, so all four steps are real measurements. The closest to the floor is the 0.50 → 0.60 step: measurable, but too thin to build a policy on.

Where the Defense Collapses First

Overall recall is an average and puts all five harmful classes in one pot. The third block separates it per class and adds the share of the rule-based filter in the same classes.

THRESHOLDS = (0.30, 0.40, 0.50, 0.60, 0.80)
print("per-class recall: the share the defense caught in that class")
print(f"{'class':<20} {'records':>7} {'rule':>7} " + " ".join(f"{th:>6.2f}" for th in THRESHOLDS))
for c in CLASSES[1:]:
    g = [k for k in RECORDS if k["class"] == c]
    rule_recall = sum(1 for k in g if rule_filter(k)) / len(g)
    print(f"{c:<20} {len(g):>7} {rule_recall:>7.4f} " +
          " ".join(f"{sum(1 for k in g if model_filter(k, th)) / len(g):>6.4f}" for th in THRESHOLDS))


def second_score(k):
    """Same record, second run: the noise of the score is redrawn with the second seed."""
    p = sum(WEIGHT[o] for o in FEATURES if k["features"][o]) + SOURCE_BONUS[k["source"]]
    r = rng(20260219 ^ trace_id(k["no"]))
    return min(max(p + (r() - 0.5) * 0.20, 0.0), 1.6)


print(f"\n{'threshold':<9} {'recall':>21} {'steps':>5} {'false block rate':>21} {'steps':>5}"
      f" {'changed decisions':>17}")
for th in THRESHOLDS:
    m1 = measure(RECORDS, lambda k, th=th: model_filter(k, th))
    m2 = measure(RECORDS, lambda k, th=th: second_score(k) >= th)
    d_recall = abs(m2["recall"] - m1["recall"])
    d_false = abs(m2["false_block_rate"] - m1["false_block_rate"])
    changed = sum(1 for k in RECORDS if model_filter(k, th) != (second_score(k) >= th))
    print(f"{th:<9.2f} {m1['recall']:>10.4f} {m2['recall']:>10.4f} {d_recall / (1 / 93):>5.1f}"
          f" {m1['false_block_rate']:>10.4f} {m2['false_block_rate']:>10.4f}"
          f" {d_false / (1 / 147):>5.1f} {changed:>17}")

per-class recall: the share the defense caught in that class
class                records    rule   0.30   0.40   0.50   0.60   0.80
indirect_instruction      24  1.0000 1.0000 1.0000 0.9583 0.8333 0.6667
hidden_field_request      17  0.9412 0.9412 0.8824 0.7059 0.5882 0.4706
privilege_escalation      10  0.9000 0.9000 0.9000 0.8000 0.6000 0.4000
source_spoofing           23  0.8696 0.9565 0.8696 0.8261 0.8261 0.6087
corrupted_input           19  0.2632 0.6842 0.4737 0.2105 0.2105 0.2105

threshold                recall steps      false block rate steps changed decisions
0.30          0.9032     0.9140   1.0     0.1837     0.1837   0.0                 5
0.40          0.8280     0.8280   0.0     0.1361     0.1293   1.0                13
0.50          0.7097     0.7419   3.0     0.0544     0.0544   0.0                11
0.60          0.6344     0.6344   0.0     0.0340     0.0136   3.0                 9
0.80          0.4946     0.4624   3.0     0.0068     0.0068   0.0                 5

The class table shows what a single recall number hides. At threshold 0.50, while overall recall is 0.7097, the indirect injection class is at 0.9583 and the corrupted input class at 0.2105: the difference is 0.7478, and the defense is nearly complete on the first and nearly absent on the second. Corrupted input is the class that collapses first — it falls to 0.4737 already at 0.40, hits bottom at 0.50, and stays there for three thresholds. The reason is in the weight table: the weights of the two flags that class carries do not reach the threshold. The share of the rule-based filter is also low there (0.2632), because those two flags are not in the rule list at all. That class itself is handled in a separate lesson.

The rule column gives one more comparison: at threshold 0.50 the model-based filter is below the rule-based filter in all five classes and overall (0.7097 against 0.7957). A higher precision does not mean a better defense; the two numbers go in separate directions, and which to choose is a decision, not a measurement.

The right question is not “which is better” but “at which threshold does which dominate”. The triple of the rule-based filter is 0.7629 / 0.7957 / 0.1565. The only point where the model-based filter beats it on all three at once is 0.40: 0.7938 / 0.8280 / 0.1361. Such a point is not guaranteed; on another set, no dominant threshold is also ordinary.

The second-run table gives the band on both axes at once; the steps columns divide the difference by the step of the set. At thresholds 0.40 and 0.60, recall comes out exactly the same in both runs, yet at the same thresholds the decision of 13 and 9 records changes: the total stands, the individual records do not. In recall, 0.30 moves one step, and 0.50 and 0.80 move three steps; in the false block rate, 0.40 moves one step and 0.60 three. Movements of one step are exactly on the band and do not count as findings; differences of three steps are measurable but do not carry a threshold choice. The result on both axes is the same: on this set, choosing the threshold with decimal precision is meaningless.

Summary

• The threshold is not a defense but a trade-off point: as it rises, precision goes from 0.7568 to 0.9787, recall falls from 0.9032 to 0.4946, and the false block rate drops from 0.1837 to 0.0068.
• The rate of change of raising the threshold is one to one up to 0.50; above it, each clean record released costs 2.33 and 3.25 harmful ones.
• The region most open to bypass is above 0.50 and opens without any indicator getting worse; where it is most expensive is where false alarms are already the fewest (5 records at 0.60, 0.0340).
• Per-class recall opens what the average hides: at threshold 0.50, indirect injection is 0.9583 and corrupted input 0.2105; the class that collapses first is corrupted input, and the rule-based filter is weak there too.
• No step of the sweep falls below the resolution (smallest difference: 7 steps in recall, 3 in the false block rate); all four steps are real measurements.
• In the second run recall stays the same at 0.40 and 0.60 but the decision of 13 and 9 records changes; movements of one step are exactly on the band and do not count as findings.

Next Step

In these two lessons the measure of the defense was tied to one question: was the record blocked or not. Some harm has nothing to do with blocking. When a record passes, what is lost is sometimes not an action but a piece of information: the system instruction itself, a subscriber’s identity, or an access string. There the defense changes too — instead of blocking, removing the field, that is masking. The next lesson only counts: how many records carry a hidden field name, from how many of them masking removes the field, how many the filter catches, and how many records are left that none of them catches.