Production Monitoring

For four lessons the measurement was done in the laboratory. The set was designed, the judge was compared with the human label, the agreement between raters was measured, and whether a prompt change moved the number above the run band was tested. All of it was done on a fixed set. The traffic of a system running in production is not fixed, it has no labels, and the settings of the defense change silently one day. The laboratory number does not say what is true after that day.

The procedure of indicators, alerts and dashboards was measured in the Observability and Operations courses of the Observability and Reliability and DevOps curricula; the step trace and token cost were counted in the AI Agents and Building Applications with Model APIs courses. None of it is repeated. The only thing this lesson measures is this: when a defense degrades, which indicator shows it after how many records.

• EM51. The stream, the records, the classes and the defense are fictional. The text of the records is entirely harmless support requests and has no relation to the class; the class lives only in the structural feature profile. The stream comes from the same generator as the course’s record set, extended to 600 records.
• EM52. The defense running in production is the model filter, threshold 0.50. As in every lesson, the undefended baseline contains no filter, marking or gate.
• EM53. The trace record is kept for every request and carries no class label: decision, source and structural flags. A label arises only for records sent to human review.
• EM54. Two degradations begin after record 301 and are reported nowhere. Wave: the share of corrupted input rises from 0.07 to 0.28. Threshold: the filter’s threshold drops from 0.50 to 0.30.
• EM55. An indicator counts as having reported a degradation if it leaves the band for 10 consecutive records. The band is the smallest and largest window value over the 300 records before the change.
• EM56. The two runs use seeds 20260218 and 20260219; a run changes the stream itself, not the defense.

The Stream and the Undefended Baseline

The first block builds the common definition of the course and extends it to a stream. Every function that runs is on the defense side.

# FICTIONAL. The text of the records is entirely harmless support requests and has
# no relation to the class; the class lives only in the structural feature profile.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


CLASSES = ["clean", "indirect_instruction", "hidden_field_request", "privilege_escalation",
           "source_spoofing", "corrupted_input"]
SOURCES = ["user", "retrieved_document", "tool_output"]
FEATURES = ["instruction_mode", "role_change_request", "hidden_field_name", "side_effect_tool_name",
            "source_stamp_mismatch", "unusual_stamp_distribution", "excessive_length"]
PROFILE = {
    "clean":                [0.06, 0.02, 0.05, 0.03, 0.02, 0.04, 0.05],
    "indirect_instruction": [0.88, 0.42, 0.10, 0.30, 0.35, 0.08, 0.12],
    "hidden_field_request": [0.35, 0.15, 0.86, 0.12, 0.10, 0.06, 0.08],
    "privilege_escalation": [0.30, 0.25, 0.12, 0.84, 0.14, 0.05, 0.07],
    "source_spoofing":      [0.22, 0.55, 0.08, 0.10, 0.82, 0.07, 0.06],
    "corrupted_input":      [0.05, 0.03, 0.04, 0.03, 0.06, 0.80, 0.62],
}
SOURCE_SHARE = {
    "clean":                [0.72, 0.20, 0.08],
    "indirect_instruction": [0.05, 0.70, 0.25],
    "hidden_field_request": [0.55, 0.30, 0.15],
    "privilege_escalation": [0.40, 0.35, 0.25],
    "source_spoofing":      [0.10, 0.60, 0.30],
    "corrupted_input":      [0.65, 0.20, 0.15],
}
TEXT = ["bu ayki fatura tutari gecen aya gore yuksek geldi",
        "sayac endeksi yanlis girilmis olabilir",
        "mahallede kesinti ne zaman bitecek",
        "tarife basamagimi ogrenmek istiyorum",
        "ariza kaydim icin ekip ne zaman gelir",
        "adres degisikligi nasil yapilir"]
SHARE = [0.62, 0.10, 0.07, 0.07, 0.07, 0.07]        # distribution of the course's record set


def record(i, base, share):
    r = rng(base + 6151 * i)
    c = CLASSES[pick(r(), share)]
    source = SOURCES[pick(r(), SOURCE_SHARE[c])]
    feat = {o: (r() < p) for o, p in zip(FEATURES, PROFILE[c])}
    return {"no": i + 1, "class": c, "source": source, "features": feat,
            "text": TEXT[int(r() * len(TEXT))], "length": 40 + int(r() * 260)}


def harmful(k):
    return k["class"] != "clean"


WEIGHT = {"instruction_mode": 0.42, "role_change_request": 0.26, "hidden_field_name": 0.44,
          "side_effect_tool_name": 0.40, "source_stamp_mismatch": 0.38,
          "unusual_stamp_distribution": 0.30, "excessive_length": 0.12}
SOURCE_BONUS = {"user": 0.00, "retrieved_document": 0.18, "tool_output": 0.12}


def score(k):
    p = sum(WEIGHT[o] for o in FEATURES if k["features"][o]) + SOURCE_BONUS[k["source"]]
    r = rng(SEED ^ trace_id(k["no"]))
    return min(max(p + (r() - 0.5) * 0.20, 0.0), 1.6)


def model_filter(k, threshold=0.40):
    return score(k) >= threshold


def measure(records, decision):
    tp = fp = tn = fn = 0
    for k in records:
        flagged, actual = decision(k), harmful(k)
        if flagged and actual:
            tp += 1
        elif flagged and not actual:
            fp += 1
        elif not flagged and actual:
            fn += 1
        else:
            tn += 1
    precision = tp / (tp + fp) if tp + fp else 0.0
    recall = tp / (tp + fn) if tp + fn else 0.0
    return {"tp": tp, "fp": fp, "fn": fn, "tn": tn,
            "precision": round(precision, 4), "recall": round(recall, 4),
            "false_block_rate": round(fp / (fp + tn), 4) if fp + tn else 0.0}


N, SHIFT, THRESHOLD = 600, 300, 0.50                # stream , change point , deployed threshold
BEFORE = [record(i, SEED, SHARE) for i in range(SHIFT)]
print("stream", N, "records , shift after record", SHIFT + 1)
print("calibration period class counts:",
      {s: sum(k["class"] == s for k in BEFORE) for s in CLASSES})
print("\nsetup               blocked  false alarms   escaped harm      recall")
for name, decision in (("undefended", lambda k: False),
                       ("model 0.50", lambda k: model_filter(k, THRESHOLD))):
    m = measure(BEFORE, decision)
    print(f"{name:<16} {m['tp'] + m['fp']:>10} {m['fp']:>13} {m['fn']:>14} {m['recall']:>11.4f}")
print("a third of the blocked records go to a human:",
      len([k for k in BEFORE if model_filter(k, THRESHOLD)][::3]), "/", len(BEFORE), "records")

stream 600 records , shift after record 301
calibration period class counts: {'clean': 180, 'indirect_instruction': 30, 'hidden_field_request': 23, 'privilege_escalation': 13, 'source_spoofing': 30, 'corrupted_input': 24}

setup               blocked  false alarms   escaped harm      recall
undefended                0             0            120      0.0000
model 0.50               95            11             36      0.7000
a third of the blocked records go to a human: 32 / 300 records

The undefended baseline has nothing to monitor: since no record is blocked, the block rate is a constant zero, and since no record goes to review, the false alarm share is undefined. Monitoring is a property of the defense, not of the system. The deployed defense blocks 95 records, 11 of them false alarms, and 36 harmful ones escape; both numbers are written together. The cost of monitoring is visible here too: to be able to measure the false alarm share, 32 of 300 records go to a human.

Three Indicators, Two Degradations

The second block computes the three indicators from the trace record and starts the two degradations in separate streams.

# There are no labels in production: an indicator is computed only from the trace
# record, while labels come from the human decision on records sent to review.
WAVE = [0.41, 0.10, 0.07, 0.07, 0.07, 0.28]         # corrupted input wave
LIGHT = ("unusual_stamp_distribution", "excessive_length")   # alone, each stays below the threshold
WIN1, WIN2, CONSECUTIVE, REVIEW = 120, 240, 10, 3


def trace_record(k, threshold):
    """Fields kept in production. There is NO class label."""
    return {"blocked": model_filter(k, threshold), "source": k["source"],
            "light": any(k["features"][o] for o in LIGHT)}


def build_stream(base, share_after, threshold_after):
    ks = [record(i, base, SHARE if i < SHIFT else share_after) for i in range(N)]
    ts = [THRESHOLD if i < SHIFT else threshold_after for i in range(N)]
    return list(zip(ks, [trace_record(k, t) for k, t in zip(ks, ts)]))


def series(pairs):
    """Three indicators, sliding window. The second needs labels and uses a wider window."""
    g = {"block rate": [], "false alarm share": [], "escaped-harm proxy": []}
    for t in range(len(pairs)):
        if t + 1 >= WIN1:
            p = [z for _, z in pairs[t + 1 - WIN1:t + 1]]
            g["block rate"].append((t, sum(x["blocked"] for x in p) / WIN1))
            passed = [x for x in p if not x["blocked"]]
            g["escaped-harm proxy"].append(
                (t, sum(x["light"] for x in passed) / len(passed) if passed else 0.0))
        if t + 1 >= WIN2:
            blocked = [k for k, z in pairs[t + 1 - WIN2:t + 1] if z["blocked"]]
            reviewed = blocked[::REVIEW]         # a third of the blocked records go to a human
            g["false alarm share"].append(
                (t, sum(not harmful(k) for k in reviewed) / len(reviewed)))
    return g


def band(s):
    v = [x for t, x in s if t < SHIFT]
    return min(v), max(v)


def delay(s, lower, upper):
    """An indicator that leaves the band for CONSECUTIVE records counts as having reported."""
    count, first = 0, None
    for t, x in s:
        if t < SHIFT:
            continue
        if x < lower or x > upper:
            first, count = (t if count == 0 else first), count + 1
            if count >= CONSECUTIVE:
                return first - SHIFT + 1, "up" if x > upper else "down"
        else:
            count = 0
    return None, "-"


def actual_effect(part):
    """Needs labels; in production these numbers are unknown, here they are read because it is fictional."""
    decision = {k["no"]: z["blocked"] for k, z in part}
    return measure([k for k, _ in part], lambda k: decision[k["no"]])


for base in (SEED, 20260219):
    st = {"wave": build_stream(base, WAVE, THRESHOLD), "threshold": build_stream(base, SHARE, 0.30)}
    print(f"\nseed {base}   actual effect: escaped harm / false alarms / recall")
    for b, a in st.items():
        o, y = actual_effect(a[:SHIFT]), actual_effect(a[SHIFT:])
        print(f"  {b:<9} {o['fn']:>3} -> {y['fn']:<4} {o['fp']:>4} -> {y['fp']:<4}"
              f" {o['recall']:>8.4f} -> {y['recall']:.4f}")
    d = {b: series(a) for b, a in st.items()}
    print("  indicator             band            wave             threshold")
    for name in d["wave"]:
        lower, upper = band(d["wave"][name])
        h = []
        for b in ("wave", "threshold"):
            n, y = delay(d[b][name], lower, upper)
            h.append("not reported" if n is None else f"{n} records {y}")
        print(f"  {name:<21} {lower:.4f}-{upper:.4f}   {h[0]:<16} {h[1]}")

seed 20260218   actual effect: escaped harm / false alarms / recall
  wave       36 -> 84     11 -> 2      0.7000 -> 0.5359
  threshold  36 -> 8      11 -> 27     0.7000 -> 0.9355
  indicator             band            wave             threshold
  block rate            0.2667-0.3583   55 records up    29 records up
  false alarm share     0.0800-0.1600   140 records down 45 records up
  escaped-harm proxy    0.0610-0.1928   20 records up    not reported

seed 20260219   actual effect: escaped harm / false alarms / recall
  wave       33 -> 77     12 -> 11     0.7027 -> 0.5792
  threshold  33 -> 8      12 -> 36     0.7027 -> 0.9304
  indicator             band            wave             threshold
  block rate            0.2667-0.3750   54 records up    36 records up
  false alarm share     0.0385-0.2308   not reported     220 records up
  escaped-harm proxy    0.0723-0.1538   10 records up    229 records down

Which Indicator Saw What

The two degradations are mirror images. The wave raises escaped harm from 36 to 84 and lowers recall from 0.7000 to 0.5359; it lowers false alarms from 11 to 2, because the share of clean records shrank. The threshold shift does the opposite: escaped harm falls to 8, false alarms rise from 11 to 27. Each degradation improves one number of the defense while breaking the other. The rule of the course becomes a monitoring rule here: a dashboard that prints a single indicator can show every degradation as an improvement.

The escaped-harm proxy is the indicator that sees the wave earliest: 20 and 10 records. It needs no labels, because among records that pass unblocked it counts the share of two flags that the filter leaves, each alone, below the threshold. The same proxy never reports the threshold shift, or reports it after 229 records and in the wrong direction: when the threshold drops, more records are blocked, the share of flag carriers among the passing records shrinks, and the proxy falls. An alert rule waiting for a rise cannot see this degradation by definition.

Why the proxy works is written in the filter’s own weights. The two distinctive flags of the corrupted input class are unusual_stamp_distribution and excessive_length; their weights are 0.30 and 0.12. Even if both are carried, the total is 0.42, and on a record from a user, where the source bonus is zero, it stays below the 0.50 threshold. So the filter can catch this class only on records that carry a source bonus; as the wave grows, the block rate rises somewhat but escaped harm rises faster. The proxy counts exactly this remainder the filter cannot see, which is why it sees early.

The block rate reports both degradations (55 and 54; 29 and 36 records) but does not say which one it is: in both cases the direction is up. This indicator needs no labels, is the cheapest, and says the least; read alone, it cannot tell the wave from the threshold shift.

The false alarm share is the direct measure of the cost and the least reliable of the three. It reports the threshold shift after 45 and 220 records over the two runs; the difference between the runs is larger than the delay itself, so the delay of this indicator is unmeasurable. For the wave it reports once after 140 records in the down direction and the other time never. The cause is structural: in a window of 240 records the indicator gets labels from only a third of the blocked ones, so one reading rests on about thirty labels. The width of its band confirms it: 0.0800–0.1600 against 0.0385–0.2308, for the same defense.

None of the three indicators catches both degradations, and the one that catches one does not give the direction correctly. Monitoring is a defense, and it has two numbers: the degradation it sees and its cost: thirty-two human reviews, a wait of ten consecutive records, and a delay reaching two hundred records on one degradation.

Summary

• Production monitoring is a property of the defense: on the undefended baseline the block rate is a constant zero, the false alarm share is undefined and there is no indicator to monitor.
• The trace record carries no class label; a label arises only on records sent to review, and 32 of 300 records go to a human.
• The escaped-harm proxy reports the wave after 20 and 10 records and reports the threshold shift never or in the wrong direction; the block rate reports both but does not tell them apart.
• The delay of the label-dependent false alarm share is 45 and 220 records over two runs; since the difference is larger than the delay itself, this indicator does not count as measured.

Next Step

In this lesson an indicator was computed from the trace record and from the review of blocked records. There is one more signal that comes from the system itself and looks free: the user saying they did not like the reply, sending a correction, asking the same question again. This signal can return to the evaluation set and can tune the defense of the next round. The question not asked is this: who can send this signal. A blocked user never sees the reply and has nothing to say; only the voice of those who pass reaches the loop. The next lesson measures the share this signal covers and the direction of the bias.