Intrusion Detection and Prevention

The previous three lessons each treated the boundary as a mechanism that decides once. A flow arrives, its fields are read, it runs through the rule chain, it passes or blocks. The decision is exact because the criterion is exact: a flow either is sourced from dmz or it is not, there is nothing in between.

This lesson looks at a criterion that is not exact. An intrusion detection system evaluates a flow not by its fields but by its behavior, and behavior is reduced to a number. That number is compared against a threshold. What is new is not the threshold itself but the number’s overlap between two classes — and this entire lesson takes place inside that overlap.

A Threshold Is Not a Rule

The rule chain’s criterion is binary, and it distinguishes what it distinguishes flawlessly. The threshold’s criterion is continuous, and the values of the two classes it distinguishes interleave. The numbers of flows intent passes are low, of those it blocks are high; but the two distributions stack on top of each other in a band, and inside that band no threshold gives the right answer.

# taught scale , not executed
# the axis is the score , the numbers come from the measurement block

  flows intent passes    |==========|
  flows intent blocks           |==============|
                                ^^^^  overlap band

  the threshold sits somewhere on this axis :
    shift left  -> false deny grows , false admit shrinks
    shift right -> false admit grows , false deny shrinks
    it cannot step outside the band

This is not a flaw in the mechanism; it is the nature of the criterion. If the criterion distinguished flawlessly, a rule could already be written and there would be no need for a threshold.

What the Two Systems See

The network-based system evaluates the flow itself and sees all forty flows regardless of zone. What it sees is the outside of the flow; its number is noisy.

The host-based system runs on the endpoint the flow lands on. Because it sees the context at the endpoint, its number is sharper, but it sees only the flow landing on its own host; a flow landing on another zone never enters its field, and it has no decision about that flow.

The Perimeter Defense topic of the Cybersecurity curriculum covered these systems by their placement and their signature procedure: where the sensor goes, how a signature is written, how a given record gets matched. That narrative is not repeated here. What is measured is not placement but how the two-way error behaves as a function of the threshold.

The measurement’s assumptions:

• NS22 — The forty flows are taken unchanged from the shared fixture; the oracle is intent itself, and it says whether a flow really matches intent.
• NS23 — The score is derived from the oracle: a flow intent blocks starts from a base of 7, one intent passes from a base of 2, and noise from a separate generator is added on top. This is the fixture because what a detection system sees is the violation’s noisy trace; the overlap band is known because we wrote the trace.
• NS24 — The network-based system’s noise width is 7, the host-based system’s is 6. The host-based system’s sharper reading comes from this width difference; the two scores come from separate generators and are independent of each other.
• NS25 — The host-based system sees only flows whose destination is int or mgmt; for other flows its decision is pass, because it has no decision.
• NS26 — The mechanism’s decision has the form score < threshold; as the threshold rises, fewer flows are blocked.
• NS27 — When the two run together, a flow passes only if both pass it.
• NS28 — The blocked column counts every flow the mechanism blocks, right and wrong alike. In detection mode this column is the alert count; in prevention mode it is the cut flow count.
• NS29 — In a set of forty events, the smallest measurable difference is 1/40 = 0.025.

Measurement

"""Intrusion detection and prevention: how the two-way error shifts as the threshold moves.

Part 1 - network-based mechanism, threshold sweep.
Part 2 - host-based mechanism, and the two mechanisms combined.
"""
SEED = 20260811
ZONES = ("ext", "dmz", "int", "mgmt")
PORTS = (80, 443, 22, 3306, 8080)
HOST_SEEN = ("int", "mgmt")


def generator(seed):
    d = seed % 2147483646 + 1

    def r(n):
        nonlocal d
        d = (d * 48271) % 2147483647
        return d % n
    return r


def flows(count=40, seed=SEED):
    r, out = generator(seed), []
    for i in range(count):
        out.append({"no": i + 1, "source": ZONES[r(4)],
                    "dest": ZONES[r(4)], "port": PORTS[r(5)]})
    return out


def policy(a):
    """Intent: only the DMZ's web entry from outside; nobody from outside to mgmt."""
    if a["source"] == "ext":
        return a["dest"] == "dmz" and a["port"] in (80, 443)
    if a["source"] == "dmz":
        return a["dest"] == "int" and a["port"] == 3306
    if a["source"] == "mgmt":
        return True
    return a["dest"] != "mgmt"


def diff(events, intent, device):
    d = {"correct_pass": 0, "correct_block": 0, "false_admit": 0, "false_deny": 0}
    for o in events:
        n, g = intent(o), device(o)
        if n and g:
            d["correct_pass"] += 1
        elif not n and not g:
            d["correct_block"] += 1
        elif g:
            d["false_admit"] += 1
        else:
            d["false_deny"] += 1
    return d


def score(ak, field, width, seed):
    """The violation's noisy trace: a forbidden flow starts at 7, a permitted one at 2."""
    r = generator(seed)
    for a in ak:
        a[field] = (7 if not policy(a) else 2) + r(width)
    return ak


def network_based(threshold):
    return lambda a: a["net_score"] < threshold


def host_based(threshold):
    """Sees only the flow landing on its own host; passes everything else."""
    return lambda a: a["host_score"] < threshold if a["dest"] in HOST_SEEN else True


def combined(th_net, th_host):
    return lambda a: network_based(th_net)(a) and host_based(th_host)(a)


AK = score(score(flows(), "net_score", 7, SEED + 5),
           "host_score", 6, SEED + 7)

print(f"flow {len(AK)} | intent passes {sum(1 for a in AK if policy(a))} "
      f"| seen by the host-based mechanism "
      f"{sum(1 for a in AK if a['dest'] in HOST_SEEN)}")
print(f"network score: permitted {min(a['net_score'] for a in AK if policy(a))}"
      f"-{max(a['net_score'] for a in AK if policy(a))}, "
      f"forbidden {min(a['net_score'] for a in AK if not policy(a))}"
      f"-{max(a['net_score'] for a in AK if not policy(a))}")
print()
print(f"{'mechanism':<16s} {'threshold':>9s} {'correct':>7s} {'false admit':>11s} "
      f"{'false deny':>11s} {'blocked':>7s}")
trials = ([("network-based", e, network_based(e)) for e in range(6, 13)]
          + [("host-based", e, host_based(e)) for e in range(6, 10)]
          + [("combined", e, combined(e, e)) for e in (8, 9)])
for name, e, f in trials:
    d = diff(AK, policy, f)
    print(f"{name:<16s} {e:9d} {d['correct_pass'] + d['correct_block']:7d} "
          f"{d['false_admit']:11d} {d['false_deny']:11d} "
          f"{sum(1 for a in AK if not f(a)):7d}")

print()
print("network-based, threshold 7 — pointlessly blocked:",
      [(a["no"], a["net_score"]) for a in AK
       if policy(a) and not network_based(7)(a)])
print("network-based, threshold 9 — missed:",
      [(a["no"], a["net_score"]) for a in AK
       if not policy(a) and network_based(9)(a)])
print("forbidden flows the host-based mechanism never sees:",
      sum(1 for a in AK if not policy(a) and a["dest"] not in HOST_SEEN))

flow 40 | intent passes 22 | seen by the host-based mechanism 21
network score: permitted 2-8, forbidden 7-13

mechanism        threshold correct false admit  false deny blocked
network-based            6      27           0          13      31
network-based            7      33           0           7      25
network-based            8      32           4           4      18
network-based            9      35           5           0      13
network-based           10      33           7           0      11
network-based           11      31           9           0       9
network-based           12      26          14           0       4
host-based               6      28           8           4      14
host-based               7      29           8           3      13
host-based               8      30          10           0       8
host-based               9      29          11           0       7
combined                 8      35           1           4      21
combined                 9      38           2           0      16

network-based, threshold 7 — pointlessly blocked: [(7, 8), (11, 7), (20, 7), (21, 8), (24, 8), (30, 8), (38, 7)]
network-based, threshold 9 — missed: [(2, 7), (12, 7), (31, 8), (36, 7), (37, 7)]
forbidden flows the host-based mechanism never sees: 8

The Two Directions of Tightening the Threshold

The network-based system’s scores fall in 2–8 for permitted flows, 7–13 for forbidden ones. The overlap band consists of exactly two values: 7 and 8. The two lists in the lower table confirm this — all seven pointlessly blocked flows, and all five missed flows, score 7 or 8. No flow outside the band produces an error.

When the threshold is pulled to 7, false admit drops to 0. Every forbidden flow is blocked; flawless on paper. The cost shows up in the other column: 7 permitted flows are blocked for nothing. When the threshold opens to 9, false deny drops to 0 this time and false admit climbs to 5. Tightening down to 6 raises false deny to 13; opening up to 12 raises false admit to 14.

The two columns are not zeroed at the same time and cannot be — because the scores of flows inside the overlap band are indistinguishable from each other. This is different from the situation in the previous two lessons: there, the two-way error was the result of a misbuilt set, and the correct set zeroed both. Here there is no correct threshold; there is only a decision about which error is preferred. Zeroing one direction hides the other, and as long as the hidden direction is not measured, the mechanism looks flawless.

The peak of the correct column is at threshold 9, at 35. The point where the errors are equal is threshold 8 (4 and 4), and correct there is 32. Equalizing the two does not maximize total correctness, because the two classes are not equal in size: intent passes 22 flows and blocks 18. Which one is more expensive is not something the measure can say; it is something policy says.

The Floor of Visibility

The host-based system produces a sharper score, but its false admit does not drop below 8 at any threshold. The reason is not in the threshold: 19 of the forty flows are outside its field of view, and 8 of these are flows intent blocks. There is no decision about an unseen flow, and no error — only passage. The one error a threshold cannot repair is the error born from an unseen event.

When the two run together, the table changes. At threshold 9, the network-based system misses 5 flows, the host-based one misses 11; the two together miss 2. False deny is still 0 and correct climbs to 38 — a number neither system reaches alone. The gain does not come from combining as such but from the blind spots not overlapping: the flow the network-based system loses in noise, the host-based system sees sharply, and the zone the host-based system never sees, the network-based system sees. The remaining 2 missed flows are where both are wrong, and adding more mechanisms does not close that; changing the criterion does.

The Difference Between Detection and Prevention

Every row so far is the same in both modes. An intrusion prevention system cuts the flow; a detection system passes the flow and writes a record. The false-admit and false-deny counts come out the same in both, because both give the same decision. What changes is the cost of that decision, and the cost sits in the blocked column.

In prevention mode this column is the cut flow count: at threshold 6, 31 of the forty flows are cut, and 13 of them should not have been. In detection mode the same column is the alert count: 31 records are written, 13 are false alarms, and a person has to read all of them.

This distinction directly affects the threshold decision. Pulling the threshold from 9 to 6 recovers the 5 missed flows; in exchange, the blocked count climbs from 13 to 31 — 2.4 times. In prevention mode this means eighteen extra flows cut; in detection mode, eighteen extra records. The same measure, two different bills.

Surfaces Left Open, and Narrowings

First surface — missed flows. At threshold 9, five forbidden flows pass; with the two mechanisms combined, two pass, and neither has a record behind it. In a set of forty, two flows are a share of 0.050, twice the measurement band; it cannot be dismissed. Narrowing: not lowering the threshold — that grows the other column — but changing the criterion. All of these flows have fields the rule chain already distinguishes exactly; a threshold-based mechanism is placed alongside the rule chain, not in place of it.

Second surface — the unseen zone. Eight of the nineteen flows the host-based system does not see are forbidden. The system makes no decision about these eight flows, and a decision not made produces no record either; to the auditing party, silence and cleanliness look the same. Narrowing: report the field of view itself — write down, in flow counts, which zones are monitored and which are not. A measurement whose scope is unknown cannot be read.

Third surface — the alert pile. Tightening the threshold raises alerts from 13 to 31, and thirteen of the eighteen extra records are false alarms. An unread alert does the same job as an unwritten one; a tight threshold can give back its own gain this way. Narrowing: choose the threshold not as a single number but separately by zone — tight for flows landing in the management zone, loose for flows talking to the outside. The measurement directly supports this: the difference between the two systems already varied by zone.

Summary

• Because a threshold-based mechanism’s criterion is continuous, the two classes’ values overlap in a band; in this measurement the band is 7–8, and the entire two-way error is born inside it.
• In the network-based system, at threshold 7 false admit is 0 and false deny is 7; at threshold 9 false deny is 0 and false admit is 5. The two columns cannot be zeroed at once.
• The threshold that equalizes the errors (4 and 4 at 8) does not maximize total correctness; the classes are not equal in size, and which error is more expensive is policy’s decision, not the measure’s.
• The host-based system’s false admit never drops below 8 at any threshold, because it never sees 19 of the forty flows; the one error a threshold cannot repair is born from an unseen event.
• Combined, the two give 38 correct at threshold 9; the gain comes from the blind spots not overlapping rather than from combining as such, and the remaining 2 missed flows are where both are wrong.
• Detection and prevention produce the same numbers but charge different costs: the blocked column is cut flows in prevention, alerts that must be read in detection — pulling the threshold from 9 to 6 grows it 2.4 times.

Next Step

All four lessons in this topic shared a single assumption: the outside is outside. The rule chain, the zone graph, and the threshold all decided by looking at where a flow comes from, and the name ext was synonymous with “untrusted” in all three. But what if an outside endpoint has to be let in? Someone working from home, a whole segment at another site, or two networks talking over a path we do not control — none of these step outside ext, yet all of them are expected to behave like int. The next lesson builds the mechanism that lets this endpoint in, and asks the same two columns again: how are the destinations to be let in chosen, and what happens to the ones that are not.