Event Bus

The previous lesson counted the limits of the function and wrote down where it cannot be used. One question remains: how do functions connect to each other? A function does nothing on its own; something has to wake it. If the waking party is a calling function, the caller has to know the target — its address, its version, how many there are. As the number of producers grows, this knowledge multiplies separately inside every producer.

The event bus reverses this link. The producer writes the event and does not know where it will go; the target is chosen by a rule. The decision has been taken from the producer and handed to the event bus. This lesson counts what comes back in exchange for the handed-over decision.

The Handed-Over Decision: Who Wakes Up

In a direct call the target list sits inside the code: who calls whom can be read from the call stack. In rule-based routing the target list sits outside the code — a rule is a data item. The constraint that comes in exchange follows from this: routing no longer shows up in the source-repository diff, does not sit in the function’s call graph, and where an event goes can only be known by reading the rule set.

The first payback of the handed-over decision shows up in the link count. The measurement network has six components that produce events and nine targets that act on events. In a direct call, every producer has to know the targets it cares about: 54 links in the worst case, each one an address and a version held on both sides. On the event bus the producer writes to a single place and the targets read from a single place — 6 + 9 = 15 records and a rule set between them. The link count drops from a product to a sum. The constraint comes from the same place: all 54 links were visible in the code; none of the items in the rule set are.

The second constraint sits in the matching itself. If an event matches no rule, the event bus’s default behavior is to drop it; it produces no error, wakes no one, leaves no trace behind. The third is the reverse: if an event matches multiple rules, each rule wakes its own target and the same event is processed more than once. Both depend on how wide the rule set is, and both are a number.

The measurement runs on a fictional setup that carries through the whole lesson: the regional measurement network — a system where meters send in readings, readings are validated, invoices are produced, and field crews are assigned work. Both the network and the event names are fictional.

• ED1. The stream is 2,000 events; there are nine event types and the share of each type is fixed.
• ED2. Every event has three fields: type, region (four regions), priority (1–3).
• ED3. A rule is a dot-separated type pattern with optional field conditions; a * in the pattern leaves everything after that segment unrestricted.
• ED4. If an event matches the same target through two rules, it counts as a single delivery; delivery is counted per target.
• ED5. Wherever randomness is needed, the generator is written by hand and the seed is visible.
• ED6. The model builds only the routing decision; transport latency and ordering are not measured.

As the Rule Set Widens

The code below is not an event bus; it is a countable model of the routing decision. The same stream is run through three rule sets: a four-rule narrow set, a seven-rule medium set, and a ten-rule wide set that includes a catch-all rule.

// rule-engine.mjs — event bus model: rules route events carrying a pattern to targets
// Model: not a real event bus, but a countable reconstruction of the routing decision.

const SEED = 20260415;                        // generator seed: run is reproducible
let state = SEED;
const random = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648;

const TYPES = [["meter.reading.received", 46], ["meter.reading.missing", 9],
  ["meter.validation.failed", 7], ["invoice.draft.created", 12], ["invoice.finalized", 6],
  ["device.battery.low", 8], ["device.connection.lost", 5], ["field.crew.assigned", 4],
  ["field.fault.closed", 3]];
const REGIONS = ["north", "east", "west", "central"];

function events(n) {
  const list = [];
  for (let i = 0; i < n; i++) {
    let p = random() * 100, type = TYPES[0][0];
    for (const [name, weight] of TYPES) { if ((p -= weight) < 0) { type = name; break; } }
    list.push({ type, region: REGIONS[Math.floor(random() * 4)],
                 priority: 1 + Math.floor(random() * 3) });
  }
  return list;
}

// Pattern: dot-separated segments; "*" leaves everything after that segment unrestricted.
function matches(pattern, type) {
  const p = pattern.split("."), t = type.split(".");
  for (let i = 0; i < p.length; i++) {
    if (p[i] === "*") return true;
    if (p[i] !== t[i]) return false;
  }
  return p.length === t.length;
}
const fires = (rule, event) => matches(rule.pattern, event.type) &&
  (rule.region === undefined || rule.region === event.region) &&
  (rule.minPriority === undefined || event.priority >= rule.minPriority);

const R = {
  narrow: [
    { name: "R1", pattern: "meter.reading.received", target: "validation" },
    { name: "R2", pattern: "invoice.*", target: "billing" },
    { name: "R3", pattern: "device.battery.low", target: "maintenance" },
    { name: "R4", pattern: "field.*", target: "field" }],
};
R.medium = [...R.narrow,
  { name: "R5", pattern: "meter.*", target: "archive" },
  { name: "R6", pattern: "device.*", target: "monitoring" },
  { name: "R7", pattern: "meter.validation.failed", region: "north", target: "alert" }];
R.wide = [...R.medium,
  { name: "R8", pattern: "*", target: "audit" },
  { name: "R9", pattern: "invoice.finalized", target: "accounting" },
  { name: "R10", pattern: "meter.reading.received", minPriority: 3, target: "priority" }];

const STREAM = events(2000);

function scan(rules) {
  let unmatched = 0, single = 0, multi = 0, delivered = 0;
  for (const e of STREAM) {
    const h = new Set(rules.filter((r) => fires(r, e)).map((r) => r.target));
    delivered += h.size;
    if (h.size === 0) unmatched++; else if (h.size === 1) single++; else multi++;
  }
  return { unmatched, single, multi, delivered };
}

console.log("set       rules   unmatched  single target  multi target  total delivered");
for (const name of ["narrow", "medium", "wide"]) {
  const s = scan(R[name]);
  console.log(name.padEnd(8) + String(R[name].length).padStart(7) +
    String(s.unmatched).padStart(12) + String(s.single).padStart(15) +
    String(s.multi).padStart(14) + String(s.delivered).padStart(17));
}
console.log("event count: " + STREAM.length + ", seed: " + SEED);

node rule-engine.mjs

set       rules   unmatched  single target  multi target  total delivered
narrow        4         433           1567             0             1567
medium        7           0            881          1119             3119
wide         10           0              0          2000             5551
event count: 2000, seed: 20260415

The first row shows silent dropping. The four-rule set delivered 1,567 of the two thousand events to one target; 433 events matched no rule and were dropped. These events are the missing-reading, validation-failure, and lost-connection notices — two of the three types that most need attention. Dropping is not an error; for the event bus, “no match” is an ordinary outcome. There is no symptom on the producer’s side either: the event was written, accepted, and the job is done.

The second row is the other end of the constraint. Once three rules were added, unmatched fell to zero, but 1,119 events went to more than one target and total delivered climbed from 1,567 to 3,119. The same reading now wakes both the validation and the archive target. If the two targets do separate work, this is correct; if both of them also write a billing item, the same item gets written twice. The rule set does not know this difference.

The path between these two rows is the usual road taken while a rule set widens: an unmatched event is noticed, the pattern is broadened, the unmatched count falls. That is not the only thing that falls — every broadening also pulls events that previously reached one target into the new rule. With four rules no event was multi-target; after three more rules, 56 percent of the stream is multi-target. Broadening’s effect is not confined to the targeted type; the pattern also covers types it was never meant to target.

In the third row the catch-all rule keeps unmatched at zero, but now every event is multi-target: 2,000 events turned into 5,551 deliveries, 2.78 wake-ups per event. The price of escaping silent drops is triple delivery, and delivery, on the function’s side, means a call.

The Filter’s Location

Once the routing decision is handed over, a second decision can be handed over too: filtering. The validation target actually only cares about high-priority readings from the north region. This condition can be written either into the rule or into the target’s first lines. Both do the same job; the number of events carried is not the same.

// filter-location.mjs — same stream, filter in the rule and in the target; then a hidden rule is added
const SEED = 20260415;                        // same seed as the first run, same stream
let state = SEED;
const random = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648;
const TYPES = [["meter.reading.received", 46], ["meter.reading.missing", 9],
  ["meter.validation.failed", 7], ["invoice.draft.created", 12], ["invoice.finalized", 6],
  ["device.battery.low", 8], ["device.connection.lost", 5], ["field.crew.assigned", 4],
  ["field.fault.closed", 3]];
const REGIONS = ["north", "east", "west", "central"];
const STREAM = [];
for (let i = 0; i < 2000; i++) {
  let p = random() * 100, type = TYPES[0][0];
  for (const [name, weight] of TYPES) { if ((p -= weight) < 0) { type = name; break; } }
  STREAM.push({ type, region: REGIONS[Math.floor(random() * 4)],
              priority: 1 + Math.floor(random() * 3) });
}

const relevant = (e) => e.region === "north" && e.priority >= 2;   // what the target actually wants
const readings = STREAM.filter((e) => e.type === "meter.reading.received");
const UNIT = 1000;                             // fictional unit: every 1,000 deliveries is 1 transport unit

console.log("filter location".padEnd(19) + "carried".padStart(9) +
  "wasted wake-up".padStart(17) + "transport unit".padStart(16));
for (const [place, carried] of [["filter in rule", readings.filter(relevant)], ["filter in target", readings]]) {
  const wasted = carried.length - carried.filter(relevant).length;
  console.log(place.padEnd(19) + String(carried.length).padStart(9) + String(wasted).padStart(17) +
    (carried.length / UNIT).toFixed(2).padStart(16));
}

// Hidden rule: without touching the existing target's numbers, sends the same events to a second target.
const before = readings.filter(relevant).length;
const shadow = STREAM.filter((e) => e.type.startsWith("meter.reading.")).length;
console.log("\nbefore adding a rule    validation target: " + before + ", total delivered: " + before);
console.log("after adding R11        validation target: " + before + ", total delivered: " +
  (before + shadow) + ", error: 0");

node filter-location.mjs

filter location      carried   wasted wake-up  transport unit
filter in rule           145                0            0.14
filter in target         926              781            0.93

before adding a rule    validation target: 145, total delivered: 145
after adding R11        validation target: 145, total delivered: 1257, error: 0

Filtering in the rule carried 145 events, and all of them were processed. Filtering in the target carried 926 events, and 781 of them were discarded on the first line: the function woke up, read the event, saw it was irrelevant, and returned. The same job’s visible cost on the transport side is 0.14 units against 0.93 units — 6.4 times as much. The unit is fictional; the ratio follows from the rule set.

The cost of a wasted wake-up is not only transport. The 781 calls take a slot from the concurrency pool, carry a chance of cold start, and enter as noise into every metric on the function’s side: call count, error rate, average duration. The cost of moving the filter into the rule, on the other hand, is that the condition moves out of code review and into configuration — the subject of the next section.

The source has a limit on filtering, and that limit is the real constraint left outside the measurement: a filter can only look at the event’s own fields. If the validation target wanted “only readings from subscriptions whose contract is suspended,” this condition could not be written with any of the three fields carried in the event — the subscription’s status is held somewhere else. In that case the only way is to add the field the condition cares about to the event; the cost is that the producer now has to know the consumer’s need, meaning one end of the handed-over link comes back. If the field cannot be added, the 781 wasted wake-ups are unavoidable, and the choice is not where the filter sits but which cost gets accepted.

The Rule’s Invisibility Is a Surface

The last two lines of the output show a single thing. A new rule was added to the rule set that does not touch the validation target: R11, with the pattern meter.reading.*, sends events to a second target as well. The validation target’s delivery count stayed at 145, its error count stayed at zero, no metric moved. Total delivered, however, went from 145 to 1,257 — every reading event in the stream now also goes to a second place.

This is the most expensive constraint of handing over routing. Because a rule is a data item:

• It does not appear in the source-repository diff; not one line of function code changes.
• It does not disturb the metrics of existing targets, so it crosses no alert threshold.
• The content of events reaching the new target is independent of the producer’s intent: the producer wrote the reading for validation, and the rule also sent it somewhere else.

The same invisibility works in the other direction too. When a rule is deleted or its pattern narrowed, the target stops receiving traffic; the function does not error, because it is never called. The call count drops to zero, and zero reads not like a failure symptom but like “there is no work at this hour.” Between this and the 433 silent drops from the first measurement there is no observable difference: in both cases the event was written, accepted, and went nowhere. The only way to see the difference is to count the number of events produced and the number of events delivered separately and watch the gap between them.

There are three ways to work around the constraint, and all three demand a countable cost. The first is to keep the rule set in the source repository and review changes as code; its cost is that every rule change now enters the release pipeline, and a process that takes minutes gets spread over hours. The second is to keep a per-target delivery counter; the jump in total delivered from 145 to 1,257 when R11 was added is only visible this way, and its cost is ten separate counters in a ten-rule set plus a set of thresholds that reads them. The third is to ban wide patterns — the catch-all rule is removed, * is only allowed in the last segment. The cost of this is the first row of the first measurement: as the pattern narrows, the unmatched-event count climbs again and 433 events are silently dropped.

Choosing among the three means weighing the cost of a dropped event against the cost of an extra delivery. In the measurement network a dropped validation error shows up as wrong consumption on an invoice; an extra archive delivery only costs storage. In this network the rule set is wide, the filter sits in the rule, and the counter is per target.

Summary

• The event bus takes the routing decision away from the producer: the producer does not know the target, the rule chooses it. In exchange, a rule becomes a data item and steps outside code review.
• On the same 2,000-event stream, the four-rule set delivered 433 events to no target and did so without producing an error; silent dropping is the event bus’s default behavior.
• Once the rule count rose to seven, unmatched fell to zero but 1,119 events became multi-target; with ten rules total delivered reached 5,551, 2.78 wake-ups per event.
• Filtering in the rule carried 145 events, filtering in the target carried 926, and 781 of those woke the function for nothing; the transport-side difference is 0.14 units against 0.93 fictional units.
• A new rule left the existing target’s delivery count at 145 while pushing total delivered to 1,257, and it produced no error; this is why a rule’s invisibility is a surface.
• Silent dropping and a deleted rule produce the same symptom: in both cases the call count is zero. The only way to tell them apart is to count events produced and events delivered separately.

Next Step

The routing decision has been made: which event wakes whom is now written into the rule set. But this measurement never asked one thing — what happens when the target wakes up and is not standing? The model counted every delivery as successful; in reality the function can be throttled, can error, can hit its concurrency limit. Something is needed to hold the event, retry it, and keep it waiting until the consumer is ready. The next lesson measures that layer as a service: which decisions leave your hands when queue and topic services are taken over, at what threshold visibility timeout produces double processing, and at what threshold it produces delay.