Skip to content
academia.sh

Lesson 05 / 19

Instrumentation Standards

Separating telemetry collection from application code: meeting the same interface with two collector implementations, measuring how many files get touched when the implementation changes in a bound layout versus an interfaced one, and counting how many application files the collector's name appears in.

Contents

All three signals are produced in code, but where each one gets written still lives inside the service code: the logger opens its own file, the metrics are served from an endpoint, the spans fall into another file. This tie looks harmless at first. Its harm only becomes visible when the collection method changes, and the moment it does, it turns into a measurable number: how many files had to be touched.

Vendor-neutral instrumentation means the application code does not know where or how telemetry gets sent. What this lesson builds is not a name, it is an interface: the application code only says “this event happened”; whether the event gets written to a file, counted, or sent somewhere else is someone else’s job. The measure comes down to a single question — how many files change when the implementation changes.

Two Implementations of the Same Interface

The interface is made of two operations: collect takes an event, summary returns the state of what has been collected. The two implementations are direct opposites in behavior. The first writes every event to disk as its own line; the second never stores the individual event at all, it only counts by type.

// collectors.mjs — two implementations of the same interface; both implement collect and summary
import { appendFileSync, writeFileSync } from "node:fs";
export const fileCollector = (file) => {        // writes every event as its own line
  writeFileSync(file, "");
  return { collect: (o) => appendFileSync(file, JSON.stringify(o) + "\n"),
    summary: () => `events in ${file}` };
};
export const summaryCollector = () => {         // counts events by type, never writes a single event
  const count = new Map();
  return { collect: (o) => count.set(o.type, (count.get(o.type) ?? 0) + 1),
    summary: () => [...count].map(([t, n]) => `${t}=${n}`).join(" ") };
};

The face the application code sees is different. The service code never calls collect; it calls the three operations that correspond to the three signals, and it does not know where the event goes. The binding happens in a single file, in a single line.

// measurement.mjs — the single face the application code sees; it does not know which collector is bound
import { collector } from "./binding.mjs";
export const log = (fields) => collector.collect({ type: "log", ...fields });
export const metric = (name, value) => collector.collect({ type: "metric", name, value });
export const span = (name, duration) => collector.collect({ type: "trace", name, duration });
export const summary = () => collector.summary();
// binding.mjs — the single binding point; the implementation is chosen only in this line
import { fileCollector } from "./collectors.mjs";
export const collector = fileCollector("event-interfaced.txt");        // BINDING

TL5 — the five services are separate deployment units, and therefore separate source files. Rationale: what is being measured is a change’s blast radius; in a setup derived from a single source file with a role argument, every change lands in one file by definition and the measure loses its meaning. All five files are generated from the same template, and the only difference between them is the role name and the port.

Two Layouts

Both layouts do the same job. In the bound layout, the service file calls the collector by name and formats the events itself. In the interfaced layout, no collector name appears in the service file at all.

// template-bound.mjs — five service files are generated by substituting ROLE and PORT
import { createServer } from "node:http";
import { fileCollector } from "./collectors.mjs";
const collector = fileCollector("event-bound.txt");                  // BINDING
let request = 0;
createServer((req, res) => {
  if (req.url === "/summary") { res.end(collector.summary()); return; }
  const t0 = performance.now(); request += 1;
  collector.collect({ type: "log", service: "ROLE", event: "loan request processed" });
  collector.collect({ type: "metric", name: "ROLE_request", value: request });
  collector.collect({ type: "trace", name: "ROLE/work", duration: Math.round(performance.now() - t0) });
  res.end("done");
}).listen(PORT);
// template-interfaced.mjs — five service files are generated by substituting ROLE and PORT
import { createServer } from "node:http";
import { log, metric, span, summary } from "./measurement.mjs";
let request = 0;
createServer((req, res) => {
  if (req.url === "/summary") { res.end(summary()); return; }
  const t0 = performance.now(); request += 1;
  log({ service: "ROLE", event: "loan request processed" });
  metric("ROLE_request", request);
  span("ROLE/work", Math.round(performance.now() - t0));
  res.end("done");
}).listen(PORT);

Measurement

The measurement actually performs the switch. First both layouts run with the file collector, then the switchOver function changes the implementation in the relevant files, then both run again. The number of files touched is not estimated: a directory copy taken before the switch is compared against the state after.

SERVICES="loan membership catalog fee notification"
rm -rf bound interfaced bound-before interfaced-before; mkdir -p bound interfaced
cp collectors.mjs bound/ ; cp collectors.mjs measurement.mjs binding.mjs interfaced/
i=0
for s in $SERVICES; do
  sed -e "s/ROLE/$s/g" -e "s/PORT/$((9501 + i))/" template-bound.mjs > bound/$s.mjs
  sed -e "s/ROLE/$s/g" -e "s/PORT/$((9511 + i))/" template-interfaced.mjs > interfaced/$s.mjs
  i=$((i + 1))
done
switchOver() {   # replaces the file collector with the summary collector
  perl -pi -e 's/fileCollector/summaryCollector/g; s/\("event-[a-z]+\.txt"\)/()/' "$1"
}
runTrial() {   # brings up the loan service in $1, sends 5 requests, prints the summary
  (cd "$1" && node loan.mjs) & PID=$!
  sleep 0.7
  for n in 1 2 3 4 5; do curl -s -o /dev/null "http://127.0.0.1:$2/work"; done
  printf '%-10s %s\n' "$1" "$(curl -s http://127.0.0.1:$2/summary)"
  kill $PID 2>/dev/null; wait $PID 2>/dev/null
}

echo "before the switch:"
runTrial bound 9501
runTrial interfaced 9511
echo "first event record: $(head -1 bound/event-bound.txt)"

cp -r bound bound-before; cp -r interfaced interfaced-before
for s in $SERVICES; do switchOver bound/$s.mjs; done
switchOver interfaced/binding.mjs

echo
echo "after the switch:"
runTrial bound 9501
runTrial interfaced 9511

echo
printf '%-10s%34s%28s\n' layout "app files naming collector" "files touched by switch"
for y in bound interfaced; do
  named=$(grep -l "Collector" $y/*.mjs | grep -v collectors | grep -v binding | wc -l | tr -d ' ')
  touched=$(diff -rq $y-before $y | wc -l | tr -d ' ')
  printf '%-10s%34s%28s\n' "$y" "$named" "$touched"
done
echo "files touched (interfaced): $(diff -rq interfaced-before interfaced | awk '{print $4}' | paste -sd' ' -)"
before the switch:
bound      events in event-bound.txt
interfaced events in event-interfaced.txt
first event record: {"type":"log","service":"loan","event":"loan request processed"}

after the switch:
bound      log=5 metric=5 trace=5
interfaced log=5 metric=5 trace=5

layout            app files naming collector     files touched by switch
bound                                      5                           5
interfaced                                 0                           1
files touched (interfaced): interfaced/binding.mjs

Five Files versus One

Both layouts give the same summary after the switch: log=5 metric=5 trace=5. The behavior is equal, the cost is not. Five files changed in the bound layout, one in the interfaced layout. The name of that one changed file is printed too: binding.mjs. None of the five service files was touched — their state before and after the switch is byte for byte the same.

The middle column gives the reason for the gap. In the bound layout, the collector’s name appears in five application files; in the interfaced layout, in zero. The number of files touched is the direct consequence of that: wherever a name is written, changing that name requires touching just as many places. What the interface does is confine the name to a single place; it does nothing else.

How the number grows is also part of the measure. Five services today means five files; as the system grows, this count grows with the number of services, and every increase makes the same switch a notch more expensive. In the interfaced layout the count is independent of the service count: with fifty services, the binding point is still one. This is the same shape as the previous lessons’ metric field count being independent of the request count — the cost is tied to something fixed, not to something that grows.

The interface’s cost is just as plain, and should not be underestimated. Two extra files were written (measurement.mjs and binding.mjs), a layer of indirection now sits between the application code and the collector, and the interface itself became a decision: collect only takes an object with a type field, so if a collector needs a piece of information that does not fit this shape, the interface has to change. And an interface change lands right back in the application files — the independence gained is only as good as the interface’s design.

There is one more boundary. The switch was made here by changing a single line, because what was measured was the change’s blast radius. Choosing the collector at runtime, writing to more than one collector at once, or what happens to events while a collector is down are outside this lesson’s measure; none of them change the file count.

Summary

  • The interface is just two operations: collect takes an event, summary returns the state; the two implementations met the same events by one writing to disk, the other only counting.
  • After the switch, both layouts gave the same summary (log=5 metric=5 trace=5); the difference was not in behavior, it was in the cost of the change.
  • The collector’s name appears in 5 application files in the bound layout, and in 0 in the interfaced layout.
  • The same switch touched 5 files in the bound layout and 1 in the interfaced layout; the five service files stayed byte for byte the same before and after.
  • In the bound layout the cost grows with the number of services; in the interfaced layout it is fixed; in exchange there is a two-file indirection and the interface itself becoming a decision.

Next Step

The three signals are now produced in code, none stands in for another, and none is bound to a collector. But they still live in three separate places: when a question is asked, the log is read separately, the metric separately, the trace separately, and the link between them is rebuilt by hand every time. The second lesson produced the correlation id, the fourth lesson carried the trace context across the boundary; both name the same request, but no single record holds all three together. The next lesson does that, and closes the topic: the three signals are combined into a single event, and the same question is compared for how many steps and how many signals it takes to answer, combined and uncombined.

To keep your progress and take notes, Log in

My notes

Log in to take notes.

Start typing to search.

↑↓ Esc navigate · open · close