Log Integrity

The previous lesson turned on the process creation log and showed that parent-process context brings 1,702 flags down to 40. All three lessons in this topic carried the same silent assumption: the record sits there, and it is correct. But every record counted here is produced on the local disk of the very machine the item is running on, and this course’s opening assumption granted code the ability to run on the endpoint once. The record sits exactly where that code can reach it.

This lesson tests that assumption. A hash chain shows that the record is internally consistent: each record’s digest covers its sequence number, the previous digest, and the body, so removing or changing a record in the middle breaks the chain. The question to ask is what this works against. The measurement is a model, the regional measurement network is fictional, and the tampering’s method is not written down, only its result is modeled. Operating on records in a system that belongs to someone else — deleting, changing, moving — without written authorization and a defined scope is both illegal and unprofessional.

• SL16. One day’s log on one endpoint is 1,200 records. Each record carries a sequence number, a body, and a digest; the digest is computed from the sequence number, the previous digest, and the body using node:crypto. The structure is the same as the chained digest in the Backend Production course; the question being measured is different.
• SL17. The tampering instant is second 61,550, and the tampered window starts at second 54,000. 11 records are deleted from the window, and 2 records’ bodies change.
• SL18. Four tampering forms are separated: A deletion, without touching the chain; B change, refreshing only that record’s digest; C deletion, recomputing the chain; D deletion, reissuing the sequence numbers too.
• SL19. The central collector keeps a copy of the record, and the comparison is made through the record’s unchanging item identity. A record that has not reached the center cannot be compared.
• SL20. The fleet has four groups, and field laptops are offline 23,400 seconds a day. The overhead per transfer is 8 KiB; all figures are fictional.

The Four Forms the Chain Sees

// chain.mjs — the log chain produced on the endpoint, four tampering forms, and local verification.
// THIS IS A MODEL; the tampering METHOD is not written, only its result is modeled.
import { createHash } from "node:crypto";

export const SEED = 20260515, N = 1200, T = 61550;   // tampering instant (seconds)
const generator = (t) => { let s = t >>> 0;
  return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; };
const r = generator(SEED);
const SOURCE = ["process-creation", "authentication", "system-event", "scheduler"];

export const digest = (seq, prev, body) =>
  createHash("sha256").update(`${seq}|${prev}|${body}`).digest("hex").slice(0, 16);

// Chain: each record's digest is computed from its sequence number, the previous digest, and the body.
export function chain(records) {
  let prev = "start";
  return records.map((k) => { prev = digest(k.seq, prev, k.body); return { ...k, digest: prev }; });
}
const times = Array.from({ length: N }, () => Math.floor(r() * 86400)).sort((a, b) => a - b);
export const LOG = chain(times.map((t, i) => ({
  seq: i + 1, t,
  body: JSON.stringify({ t, source: SOURCE[Math.floor(r() * 4)], item: 100 + i }),
})));

// Tampering window: the last activity window. Part of the records in this window are deleted,
// the last two records are changed.
const tamperWindow = LOG.filter((k) => k.t >= 54000 && k.t <= T);
export const DELETED = tamperWindow.filter((_, i) => i % 9 === 0).slice(0, 12).map((k) => k.seq);
export const CHANGED = tamperWindow.filter((k) => !DELETED.includes(k.seq)).slice(-2)
  .map((k) => k.seq);
// The change shifts the record's timestamp; the item identity does not change.
const corrupt = (k) => JSON.stringify({ ...JSON.parse(k.body), t: JSON.parse(k.body).t + 900 });

// A: deleted, chain untouched. B: changed, only that record's digest refreshed.
// C: deleted, chain recomputed. D: deleted, sequence reissued too.
export const FORM = {
  A: () => LOG.filter((k) => !DELETED.includes(k.seq)),
  B: () => LOG.map((k) => (CHANGED.includes(k.seq)
    ? { ...k, body: corrupt(k), digest: digest(k.seq, LOG[k.seq - 2].digest, corrupt(k)) } : k)),
  C: () => chain(LOG.filter((k) => !DELETED.includes(k.seq))
    .map((k) => (CHANGED.includes(k.seq) ? { ...k, body: corrupt(k) } : k))),
  D: () => chain(LOG.filter((k) => !DELETED.includes(k.seq))
    .map((k, i) => ({ ...k, seq: i + 1,
      body: CHANGED.includes(k.seq) ? corrupt(k) : k.body }))),
};

// Local verification: the chain is run from the start, the sequence of the first mismatch is returned (0 if none).
export function verify(records) {
  let prev = "start";
  for (const k of records) {
    prev = digest(k.seq, prev, k.body);
    if (prev !== k.digest) return k.seq;
  }
  return 0;
}
// Sequence gaps: the only local trace deleted records can leave.
export const gap = (records) => records.reduce((a, k, i) =>
  (i && k.seq !== records[i - 1].seq + 1 ? a + k.seq - records[i - 1].seq - 1 : a), 0);

if (import.meta.url === `file://${process.argv[1]}`) {
  const row = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j])
    : String(v).padEnd(B[0]))).join("");
  const line = row([42, 8, 16, 14, 15]);
  console.log(`model (seed ${SEED}): ${N}-record log chain, tampering instant second ${T}`);
  console.log(`tampered window: ${DELETED.length} deleted, ${CHANGED.length} changed records ` +
    `(seq ${DELETED[0]}-${DELETED.at(-1)} and ${CHANGED.join(", ")})`);
  console.log("\n" + line(["tampering form", "records", "first mismatch", "seq gap",
    "local result"]));
  for (const [name, note] of [["A", "record deleted, chain untouched"],
    ["B", "record changed, digest refreshed"],
    ["C", "record deleted, chain recomputed"],
    ["D", "deleted, sequence and chain reissued"]]) {
    const l = FORM[name]();
    const u = verify(l), b = gap(l);
    console.log(line([`${name}: ${note}`, l.length, u || "-", b || "-",
      u || b ? "tampered" : "clean"]));
  }
}

model (seed 20260515): 1200-record log chain, tampering instant second 61550
tampered window: 11 deleted, 2 changed records (seq 743-833 and 838, 839)

tampering form                             records  first mismatch       seq gap   local result
A: record deleted, chain untouched            1189             744            11       tampered
B: record changed, digest refreshed           1200             839             -       tampered
C: record deleted, chain recomputed           1189               -            11       tampered
D: deleted, sequence and chain reissued       1189               -             -          clean

The four rows build a staircase. In the first form the records have been deleted and the chain has not been touched; two independent signals come out at once — verification stops at sequence 744, and 11 sequence gaps are counted. In the second, two adjacent records’ bodies have changed and only those two records’ digests have been recomputed; the mismatch shows up not at the first changed record but at the one after it — at sequence 839 — because that record’s digest depends on the previous record’s old digest. This is exactly what the chain buys you: making one record consistent requires rewriting every record after it too.

In the third form the whole chain has been recomputed, and verification finds no mismatch at all. One trace remains: the 11-sequence gap. In the fourth form the sequence numbers have been reissued too, and that trace disappears as well — local verification says clean. Eleven records have been deleted, two records’ timestamps have been shifted, and there is nothing on the machine that shows it.

The result comes down to one sentence: a chain produced on the same machine and verified on the same machine is not proof of integrity against whoever holds that machine, only proof of consistency. Its real contribution is that it makes partial tampering impossible: whoever tampers with it will either not touch it at all, or will have to reproduce the entire record.

The Central Copy and the Transfer Window

// center.mjs — comparison with the central copy and the fleet cost of the transfer window.
import { LOG, FORM, DELETED, CHANGED, T } from "./chain.mjs";

const local = FORM.D();                              // the set local verification calls clean
const item = (k) => JSON.parse(k.body).item;          // the record's unchanging identity
const INTERVAL = [["once a day", 86400], ["4 hours", 14400], ["hourly", 3600],
  ["5 minutes", 300], ["5 seconds", 5]];

const row = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j])
  : String(v).padEnd(B[0]))).join("");
const line1 = row([13, 11, 13, 20, 19, 15]);
const line2 = row([20, 10, 11, 14, 16, 17, 13]);

console.log(`tampering instant second ${T}; ${DELETED.length} deleted and ${CHANGED.length} ` +
  `changed records exist, the local chain shows none of them`);
console.log(line1(["interval", "at center", "in window", "deletions detected",
  "changes detected", "invisible"]));
for (const [name, w] of INTERVAL) {
  const lastTransfer = Math.floor(T / w) * w;
  const center = LOG.filter((k) => k.t <= lastTransfer);
  const localItem = new Map(local.map((k) => [item(k), k.body]));
  const del = center.filter((k) => !localItem.has(item(k))).length;
  const chg = center.filter((k) => localItem.has(item(k)) &&
    localItem.get(item(k)) !== k.body).length;
  console.log(line1([name, center.length, LOG.length - center.length,
    `${del}/${DELETED.length}`, `${chg}/${CHANGED.length}`,
    DELETED.length + CHANGED.length - del - chg]));
}

// Fleet: the cost of shortening the transfer interval and the window that cannot be closed.
const GROUP = [["field laptop", 84, 41000, 23400], ["branch endpoint", 46, 39000, 3600],
  ["meter collector", 120, 44000, 0], ["central processing", 18, 52000, 0]];
const OVERHEAD = 8192;                                // overhead per transfer (bytes)
console.log("\n" + line2(["transfer interval", "transfers", "fleet/day", "overhead MiB",
  "field window", "records at risk", "unclosed"]));
for (const [name, w] of INTERVAL) {
  const transfers = Math.round(86400 / w);
  const fleet = GROUP.reduce((a, g) => a + g[1] * transfers, 0);
  const risk = GROUP.reduce((a, g) => a + g[1] * g[2] * Math.max(w, g[3]) / 86400, 0);
  const unclosed = GROUP.filter((g) => g[3] > w).reduce((a, g) => a + g[1], 0);
  console.log(line2([name, transfers, fleet.toLocaleString("en-US"), Math.round(fleet * OVERHEAD / 1048576),
    `${Math.max(w, GROUP[0][3])} s`, Math.round(risk).toLocaleString("en-US"), unclosed]));
}
console.log(`\nshortening the interval from once a day to 5 minutes multiplies the transfer ` +
  `count by 288; on field laptops the effective window never drops below ${GROUP[0][3]} seconds`);

tampering instant second 61550; 11 deleted and 2 changed records exist, the local chain shows none of them
interval       at center    in window  deletions detected   changes detected      invisible
once a day             0         1200                0/11                0/2             13
4 hours              789          411                6/11                0/2              7
hourly               834          366               11/11                0/2              2
5 minutes            837          363               11/11                0/2              2
5 seconds            839          361               11/11                2/2              0

transfer interval    transfers  fleet/day  overhead MiB    field window  records at risk     unclosed
once a day                   1        268             2         86400 s       11,454,000            0
4 hours                      6      1,608            13         23400 s        2,267,750           84
hourly                      24      6,432            50         23400 s        1,266,500           84
5 minutes                  288     77,184           603         23400 s        1,029,083          130
5 seconds                17280  4,631,040         36180         23400 s        1,007,860          130

shortening the interval from once a day to 5 minutes multiplies the transfer count by 288; on field laptops the effective window never drops below 23400 seconds

The table above counts what the central copy closes, and the amount it closes is exactly the number of records left outside the transfer window. With once-a-day transfer, the center has no records from that day at all, and 13 of the 13 tampered items stay invisible. In the four-hour interval, six of the eleven deletions surface; in the hourly transfer, all eleven surface. The two changed records show up only in the five-second transfer, because both fall after the last transfer.

All of these rows sit inside the set the local chain calls clean. What makes the comparison possible is a second copy, not the chain.

The table below writes the bill for shortening the window. Going from once-a-day transfer to five minutes multiplies the transfer count by 288 — across the fleet, from 268 to 77,184 — and the transfer overhead runs to 603 MiB a day. At five-second transfer the same overhead becomes 35 GiB.

The last two columns show the real limit. Records at risk drops from 11.45 million to 1.03 million and stays there: pulling the interval from five minutes down to five seconds only lowers that number by two percent. The reason is the fourth column — the effective window on field laptops never drops below 23,400 seconds in any interval, because those machines spend six and a half hours of the day offline. At the five-minute interval, the window never closes at all on 130 machines. The window centralization closes is not a configuration, it is a connectivity limit.

For the record that stays inside the window, only one option remains, and it, too, is an architectural item: writing the record somewhere the machine that produced it cannot change — an append-only medium, or a signature whose key is not present on the endpoint. This does not close the window; it keeps the record inside the window from changing silently.

Summary

• The hash chain catches three tampering forms: deletion at sequence 744 with an 11-sequence gap, a digest-refreshed change at sequence 839, and, once the chain is recomputed, only the sequence gap.
• Once sequence numbers are reissued too, local verification says clean: a chain produced and verified on the same machine is proof of consistency against whoever holds that machine, not proof of integrity.
• The central copy shows 0 of the 13 tampered records with once-a-day transfer, 11 with hourly transfer, and all 13 with five-second transfer; what it closes is exactly the number of records left outside the transfer window.
• Pulling the interval from once a day to five minutes multiplies transfers by 288 and raises overhead to 603 MiB a day; at five seconds the overhead becomes 35 GiB.
• Records at risk drop from 11.45 million to 1.03 million and stay there: on 130 machines the window never closes at any interval, because it is offline time, not a setting, that determines the effective window.

Course Wrap-Up

Not a single row leaves the third column empty, and nine rows read “0 close” in the second column. The course’s rule follows from these two observations: endpoint security is not a product list, it is the count of automatically running items; every point that cannot be closed must at least be made visible, because an invisible point is a point that has been ignored. Any measurement that writes closing and making-visible into the same column overstates its own coverage.

Across sixteen lessons the second column moved in one direction: hardening lowered the item count, allowlisting and disk encryption closed large blocks, protection layers started making what they could not close visible, and the last four lessons measured only visibility. The remaining number always had the same shape.

There is one question that has gone unasked through all this measurement. For five courses the view was always from the defender’s side, and what each check missed was counted one by one — the path that got through, the claim that slipped past, the point left open, the record left blind. But who used these missed paths, in what order, and why was never addressed. Why does an attacker pick one point and leave another, which step leads to which, and when do they stop? The M24/K06 Attack Techniques and the Threat Landscape course begins with exactly this question.