Timeline Creation

The previous lesson measured when items are collected. Each collected item carries its own timestamps: moments of file change, moments of login, moments of connection in the flow record. The next step of examination is to merge these timestamps onto a single line; the result is called a timeline, and it is the backbone of the report.

What a timeline carries is a claim of order. The sentence “the configuration changed first, then the external connection was established” is not a piece of time information, it is a cause-and-effect proposition; reversed, it says the opposite. The silent assumption the order rests on is that every source reads the same clock. This lesson tests that assumption: the events of three sources are merged, the number of events that land in the wrong order is counted, skew is estimated and corrected, and how many events the correction moves are measured. The events and clocks are fictional.

• FE19. A timeline is the ordering of events from different sources onto a single axis. The information a timeline carries is not the time values, it is the order.
• FE20. Each source’s clock carries its own skew: a fixed offset and a drift that accumulates per second. Both are fictional.
• FE21. An anchor event is the same event recorded by two sources at once. Skew is estimated only from anchor events.
• FE22. The central log is the reference clock. Ordering is a relative question; absolute accuracy is not required, aligning the sources to one another is enough.
• FE23. An inverted pair is a pair of events where one actually happened first, but the timeline shows the other one first. This is the measure of ordering error.
• FE24. The true times are known in this model. They are not known in a real examination; the only foothold there is anchor events.

Three Clocks, One Timeline

// forensics/time.mjs — MODEL of three sources' fictional event records and clock skew.
// No real organization, address, or event name is written; the clocks are fictional.

export const SEED = 20260805, N = 54, WINDOW = 7200;   // a two-hour window
const generator = (t) => { let s = t >>> 0;
  return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; };

// source -> [fixed offset(s), drift(s/s)]  the central log is the reference clock
export const SOURCE = {
  "central-log": [0, 0], "endpoint-record": [-214, -0.0042], "flow-collector": [96, 0.0013],
};
export const TYPE = ["login", "configuration-change", "external-connection",
  "process-start", "file-write", "privilege-escalation"];

// Reported time: the value read on the source's own clock.
export const report = (k, t) =>
  Math.round(t + SOURCE[k][0] + SOURCE[k][1] * t);

// Events are generated with their true times; anchor events are recorded by two sources at once.
export function events() {
  const r = generator(SEED), names = Object.keys(SOURCE), list = [];
  for (let i = 0; i < N; i += 1) {
    const t = Math.floor(r() * WINDOW), k = names[Math.floor(r() * 3)];
    list.push({ no: i, t, source: k, type: TYPE[Math.floor(r() * TYPE.length)],
      anchor: k !== "central-log" && r() < 0.3 });
  }
  list.sort((a, b) => a.t - b.t || a.no - b.no);
  const record = [];
  for (const o of list) {
    record.push({ ...o, reported: report(o.source, o.t) });
    if (o.anchor) record.push({ ...o, source: "central-log", secondary: true,
      reported: report("central-log", o.t) });
  }
  return record;
}

// Inverted pair: event pairs where a truly came first but b appears first on the timeline.
export function invertedPairs(list) {
  let n = 0;
  for (let i = 0; i < list.length; i += 1)
    for (let j = i + 1; j < list.length; j += 1)
      if (list[i].t > list[j].t) n += 1;
  return n;
}
export const orderBy = (record, time) =>
  [...record].sort((a, b) => time(a) - time(b) || a.no - b.no);

Skew consists of two components, and the two behave differently. Fixed offset is how many seconds a source’s clock has been ahead or behind from the start; it shifts every record by the same amount. Drift, on the other hand, is the clock’s accumulating error; the amount of shift grows over time. A fixed offset is corrected with a single number, drift cannot be corrected — it can only be measured and written down as residual error.

Raw Timeline and Anchor-Based Correction

// forensics/timeline.mjs — raw merge, anchor-based correction, and remaining inverted pairs.
import { SOURCE, N, events, invertedPairs, orderBy } from "./time.mjs";

const y = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd());
const E = events();
const primary = E.filter((k) => !k.secondary);
const anchors = E.filter((k) => k.anchor);

const A = [-18, 8, 8, 15, 16];
y(A, "source", "records", "anchors", "true offset", "anchor estimate");
const estimate = {};
for (const k of Object.keys(SOURCE)) {
  const c = anchors.filter((x) => x.source === k && !x.secondary);
  const d = c.map((x) => x.reported -
    anchors.find((z) => z.no === x.no && z.secondary).reported);
  estimate[k] = d.length ? Math.round(d.reduce((a, b) => a + b, 0) / d.length) : 0;
  y(A, k, primary.filter((x) => x.source === k).length, c.length, SOURCE[k][0], estimate[k]);
}

const raw = orderBy(primary, (o) => o.reported);
const fixed = orderBy(primary, (o) => o.reported - estimate[o.source]);
const truth = orderBy(primary, (o) => o.t);
const moved = (a, b) => a.filter((o, i) => b[i].no !== o.no).length;

console.log("");
const B = [-24, 16, 16, 16];
y(B, "timeline", "inverted pairs", "wrong position", "largest skew");
const pairs = (N * (N - 1)) / 2;
for (const [name, d, z] of [["raw merge", raw, (o) => o.reported],
  ["anchor-corrected", fixed, (o) => o.reported - estimate[o.source]],
  ["true time", truth, (o) => o.t]])
  y(B, name, invertedPairs(d), moved(d, truth), Math.max(...d.map((o) => Math.abs(z(o) - o.t))));
console.log(`${N} events, ${pairs} pairs, ${anchors.length / 2} anchor events`);
console.log(`inverted-pair rate in the raw timeline ${(invertedPairs(raw) * 100 / pairs).toFixed(1)}%, ` +
  `${(invertedPairs(fixed) * 100 / pairs).toFixed(1)}% after correction`);
console.log(`the correction changes the timeline position of ${moved(raw, fixed)} events`);

source             records anchors    true offset anchor estimate
central-log             13       0              0               0
endpoint-record         25      12           -214            -230
flow-collector          16       7             96             100

timeline                  inverted pairs  wrong position    largest skew
raw merge                             35              33             243
anchor-corrected                       1               2              15
true time                              0               0               0
54 events, 1431 pairs, 19 anchor events
inverted-pair rate in the raw timeline 2.4%, 0.1% after correction
the correction changes the timeline position of 34 events

The gap between these two numbers is this lesson’s first result. In the raw timeline, the inverted-pair rate is 2.4 percent and looks small; in the same timeline, 33 of 54 events are in the wrong position, that is, 61 percent of the events. These two ratios do not contradict each other: a skew of a few hundred seconds shifts most events by a few positions, but it only inverts pairs whose time gap is smaller than the skew. A reading that looks at the timeline and says “the events are roughly in the right order” rests on the 97.6 percent rate; the sentences actually used in a report, however, are built about events that are close to one another.

Anchor-based correction brings the inverted-pair count down from 35 to 1 and the wrong-position count down from 33 to 2. The one remaining inverted pair is a residual, and its cause is visible in the table: the endpoint record’s true offset is −214, while the anchor estimate is −230. The sixteen-second bias comes from drift. A correction made with a single fixed number is accurate at the middle of the window and wrong at both ends; the largest remaining skew is 15 seconds. This number should be written in the report as the resolution: the order of two events closer than fifteen seconds cannot be proven with this timeline.

The last line gives the scale of the correction: the position of 34 events on the timeline changes. A narrative written before the correction would be a different text at thirty-four points after it.

Anchor Count and the Shape of the Correction

// forensics/anchor.mjs — effect of anchor count and correction shape on the remaining error.
import { SOURCE, events, invertedPairs, orderBy } from "./time.mjs";

const y = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd());
const E = events(), primary = E.filter((o) => !o.secondary);
// Anchor pair: (time read at reference source, difference between the two records).
const pair = (k) => E.filter((o) => o.anchor && o.source === k && !o.secondary).map((o) => {
  const m = E.find((z) => z.no === o.no && z.secondary);
  return [m.reported, o.reported - m.reported];
});

const flat = (c) => [Math.round(c.reduce((a, [, d]) => a + d, 0) / c.length), 0];
// Linear fit: difference = offset + drift * time.
function linear(c) {
  const n = c.length, st = c.reduce((a, [t]) => a + t, 0), sd = c.reduce((a, [, d]) => a + d, 0);
  const stt = c.reduce((a, [t]) => a + t * t, 0), std = c.reduce((a, [t, d]) => a + t * d, 0);
  const b = (n * std - st * sd) / (n * stt - st * st);
  return [(sd - b * st) / n, b];
}
const back = ([a, b]) => (o) => (o.reported - a) / (1 + b);

const A = [-8, 15, 12, 14, 14, 13];
y(A, "anchors", "flat estimate", "flat inv.", "flat skew", "linear inv.", "lin. skew");
for (const n of [2, 4, 8, 12]) {
  const p = {}, q = {};
  for (const k of Object.keys(SOURCE)) {
    const c = pair(k).slice(0, n);
    p[k] = c.length ? flat(c) : [0, 0];
    q[k] = c.length > 1 ? linear(c) : [0, 0];
  }
  const d1 = orderBy(primary, (o) => back(p[o.source])(o));
  const d2 = orderBy(primary, (o) => back(q[o.source])(o));
  const skew = (m) => Math.round(Math.max(...primary.map((o) =>
    Math.abs(back(m[o.source])(o) - o.t))));
  y(A, n, p["endpoint-record"][0], invertedPairs(d1), skew(p),
    invertedPairs(d2), skew(q));
}
console.log(`\ntrue offset ${SOURCE["endpoint-record"][0]} s, true drift ` +
  `${SOURCE["endpoint-record"][1]} s/s; endpoint-record has ${pair("endpoint-record").length} ` +
  `anchors, flow-collector has ${pair("flow-collector").length}`);

anchors   flat estimate   flat inv.     flat skew   linear inv.    lin. skew
2                  -218           0            25             0            4
4                  -221           0            22             0            1
8                  -226           1            17             0            1
12                 -230           1            15             0            1

true offset -214 s, true drift -0.0042 s/s; endpoint-record has 12 anchors, flow-collector has 7

The second column goes the opposite way from what is expected. As the anchor count rises from two to twelve, the flat estimate shifts from −218 to −230, meaning it moves away from the true offset of −214. This is not a computation error: the flat estimate is the average of the whole window, and drift accumulates across the window. More anchors do not make a wrong model’s estimate correct; they only make it wrong in a more stable way. The remaining skew shows the same thing — it falls from 25 seconds to 15 seconds and stays there.

The two-coefficient correction gives a different result with the same anchor set. When the difference is fit as a linear function of time, the remaining skew falls to 4 seconds with even two anchors, and to 1 second after four anchors. The difference lies not in the amount of data, but in the shape of the model: correcting a clock that has drift with a single fixed number is representing a sloped line with a horizontal one.

The inverted-pair columns carry no decision at this scale, and that has to be written down. In a number that has fallen from thirty-five to between zero and one, the difference between 0 and 1 is just which side a single pair falls on; the real information is in the remaining-skew column. A timeline is reported not as “no inverted pairs remain,” but as “the order of events closer than fifteen seconds cannot be proven.”

The linear correction has one condition, and it sits where the table does not read: the anchor events must be spread across the window. If they are all clustered at one end of the window, the slope estimate becomes unstable, and the correction behaves worse than a flat correction at the other end. What is decisive is not the anchor count, but the anchors’ distribution.

Pairs That Invert

// forensics/pairs.mjs — pairs that invert in the raw timeline and the false inference they cause.
import { events, orderBy } from "./time.mjs";

const y = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd());
const primary = events().filter((k) => !k.secondary);
const raw = orderBy(primary, (o) => o.reported);

const inverted = [];
for (let i = 0; i < raw.length; i += 1)
  for (let j = i + 1; j < raw.length; j += 1)
    if (raw[i].t > raw[j].t) inverted.push([raw[i], raw[j], raw[i].t - raw[j].t]);
inverted.sort((a, b) => b[2] - a[2]);

const CODE = { "central-log": "C", "endpoint-record": "E", "flow-collector": "F" };
const A = [-28, -28, 11, 13];
y(A, "appears first on timeline", "appears second on timeline", "raw gap", "true gap");
for (const [a, b] of inverted.slice(0, 5))
  y(A, `${CODE[a.source]}/${a.type}`, `${CODE[b.source]}/${b.type}`,
    `${b.reported - a.reported} s`, `${b.t - a.t} s`);

const different = inverted.filter(([a, b]) => a.source !== b.source).length;
console.log(`\n${different} of ${inverted.length} inverted pairs come from two different sources, ` +
  `${inverted.length - different} from the same source`);
console.log(`the largest inversion hides a true gap of ${inverted[0][2]} seconds`);

appears first on timeline   appears second on timeline      raw gap     true gap
E/external-connection       F/external-connection              18 s       -306 s
E/external-connection       C/configuration-change              9 s       -224 s
E/configuration-change      F/process-start                   141 s       -200 s
E/process-start             F/configuration-change            145 s       -191 s
E/external-connection       F/login                           156 s       -179 s

35 of 35 inverted pairs come from two different sources, 0 from the same source
the largest inversion hides a true gap of 306 seconds

The sentence before the last line gives this lesson’s most useful rule: 35 of 35 inverted pairs come from two different sources, none of them from within the same source. The reason is structural. All of a single source’s records are read from the same clock; however wrong that clock is, it shifts its own records by the same amount and preserves their internal order. Order breaks down only between sources.

This gives a distinction directly applicable to an examination. The order within a single source is independent of skew and needs no additional foundation. The order between two events from two sources, however, is a claim, and its foundation is only established once skew is measured. If a report writes these two with the same sentence pattern, the uncertainty the second one carries becomes invisible.

The first row of the table is the concrete counterpart of this. The endpoint’s external-connection record appears 18 seconds before the flow collector’s external-connection record on the raw timeline; in truth it comes 306 seconds after it. An examination reading these two rows side by side would conclude that the endpoint connection caused the flow record — but the order is exactly reversed, and the two records are separated by an interval longer than five minutes. The error skew produces is not a delay, it is a direction error.

Summary

• The information a timeline carries is not time values, it is order; order builds a cause-and-effect proposition, and reversed, it says the opposite.
• In the raw merge, 35 of 1,431 pairs turn out inverted (2.4 percent), but 33 of 54 events are in the wrong position; a low pair rate does not mean the timeline is correct.
• The correction estimated from nineteen anchor events brings inverted pairs down to 1 and wrong position down to 2, and changes the timeline position of 34 events.
• A fixed-offset estimate cannot correct drift: the true offset is −214 while the estimate is −230, and the largest remaining skew is 15 seconds; this number is the timeline’s resolution.
• Increasing the anchor count does not bring the flat estimate closer to the true offset (it goes from −218 to −230); a correction fit as a linear function of time brings the remaining skew down to 4 seconds with just two anchors. What is decisive is not the anchor count, but the model’s shape and the anchors’ spread across the window.
• All of the inverted pairs come from two different sources; the order within a single source is unaffected by skew, while the order between sources cannot be claimed without measurement.

Next Step

In this lesson, events arrived as ready-made records: each one had a type, a source, and a time. In a real examination, part of the record set is not ready-made in this way; it is drawn out of structures the system keeps for its own operation. An entry in a configuration store, a filesystem metadata record, a shortcut, a cache entry — none of these was written to be an “event log,” but each leaves a trace that can be ordered.

The next lesson treats these structures in terms of their evidentiary value. Each structure answers a question, but which question it answers varies from structure to structure: some show directly that something happened, some only carry an inference, and some also arise from the machine’s ordinary use. Any sentence written without counting this last category puts baseline noise into the report as a finding.