Master Data Management

Across two lessons, data always flowed in one direction: one system wrote, the others read. The source was fixed, the edge’s direction was fixed, and there was no such problem as two copies carrying a different truth. In the enterprise, this assumption does not hold for most entities. A member record lives in the membership system, in the loan service, and in the municipality’s identity service; all three are written from their own screen, and the same member’s address can carry three different values in three places.

This lesson builds that situation and counts it: how many systems write, how many fields conflict, how many values a reconciliation round fixes, and what is left over. Then one of the systems is chosen as the system of record, the same event stream is run again, and the same numbers are read again.

IN7 — the enterprise is fictional; the systems, their fields, owners, and write events are a data structure in node. Randomness is generated from a single visible seed, and the generator is inside the code.

IN8 — replication mechanics are not modeled. Rationale: keeping copies, replication lag, and eventual consistency were measured in a separate course. What is measured here is not how the data gets copied, but who writes it and the fact that two copies carry a different truth; a write is treated as visible the instant it happens.

IN9 — the reconciliation rule is modeled as “last writer wins.” Rationale: the quantity measured is not the rule itself, but the surface the rule can be applied to. In a system that cannot be written from outside, no correction can be written no matter which rule is chosen; changing the rule only changes the winner, not where the remaining inconsistency sits.

The Same Entity, Three Systems

Three systems hold the same member, and the three have different field sets. The membership system carries four fields, the loan service three, the municipality’s identity service two. All three have a different owner, and all three have their own users. The determining difference is not in the field count, but in the last column: the identity service cannot be written from outside. The enterprise can read that system, but it cannot send it a correction.

// enterprise.mjs — the three systems holding the member record: fictional enterprise model and event generator
export const SYSTEM = [
  { name: "membership", owner: "membership unit", externallyWritable: 1,
    field: ["name", "address", "phone", "email"] },
  { name: "loan", owner: "in-house development", externallyWritable: 1,
    field: ["name", "address", "phone"] },
  { name: "identity", owner: "municipal IT", externallyWritable: 0,
    field: ["name", "address"] },
];
export const FIELD = ["name", "address", "phone", "email"];
export const MEMBERS = 120, WRITES = 6, SEED = 20260411;   // member count, writes per member, visible seed
const makeRng = (t) => () => ((t = (t * 1103515245 + 12345) % 2147483648) / 2147483648);

// each system's own user writes from their own screen; events are generated from a single seed
export const events = () => {
  const r = makeRng(SEED), o = [];
  for (let u = 1; u <= MEMBERS; u += 1) {
    for (let w = 1; w <= WRITES; w += 1) {
      const s = SYSTEM[Math.floor(r() * SYSTEM.length)];
      const f = s.field[Math.floor(r() * s.field.length)];
      o.push({ member: u, system: s.name, field: f, value: `d${Math.floor(r() * 4)}`, time: w });
    }
  }
  return o;
};

if (process.argv[1].endsWith("enterprise.mjs")) {
  const O = events();
  console.log(`${SYSTEM.length} systems hold the same entity, ` +
    `${new Set(SYSTEM.map((s) => s.owner)).size} distinct owners; ` +
    `externally writable ${SYSTEM.filter((s) => s.externallyWritable).length}`);
  for (const s of SYSTEM) {
    console.log(`  ${s.name.padEnd(12)}${String(s.field.length).padStart(2)} fields  ` +
      `${String(O.filter((o) => o.system === s.name).length).padStart(3)} writes  ${s.owner}`);
  }
  console.log(`total ${O.length} writes, ${MEMBERS} members, ${FIELD.length} fields`);
}

3 systems hold the same entity, 3 distinct owners; externally writable 2
  membership   4 fields  224 writes  membership unit
  loan         3 fields  258 writes  in-house development
  identity     2 fields  238 writes  municipal IT
total 720 writes, 120 members, 4 fields

Two Write Arrangements

In the distributed arrangement, a write lands wherever it came from and stays there. In the golden record arrangement, every write that comes from an externally writable system’s screen goes to the system of record first, then spreads out to the other writable systems; a write coming from the municipality’s own screen stays outside this arrangement. The reconciliation round runs with the same function in both arrangements: for every conflicting field, the most recently written value is found and applied to the writable systems.

// arrangement.mjs — two write arrangements and a reconciliation round; store format: store[system][member][field]
import { SYSTEM } from "./enterprise.mjs";
const find = (s) => SYSTEM.find((x) => x.name === s);
export const empty = () => Object.fromEntries(SYSTEM.map((s) => [s.name, {}]));
export const put = (store, sys, member, field, value, time) => {   // if it is not carried, the write step is not spent
  if (!find(sys).field.includes(field)) return 0;
  (store[sys][member] ??= {})[field] = { value, time };
  return 1;
};
// distributed arrangement: a write lands wherever the system's screen it came from
export const distributed = (store, o, down) => (down.includes(o.system)
  ? { step: 0, first: null }
  : { step: put(store, o.system, o.member, o.field, o.value, o.time), first: o.system });
// golden record arrangement: every externally writable write goes to the system of record first, then spreads out
export const golden = (store, o, down, recordSystem = "membership") => {
  if (!find(o.system).externallyWritable) return distributed(store, o, down);
  if (down.includes(recordSystem)) return { step: 0, first: null };
  let step = put(store, recordSystem, o.member, o.field, o.value, o.time);
  for (const s of SYSTEM.filter((x) => x.externallyWritable && x.name !== recordSystem)) {
    step += put(store, s.name, o.member, o.field, o.value, o.time);
  }
  return { step, first: recordSystem };
};
// conflict: more than one different value stands across systems for the same (member, field)
export const conflict = (store, members, fields) => members.flatMap((m) => fields.flatMap((f) => {
  const v = SYSTEM.map((s) => store[s.name][m]?.[f]).filter(Boolean);
  return new Set(v.map((x) => x.value)).size > 1 ? [{ m, f }] : [];
}));
// reconciliation round: the last writer wins, and the winner is applied to the writable systems
export const reconcile = (store, c) => {
  let corrected = 0;
  for (const { m, f } of c) {
    const winner = SYSTEM.map((s) => ({ ...store[s.name][m]?.[f] })).filter((x) => x.value)
      .sort((x, y) => y.time - x.time)[0];
    for (const s of SYSTEM.filter((x) => x.externallyWritable)) {
      const now = store[s.name][m]?.[f];
      if (now && now.value !== winner.value) { put(store, s.name, m, f, winner.value, winner.time); corrected += 1; }
    }
  }
  return corrected;
};

Measurement

The tool runs seven hundred twenty write events first through the distributed arrangement, then through the golden record arrangement. After each run, conflicting field pairs are counted, a reconciliation round runs, and the conflict is counted again afterward. The final measure repeats the same event stream with the membership system down and counts how many writes are accepted.

// measure.mjs — runs the same event stream through both arrangements and counts the inconsistency
import { FIELD, SYSTEM, MEMBERS, events } from "./enterprise.mjs";
import { golden, empty, conflict, distributed, reconcile } from "./arrangement.mjs";
const EVENTS = events(), MEMBER_IDS = [...Array(MEMBERS).keys()].map((i) => i + 1);
const run = (write, down = []) => {
  const store = empty(), writers = new Set();
  let step = 0, accepted = 0;
  for (const o of EVENTS) {
    const { step: n, first } = write(store, o, down);
    step += n;
    if (first) { accepted += 1; writers.add(first); }
  }
  return { store, step, accepted, writers: writers.size };
};
const measure = (write) => {
  const t = run(write);
  const before = conflict(t.store, MEMBER_IDS, FIELD);
  const corrected = reconcile(t.store, before);
  const after = conflict(t.store, MEMBER_IDS, FIELD);
  return { ...t, before, corrected, after, down: run(write, ["membership"]).accepted };
};
const DIST = measure(distributed), GOLD = measure(golden);
const row = (name, x, y) => console.log(`${name.padEnd(45)}${String(x).padStart(13)}${String(y).padStart(15)}`);

row("measure", "distributed", "golden record");
row("systems receiving user writes", DIST.writers, GOLD.writers);
row("write steps per event", (DIST.step / EVENTS.length).toFixed(2), (GOLD.step / EVENTS.length).toFixed(2));
row("conflicting (member, field) pairs", DIST.before.length, GOLD.before.length);
row("fields corrected in reconciliation", DIST.corrected, GOLD.corrected);
row("inconsistencies remaining after reconciliation", DIST.after.length, GOLD.after.length);
row("writes accepted while membership is down", `${DIST.down}/${EVENTS.length}`, `${GOLD.down}/${EVENTS.length}`);
console.log();
console.log(`${"field".padEnd(9)}${"carriers".padStart(9)}${"dist. conflict".padStart(16)}` +
  `${"dist. remaining".padStart(17)}${"golden conflict".padStart(17)}${"golden remaining".padStart(18)}`);
for (const f of FIELD) {
  const n = (l) => l.filter((x) => x.f === f).length;
  console.log(`${f.padEnd(9)}${String(SYSTEM.filter((s) => s.field.includes(f)).length).padStart(9)}` +
    `${String(n(DIST.before)).padStart(16)}${String(n(DIST.after)).padStart(17)}` +
    `${String(n(GOLD.before)).padStart(17)}${String(n(GOLD.after)).padStart(18)}`);
}

measure                                        distributed  golden record
systems receiving user writes                            3              2
write steps per event                                 1.00           1.61
conflicting (member, field) pairs                      131             95
fields corrected in reconciliation                      95             94
inconsistencies remaining after reconciliation           48             48
writes accepted while membership is down           496/720        238/720

field     carriers  dist. conflict  dist. remaining  golden conflict  golden remaining
name             3              62               32               53                32
address          3              54               16               42                16
phone            2              15                0                0                 0
email            1               0                0                0                 0

Where Conflict Comes From

The first column of the second table is explanatory on its own. The email field lives only in the membership system and never conflicts in either arrangement. Phone lives in two systems, address and name in three; the conflict counts follow the same order. Conflict is born not from a field’s importance, how often it gets filled, or data quality, but from how many systems write that field. For four fields across a hundred twenty members, the distributed arrangement measures a hundred thirty-one conflicting pairs; all of them are in the three fields more than one system writes.

The reconciliation round corrects ninety-five field values out of a hundred thirty-one conflicts, and forty-eight conflicts remain. Where the remainder falls is in the fourth column of the second table: thirty-two in name, sixteen in address. These two fields are exactly the two fields the identity service carries. All fifteen conflicts in the phone field are corrected, because both systems that carry phone are writable.

The source of the remaining inconsistency is not weakness in the reconciliation rule. The last-writer-wins rule determines a winner in every conflict; the problem is that the winner cannot be written to the other systems. When the winning value turns out different from the value in the municipality’s system, the enterprise cannot correct that system. The number forty-eight is not a measure of data quality — it is a measure of an authority boundary.

What the Golden Record Changes

When the membership system is chosen as the system of record, the table changes in four places. The number of systems receiving user writes drops from three to two — two, not three, because the third writer sits outside the enterprise’s authority boundary. The conflicting-pair count drops from a hundred thirty-one to ninety-five. Conflict in the phone field disappears entirely: since the two writable systems are now fed from the same write path, that field can no longer produce two different truths.

Conflict in the name and address fields drops but does not reach zero — from sixty-two to fifty-three, and from fifty-four to forty-two. All of the remaining conflict sits between the enterprise’s own copies and the municipality’s copy.

The fifth row is this lesson’s harshest number: the remaining inconsistency is exactly forty-eight in both arrangements. This is not a coincidence — it is a structural consequence of the mechanism. Reconciliation’s winner is the most recently written value for that field; which screen it came from and which system it landed in does not change the winner. Whenever the winner did not come from the identity service, that service’s copy stays wrong, and because it cannot be written from outside, it cannot be corrected. The golden record choice does not reduce even one of these forty-eight conflicts.

This is where master data management’s real limit shows up. A golden record produces consistency on the surface that write authority reaches. For every copy that sits outside that surface, the only thing it can produce is knowing where the inconsistency is. What can be done for the forty-eight records is not to correct them, but to make them reportable and to write down, in the enterprise’s own decisions, which copy to trust.

What the Choice Costs

The second and sixth rows give the cost. Write steps per event go from 1.00 to 1.61. The increase is not twofold, because only the fields more than one writable system carries get spread out; email writes stay at a single step. The write path’s lengthening also comes back as a measurable delay: the user now writes not to their own system’s database, but to another unit’s system, and waits for that write to succeed.

The sixth row is a harsher form of this. With the membership system down, four hundred ninety-six of seven hundred twenty writes are accepted in the distributed arrangement, while only two hundred thirty-eight are accepted in the golden record arrangement — and those are only the ones coming from the municipality’s own screen. All of the enterprise’s member writes are now tied to a single unit’s system. In the distributed arrangement, the membership system being down did not stop the loan service from updating a member’s address; in the golden record arrangement, it does.

The cost paid is not only technical. Choosing a system of record turns one unit’s system into a precondition for other units’ operation: availability target, maintenance window, and release calendar stop being decisions that unit can make on its own. At enterprise scale, the golden record decision looks like a data decision, but what it pays for is a dependency and a coordination obligation.

Summary

• When the same member record is held in three systems, 720 write events produce 131 conflicting (member, field) pairs in the distributed arrangement; conflict is born only in fields more than one system writes.
• The email field, which lives in a single system, never conflicts in either arrangement; the phone field, which lives in two systems, drops from 15 conflicts to 0 in the golden record arrangement.
• A reconciliation round corrects 95 field values in the distributed arrangement and 94 in the golden record arrangement; the remaining inconsistency is 48 in both arrangements, and all of it sits in the two fields the non-externally-writable system carries.
• The golden record choice drops conflict from 131 to 95 but does not reduce the remaining inconsistency by even one: correction only works on the surface write authority reaches.
• The choice costs a rise in write steps per event from 1.00 to 1.61, and a drop in writes accepted while the system of record is down from 496/720 to 238/720.

Next Step

Across three lessons, every measurement stayed on the operating path: a record was written, passed through an edge, was read in a system. The management unit, though, does not ask about any of these one at a time; it wants the period’s loan count, the per-branch late rate, and membership movements together, and answering these questions requires laying three separate systems’ data side by side. Running these questions on the operating systems was never measured: the load a reporting query puts on the source system, bringing three systems’ different field names into a single resolved format, and the delay that transformation adds to the result’s freshness. The next lesson separates the resolution load from the operating path and counts these three quantities.