Data and Model Versioning

The previous lesson’s ledger wrote down how many rows each run read. All eighteen runs read their data from the same place, and the name of that place never changed. When the ledger says “2200 rows”, we assumed that two runs read the same 2200 rows, but that assumption was never tested anywhere.

This lesson tests that assumption. Data lineage, that is, the record of the link from a serving value back to the source, was measured in the Introduction to Data Engineering course and is not repeated here. The question here is narrower: when the content under a name changes and the name does not, how many runs become indistinguishable from one another.

• RP19. The fictional producer brings the data in four deliveries. The deliveries are 2200, 2180, 2160, and again 2200 rows; all four are written under the same measurement-2026 name. The fourth delivery is exactly identical in content to the first.
• RP20. A data version is the pairing of a name with a content. Three schemes are compared: unversioned (a single name, overwritten), date-named (one name per delivery), and content-addressed (the name carries the signature produced from the content).
• RP21. A signature is a short digest produced from the content and is written within the lesson; no external library is called. A signature is not a security tool but an identity tool.
• RP22. The baseline is the unversioned setup: a single name, overwritten content, no retroactive reproduction.
• RP23. The units measured are the number of names, ambiguous runs, and stored rows. An ambiguous run is a run that carries the same name yet gives a different result under that name.
• RP24. The feature code is fixed in this lesson; all four runs see the full history and round to four digits. The only thing that changes is the content of the data.
• RP25. The label distribution is {0: 1752, 1: 448}, the base class ratio is 0.7964, and the accuracy numbers in this lesson carry no performance claim. The resolution is 0.0017.

The Content Under the Same Name

The first block rebuilds, verbatim, the fictional source, pipeline and model of the previous two lessons.

# FICTIONAL. Meter readings, the pipeline and the model are defined within the lesson; no real
# learning library or model registry product is called.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


REGION = ["east", "west", "north", "south", "central"]
TYPE = ["residential", "commercial"]
PERIODS = [f"2026-{a:02d}" for a in range(1, 13)]
SUBSCRIBERS = {}
for i in range(200):
    r = rng(SEED + 37 * i)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)],
                                    "start": 10000 + int(r() * 60000)}


def generate_source():
    records = []
    for i, (sub, a) in enumerate(SUBSCRIBERS.items()):
        e = a["start"]
        for p in PERIODS:
            r = rng(SEED + 101 * i + 7 * PERIODS.index(p))
            e += 4 + int(r() * 46)
            records.append({"subscriber": sub, "period": p, "region": a["region"], "type": a["type"],
                            "index": e, "unit": "m3"})
    return records


def s1_screen(records):
    REQUIRED = ("subscriber", "period", "region", "index", "unit")
    keep = [k for k in records if all(a in k for a in REQUIRED)]
    return keep, len(records) - len(keep)


def s2_dedupe(records):
    seen, keep = set(), []
    for k in records:
        key = (k["subscriber"], k["period"])
        if key in seen:
            continue
        seen.add(key)
        keep.append(k)
    return keep, len(records) - len(keep)


def s3_consumption(records):
    table = {(k["subscriber"], k["period"]): k for k in records}
    out, dropped = [], 0
    for k in records:
        i = PERIODS.index(k["period"])
        if i == 0:
            continue
        prev = table.get((k["subscriber"], PERIODS[i - 1]))
        if prev is None:
            dropped += 1
            continue
        try:
            f = int(k["index"]) - int(prev["index"])
        except (TypeError, ValueError):
            dropped += 1
            continue
        out.append(dict(k, m3=f))
    return out, dropped


TRAIN = PERIODS[1:9]
TEST = PERIODS[9:]
THRESHOLD = 40


def rows():
    v, _ = s1_screen(generate_source())
    v, _ = s2_dedupe(v)
    v, _ = s3_consumption(v)
    return v


def label(k):
    return 1 if k["m3"] > THRESHOLD else 0


def features(records, window=None, digits=4):
    """Independent of the weights , computed once."""
    ordered = sorted(records, key=lambda k: PERIODS.index(k["period"]))
    sub, reg = {}, {}
    for k in ordered:
        sub.setdefault(k["subscriber"], []).append((PERIODS.index(k["period"]), k["m3"]))
        reg.setdefault(k["region"], []).append((PERIODS.index(k["period"]), k["m3"]))

    def mean(series, i):
        chosen = [v for d, v in series if (d < i and (window is None or d >= i - window))]
        return round(sum(chosen) / len(chosen), digits) if chosen else 0.0

    feats = {}
    for k in records:
        i = PERIODS.index(k["period"])
        feats[(k["subscriber"], k["period"])] = {
            "subscriber_mean": mean(sub[k["subscriber"]], i),
            "region_mean": mean(reg[k["region"]], i),
            "commercial": 1 if k["type"] == "commercial" else 0}
    return feats


GRID = [(a, b, c) for a in (0.6, 0.8, 1.0) for b in (0.0, 0.2, 0.4) for c in (0, 4, 8)]


def score(o, w):
    return w[0] * o["subscriber_mean"] + w[1] * o["region_mean"] + w[2] * o["commercial"]


def train(records, feats, grid=None, threshold=THRESHOLD, periods=None):
    grid = GRID if grid is None else grid
    periods = TRAIN if periods is None else periods
    subset = [k for k in records if k["period"] in periods]
    best, best_w = -1.0, None
    for w in grid:
        d = sum(1 for k in subset
                if (1 if score(feats[(k["subscriber"], k["period"])], w) > threshold else 0) == label(k))
        o = d / len(subset)
        if o > best:
            best, best_w = o, w
    return {"weights": best_w, "train_accuracy": round(best, 4),
            "candidates": len(grid), "train_rows": len(subset)}


def evaluate(records, model, feats, threshold=THRESHOLD, periods=None):
    periods = TEST if periods is None else periods
    subset = [k for k in records if k["period"] in periods]
    d = sum(1 for k in subset
            if (1 if score(feats[(k["subscriber"], k["period"])], model["weights"]) > threshold else 0)
            == label(k))
    return round(d / len(subset), 4)

The four deliveries run in the block below. In each delivery the same code reads the data under the same name, trains the model, and writes the result under the name consumption-threshold. signature produces a short digest from the content and is only reported for now; it enters no decision.

S = rows()


def signature(text):
    """A short digest produced from the content. Not security , an identity tool."""
    h = 2166136261
    for c in text:
        h = ((h ^ ord(c)) * 16777619) & 0xFFFFFFFF
    return f"{h:08x}"


def data_signature(records):
    return signature("|".join(f"{k['subscriber']}:{k['period']}:{k['m3']}" for k in records))


DELIVERIES = [2200, 2180, 2160, 2200]
RUNS = []
for no, n in enumerate(DELIVERIES, start=1):
    rec = S[:n]
    feats = features(rec)
    m = train(rec, feats)
    RUNS.append({"run": no, "data_name": "measurement-2026", "rows": n,
                 "data_signature": data_signature(rec), "model_name": "consumption-threshold",
                 "weights": m["weights"], "test": evaluate(rec, m, feats)})
print(f"{'delivery':<10}{'data name':<18}{'rows':>7}{'data signature':>16}"
      f"{'model name':>24}{'weights':>17}{'test':>9}")
for k in RUNS:
    print(f"{k['run']:<10}{k['data_name']:<18}{k['rows']:>7}{k['data_signature']:>16}"
          f"{k['model_name']:>24}{str(k['weights']):>17}{k['test']:>9.4f}")
print()
print("different test numbers:", len(set(k["test"] for k in RUNS)),
      " different weights:", len(set(k["weights"] for k in RUNS)),
      " different data contents:", len(set(k["data_signature"] for k in RUNS)))

delivery  data name            rows  data signature              model name          weights     test
1         measurement-2026     2200        bd88a253   consumption-threshold    (0.6, 0.4, 0)   0.8067
2         measurement-2026     2180        f008c71c   consumption-threshold    (0.6, 0.4, 0)   0.8064
3         measurement-2026     2160        cebfcc57   consumption-threshold    (0.6, 0.4, 0)   0.8078
4         measurement-2026     2200        bd88a253   consumption-threshold    (0.6, 0.4, 0)   0.8067

different test numbers: 3  different weights: 1  different data contents: 3

In the table, two columns never change: the data name is measurement-2026 in all four rows, and the model name is consumption-threshold in all four rows. One more column never changes: the weights are (0.6, 0.4, 0) in all four runs, that is, the model file that is produced is exactly the same. Two columns change: the content signature takes three different values, and the test accuracy gives three different numbers.

How Many Contents a Name Has

What is to be counted is now clear. If a naming scheme leaves runs that carry the same name yet give different results, those runs are ambiguous: which content they came from cannot be read from the name.

def measure(identity):
    """How many names there are , how many runs stay ambiguous , how many rows are stored."""
    group = {}
    for k in RUNS:
        group.setdefault(identity(k), []).append(k)
    ambiguous = sum(len(v) for v in group.values() if len(set(x["test"] for x in v)) > 1)
    return len(group), ambiguous, sum(v[-1]["rows"] for v in group.values())


print(f"{'scheme':<18}{'data names':>12}{'ambiguous runs':>16}{'stored rows':>14}")
for name, ident in (("unversioned", lambda k: k["data_name"]),
                    ("date named", lambda k: f"{k['data_name']}@t{k['run']}"),
                    ("content addressed", lambda k: f"{k['data_name']}@{k['data_signature']}")):
    a, b, s = measure(ident)
    print(f"{name:<18}{a:>12}{b:>16}{s:>14}")
print()
print(f"{'model identity':<26}{'versions':>10}{'ambiguous runs':>16}")
for name, ident in (("weights only", lambda k: signature(str(k["weights"]))),
                    ("data signature + weights", lambda k: signature(f"{k['data_signature']}|{k['weights']}"))):
    a, b, _ = measure(ident)
    print(f"{name:<26}{a:>10}{b:>16}")
print()
print("rows read for the signature:", sum(DELIVERIES))

scheme              data names  ambiguous runs   stored rows
unversioned                  1               4          2200
date named                   4               0          8740
content addressed            3               0          6540

model identity              versions  ambiguous runs
weights only                       1               4
data signature + weights           3               0

rows read for the signature: 8740

In the unversioned setup there is a single name, and all four runs are ambiguous. This row is the data-side form of the course’s rule: when the number 0.8078 goes into a report, the content that produced it no longer exists, because the fourth delivery overwrote the third. The stored rows column confirms it: the unversioned scheme takes up the least room, 2200 rows, and in return loses the whole history.

The date-named scheme brings the ambiguity to zero, but it has a flaw: it produces four names although there are three contents. The first and the fourth deliveries are exactly the same content, and since the date scheme cannot see this, it stores the same 2200 rows twice. The content-addressed scheme covers the same ambiguity with three names and 6540 rows; the fourth delivery takes up no new room, because its signature is already in the ledger.

What the Signature Signs

data_signature signs the pipeline’s output, that is, the triple of subscriber, period, and computed consumption. This is a choice and it has a cost: a producer-side change that the pipeline swallows never shows up in the output signature. The block below measures this. Suppose the producer sends sixty records twice in July; the source has changed, but the deduplication stage drops the repeated rows.

def source_signature(records):
    return signature("|".join(f"{k['subscriber']}:{k['period']}:{k['index']}" for k in records))


def feed_pipeline(source):
    v, _ = s1_screen(source)
    v, _ = s2_dedupe(v)
    v, _ = s3_consumption(v)
    return v


raw = generate_source()
# The producer sent 60 records twice: the source changed , the pipeline swallows it.
duplicated = raw + [dict(k) for k in raw if k["subscriber"] <= "A060" and k["period"] == "2026-07"]
print(f"{'delivery':<12}{'source rows':>13}{'source signature':>18}"
      f"{'output rows':>13}{'output signature':>18}{'test':>9}")
for name, source in (("intact", raw), ("duplicated", duplicated)):
    c = feed_pipeline(source)
    feats2 = features(c)
    m2 = train(c, feats2)
    print(f"{name:<12}{len(source):>13}{source_signature(source):>18}{len(c):>13}"
          f"{data_signature(c):>18}{evaluate(c, m2, feats2):>9.4f}")
print()
print(f"{'delivery':<10}{'rows':>7}{'test rows':>11}{'resolution':>12}")
for n in DELIVERIES[:3]:
    test_rows = [k for k in S[:n] if k["period"] in TEST]
    print(f"{n:<10}{n:>7}{len(test_rows):>11}{1 / len(test_rows):>12.4f}")

delivery      source rows  source signature  output rows  output signature     test
intact               2400          d501f04f         2200          bd88a253   0.8067
duplicated           2460          93576832         2200          bd88a253   0.8067

delivery     rows  test rows  resolution
2200         2200        600      0.0017
2180         2180        594      0.0017
2160         2160        588      0.0017

The source signature changed, the output signature stayed exactly the same, and so did the accuracy. The version records of the two deliveries point to the same rows, although the content that came from the producer is different. This is the correct behavior of the pipeline, and it is a limit of versioning: the output signature does not see a change the pipeline swallowed. In the opposite direction it is also lacking; a scheme that signs the source flags a version difference here although the data going to the model has not changed at all. The two signatures answer two separate questions, and a data version stays incomplete unless it is written which one is kept.

The lower table closes another detail. As the delivery gets smaller, the test set gets smaller too: 600, 594, and 588 rows. Across the three deliveries the resolution stays at 0.0017, but the numbers being compared were not measured on the same set. For this reason the difference between 0.8067 and 0.8064 cannot be read as a performance difference.

A Model’s Identity Is Not Only Its Weights

The lower table carries the lesson’s quietest finding. The model file is the same in all four runs, so a scheme that names the model by its own content collects the four runs into a single version and leaves all four ambiguous. This is the mirror image of the error on the data side: there a name concealed three contents, here a content conceals three numbers.

The four model files look interchangeable, because they carry the same weights. They are not interchangeable: each was measured on a different data set, and the numbers they report are 0.8067, 0.8064, and 0.8078. The differences of 0.0014 and 0.0011 in between are below the resolution of 0.0017, and the numbers were moreover not measured on the same set; the reported number is not enough to tell these four files apart. When the data signature is added to the weights, the identity splits into three versions and ambiguous runs fall to 0. A model version is not the signature of the model file but the joint signature of the model file and the data version that produced it.

The Cost of Versioning and What It Does Not Say

The cost has three items. The first is stored rows: the content-addressed scheme holds 6540 rows, 2.97 times the unversioned scheme. The second is rows read: since the signature is computed from the content, all the rows are read once more at every delivery, 8740 rows over four deliveries. The third is the added step: the write operation now has to compute a signature and append a ledger row.

What is gained in return is narrow and exact: versioning says which content produced which number. What it does not say is which content is correct. Which of the three deliveries came in incomplete, which was whole, and whether the number 0.8078 can be trusted are things none of the three signatures tells; what tells them is the checks measured in the Introduction to Data Engineering course. Versioning is not a quality tool but an identity tool, and its contribution is measured only by the number of ambiguous runs.

Summary

• Four deliveries brought 2200, 2180, 2160, and again 2200 rows under the same measurement-2026 name; the name never changed, the content received three different signatures, and the test accuracy came out 0.8067, 0.8064, and 0.8078.
• In the unversioned scheme there is a single name and all four runs are ambiguous; in the content-addressed scheme there are three names and ambiguous runs are 0.
• The date-named scheme closes the ambiguity in the same way but produces four names and stores 8740 rows; the content-addressed scheme does the same job with three names and 6540 rows, that is, 2.97 times the unversioned scheme, because it sees that the fourth delivery is the same as the first.
• The output signature does not see a change the pipeline swallowed: when sixty records arrive twice, the source signature changes and the output signature and the accuracy stay exactly the same. A data version stays incomplete unless it is written which signature is kept.
• The model file is exactly the same in all four runs. A model identity that looks only at the weights collects the four runs into a single version; when the data signature is added, three versions separate and ambiguous runs become 0.
• Versioning says which content produced which number, but it does not say which is correct; that question belongs to the checks.

Next Step

In three lessons the run itself was recorded and versioned: which items it was run with was written down, which content it read was tied to a signature, and the three contents behind the same name were told apart. All of this work was done on the training side. It was never asked whether the feature that goes into the model could be computed by one piece of code during training and by another piece of code when called in production. The next lesson puts those two paths side by side and measures a single question: when the weights are exactly the same, why does the accuracy drop.