Compliance Requirements

In four lessons the model was released, measured in the shadow and the canary, compared, and its decisions were recorded. The four share one trait: the addressee of the measure is internal. The team that sets the canary share, the minimum detectable effect and the room the decision record takes also decides whether they are enough.

This lesson moves the addressee outside. A decision a model produces creates a debt toward its subject, and what that debt is is not left to the team’s preference. The lesson takes it as four obligation classes — record-keeping, explanation, contest and retention period — and counts one engineering counterpart for each: how many additional fields per decision and how many stored records.

• RG41. The source, the pipeline, the feature path, the model and the decision log are fictional. No name of a regulation, standard, institution, country, certificate or audit body is written; no product, service or library is called. Seed 20260218.
• RG42. Obligations are referred to by their classes, and what is measured is not their sources but what they ask of the system: record-keeping asks that a decision be recomputable, explanation that the justification of a single decision be shown, the right to contest that the result of the review be written, and the retention period that the record be deleted after a certain time.
• RG43. The baseline is the setup without obligations: the decision log keeps only the three fields needed to present, and it is assumed that nothing is owed to the outside.
• RG44. The model and the label come from the shared definition: weights (0.6, 0.0, 4), decision threshold 18, balanced accuracy 0.9864. Monitoring is six periods, 1200 decisions, base class ratio 0.5342, and the smallest step of period accuracy is 0.0050.
• RG45. The record is tested with a fictional late-arriving reading: in the period 2026-04 the meters of 40 subscribers were not read on time, and the rows were added to the source after the decisions were made. The source is 2120 rows at decision time and 2200 rows today.
• RG46. The contest window is three periods: a decision can be contested at the latest at the end of the third period. The retention period is also written in periods.

Audit trail, ownership and retention policy were measured in the Introduction to Data Engineering course and the Governance, Risk and Compliance course; their procedures are not repeated. The previous lesson built the decision record. This lesson takes that record as data and asks one question: what does each of the four classes put on top of it.

Translating an Obligation into a Record

The first block sets up the source, the consumption stage of the pipeline, the feature path and the model; then it produces the same decisions twice — once from the source at decision time, once from today’s source.

# FICTIONAL. The meter readings of the measurement network, the first three stages of the
# pipeline, the feature path, the threshold model and the decision log are generated inside
# the lesson; no regulation , audit body , product or service is called. Seed 20260218.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


REGION = ["east", "west", "north", "south", "central"]
TYPE = ["residential", "commercial"]
PERIODS = [f"2026-{a:02d}" for a in range(1, 13)]
SUBSCRIBERS = {}
for i in range(200):
    r = rng(SEED + 37 * i)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)],
                                    "start": 10000 + int(r() * 60000)}


def generate_source():
    records = []
    for i, (sub, a) in enumerate(SUBSCRIBERS.items()):
        e = a["start"]
        for p in PERIODS:
            r = rng(SEED + 101 * i + 7 * PERIODS.index(p))
            e += 4 + int(r() * 46)
            records.append({"subscriber": sub, "period": p, "region": a["region"], "type": a["type"],
                            "index": e, "unit": "m3"})
    return records


def consumption(records):
    """Consumption from the period difference. If there is no previous period the row cannot be computed."""
    table = {(k["subscriber"], k["period"]): k for k in records}
    out = []
    for k in records:
        i = PERIODS.index(k["period"])
        previous = table.get((k["subscriber"], PERIODS[i - 1])) if i else None
        if previous is not None:
            out.append(dict(k, m3=int(k["index"]) - int(previous["index"])))
    return out


def features(records):
    """Independent of the weights , computed once."""
    ordered = sorted(records, key=lambda k: PERIODS.index(k["period"]))
    sub, reg = {}, {}
    for k in ordered:
        sub.setdefault(k["subscriber"], []).append((PERIODS.index(k["period"]), k["m3"]))
        reg.setdefault(k["region"], []).append((PERIODS.index(k["period"]), k["m3"]))

    def average(series, i):
        chosen = [v for d, v in series if d < i]
        return round(sum(chosen) / len(chosen), 4) if chosen else 0.0

    return {(k["subscriber"], k["period"]): {
        "subscriber_mean": average(sub[k["subscriber"]], PERIODS.index(k["period"])),
        "region_mean": average(reg[k["region"]], PERIODS.index(k["period"])),
        "commercial": 1 if k["type"] == "commercial" else 0} for k in records}


GRID = [(a, b, c) for a in (0.6, 0.8, 1.0) for b in (0.0, 0.2, 0.4) for c in (0, 4, 8)]
DECISION_GRID = (18, 22, 26, 30, 34)
MONITORED = PERIODS[6:]
RULE_THRESHOLD, RULE_COMMERCIAL = 30, 6


def score(o, w):
    return w[0] * o["subscriber_mean"] + w[1] * o["region_mean"] + w[2] * o["commercial"]


def predict(o, w, decision):
    return 1 if score(o, w) > decision else 0


def label(o, threshold=RULE_THRESHOLD):
    return 1 if o["subscriber_mean"] + RULE_COMMERCIAL * o["commercial"] > threshold else 0


def train(records, feats):
    """The training criterion is balanced accuracy; the decision threshold is A SEPARATE setting and is searched."""
    subset = [k for k in records if k["period"] in PERIODS[1:9]]
    best = (-1.0, None, None)
    for w in GRID:
        for decision in DECISION_GRID:
            tp = tn = fp = fn = 0
            for k in subset:
                o = feats[(k["subscriber"], k["period"])]
                p, g = predict(o, w, decision), label(o)
                tp, fp, fn, tn = (tp + (p and g), fp + (p and not g),
                                  fn + (not p and g), tn + (not p and not g))
            total = (tp / (tp + fn) if tp + fn else 0.0) + (tn / (tn + fp) if tn + fp else 0.0)
            if total / 2 > best[0]:
                best = (total / 2, w, decision)
    return {"weights": best[1], "decision_threshold": best[2], "balanced": round(best[0], 4)}


# FICTIONAL late-arriving reading: in 2026-04 the meters of 40 subscribers were not read on time
# and the rows were added to the source AFTER the decisions were made.
RAW = generate_source()
AT_DECISION = consumption([k for k in RAW
                           if not (k["period"] == "2026-04" and k["subscriber"] <= "A040")])
TODAY = consumption(RAW)
FEATS_A, FEATS_B = features(AT_DECISION), features(TODAY)
M = train(TODAY, FEATS_B)
MONITORED_ROWS = [k for k in TODAY if k["period"] in MONITORED]
share1 = sum(label(FEATS_B[(k["subscriber"], k["period"])]) for k in MONITORED_ROWS) / len(MONITORED_ROWS)
print(f"model: weights {M['weights']} , decision threshold {M['decision_threshold']} ,"
      f" balanced accuracy {M['balanced']}")
print(f"monitoring periods {len(MONITORED)} , decisions {len(MONITORED_ROWS)} ,"
      f" base class ratio {round(max(share1, 1 - share1), 4)}")

# Baseline record: the least that is needed to present.
BASE_FIELDS = ("subscriber", "period", "prediction")
# The counterpart of the record-keeping obligation: EVERYTHING the decision READ is stored.
S1_FIELDS = ("model_version", "subscriber_mean", "region_mean", "commercial")
S1_MODEL_FIELDS = ("weights", "decision_threshold")
VERSION = {"s1": {"weights": M["weights"], "decision_threshold": M["decision_threshold"]}}

ledger = []
for k in [k for k in AT_DECISION if k["period"] in MONITORED]:
    o = FEATS_A[(k["subscriber"], k["period"])]
    ledger.append({"subscriber": k["subscriber"], "period": k["period"], "model_version": "s1",
                  "prediction": predict(o, M["weights"], M["decision_threshold"]),
                  "subscriber_mean": o["subscriber_mean"], "region_mean": o["region_mean"],
                  "commercial": o["commercial"]})

ledger_same = sum(1 for r in ledger if predict(r, VERSION[r["model_version"]]["weights"],
                                       VERSION[r["model_version"]]["decision_threshold"]) == r["prediction"])
feature_diff = sum(1 for r in ledger
                if FEATS_B[(r["subscriber"], r["period"])]["subscriber_mean"] != r["subscriber_mean"])
decision_diff = sum(1 for r in ledger
             if predict(FEATS_B[(r["subscriber"], r["period"])], M["weights"],
                        M["decision_threshold"])
             != r["prediction"])
print(f"\nsource rows: at decision time {len(AT_DECISION)} , today {len(TODAY)}")
print(f"recomputation from the ledger    : exact {ledger_same}/{len(ledger)} , differing"
      f" {len(ledger) - ledger_same}")
print(f"recomputation from today's source: exact {len(ledger) - decision_diff}/{len(ledger)} ,"
      f" differing {decision_diff} , feature values changed {feature_diff}")

model: weights (0.6, 0.0, 4) , decision threshold 18 , balanced accuracy 0.9864
monitoring periods 6 , decisions 1200 , base class ratio 0.5342

source rows: at decision time 2120 , today 2200
recomputation from the ledger    : exact 1200/1200 , differing 0
recomputation from today's source: exact 1164/1200 , differing 36 , feature values changed 239

The last two lines show why the record-keeping obligation is a question of fields. When the decision log carries the value of the features at the time of the decision, 1200 of 1200 decisions are recomputed exactly. When the same decisions are to be reproduced from today’s source, the features of 239 decisions have changed and 36 decisions come out different. Nobody touched the source; only the 80 late rows have settled into place.

This is a measurement topic before it is a compliance topic. When the justification of a decision is asked, “we look at the source and recompute” is no answer unless the state of the source at decision time has been stored. The late-arriving reading is one of the six degradation classes of the pipeline and was measured as detection lag in the Introduction to Data Engineering course; what is new here is the retrospective face of the same degradation — the pipeline has recovered, the record has been corrupted.

The Field Cost of the Four Classes

The second block counts the four classes in a single table and sweeps the retention period.

# This block continues the PERIODS , MONITORED , FEATS_B , M , VERSION , ledger , score ,
# predict and label definitions of the first block.
MARGIN, CONTEST_WINDOW, RETENTION = 1.0, 3, 3


def explanation(r):
    """DERIVED from the fields of class 1; asks for no new field."""
    w = VERSION[r["model_version"]]["weights"]
    e = VERSION[r["model_version"]]["decision_threshold"]
    p = score(r, w)
    return {"subscriber_contribution": round(w[0] * r["subscriber_mean"], 4),
            "region_contribution": round(w[1] * r["region_mean"], 4),
            "type_contribution": round(w[2] * r["commercial"], 4),
            "score": round(p, 4), "distance_to_threshold": round(p - e, 4)}


contested = [r for r in ledger if r["prediction"] == 1
             and abs(explanation(r)["distance_to_threshold"]) <= MARGIN]
example = contested[0]
print(f"example of a negative decision closest to the threshold: {example['subscriber']} {example['period']}"
      f" prediction {example['prediction']} actual {label(example)}")
print(f"explanation {explanation(example)}")

wrong_near = sum(1 for r in contested if label(r) != r["prediction"])
wrong_all = sum(1 for r in ledger if label(r) != r["prediction"])
n, nb, ns = len(ledger), len(BASE_FIELDS), len(S1_FIELDS)
s1 = n * (nb + ns) + len(S1_MODEL_FIELDS)
print(f"\nclass                       fields per decision   stored values  recomputation")
print(f"baseline (obligation-free)  {nb:>19}  {n * nb:>14}  {n - decision_diff:>9}/{n}")
print(f"record-keeping              {nb + ns:>19}  {s1:>14}  {n:>9}/{n}")
print(f"explanation                 {nb + ns:>19}  {s1:>14}  {n:>9}/{n}")
print(f"contest                     {str(nb + ns) + ' (+3)':>19}  {s1 + 3 * len(contested):>14}"
      f"  {n:>9}/{n}")
kept = len([d for d in MONITORED if PERIODS.index(d) > len(PERIODS) - 1 - RETENTION]) * 200
print(f"retention ({RETENTION} periods)       {nb + ns:>19}  {kept * (nb + ns) + 2:>14}  {kept:>9}/{n}")
print(f"\nnegative decisions within {MARGIN} point of the threshold: {len(contested)}"
      f" (share {round(len(contested) / n, 4)}) ; of these wrong {wrong_near} ,"
      f" wrong in the whole log {wrong_all}")
print(f"monitoring accuracy {round((n - wrong_all) / n, 4)} , step {round(1 / 200, 4)}")

print(f"\nretention (periods)   records kept  records deleted  deleted while contest open")
for R in (1, 2, 3, 6, 12):
    kept_periods = [d for d in MONITORED if PERIODS.index(d) > len(PERIODS) - 1 - R]
    broken = 0
    for t in range(len(PERIODS)):
        for d in MONITORED:
            i = PERIODS.index(d)
            if i <= t - R and i + CONTEST_WINDOW >= t:
                broken += 200
    print(f"{R:>19}  {len(kept_periods) * 200:>13}  {n - len(kept_periods) * 200:>15}  {broken:>26}")

example of a negative decision closest to the threshold: A002 2026-08 prediction 1 actual 1
explanation {'subscriber_contribution': 18.6, 'region_contribution': 0.0, 'type_contribution': 0, 'score': 18.6, 'distance_to_threshold': 0.6}

class                       fields per decision   stored values  recomputation
baseline (obligation-free)                    3            3600       1164/1200
record-keeping                                7            8402       1200/1200
explanation                                   7            8402       1200/1200
contest                                  7 (+3)            8726       1200/1200
retention (3 periods)                         7            4202        600/1200

negative decisions within 1.0 point of the threshold: 108 (share 0.09) ; of these wrong 26 , wrong in the whole log 26
monitoring accuracy 0.9783 , step 0.005

retention (periods)   records kept  records deleted  deleted while contest open
                  1            200             1000                        2400
                  2            400              800                        1400
                  3            600              600                         600
                  6           1200                0                           0
                 12           1200                0                           0

The table shows that the four classes are not equal. Only record-keeping really asks for new fields per decision: four fields, from 3600 stored values to 8402. The explanation obligation adds zero fields. The example row shows this on one decision: for subscriber A002’s 2026-08 decision the three contributions, the score and the distance to the threshold are written, and all five are derived from the fields of the record-keeping class. As long as the weights and the threshold are stored, the explanation is a computation, not a record. The right to contest is met in part: record-keeping has already paid for recomputing the decision, what is new is only the result of the review, and that is written not for all decisions but for 108 of them.

The number 108 is the sharpest reading of the lesson. The negative decisions within one point of the threshold make up 0.0900 of the 1200 decisions, but 26 of the model’s 26 wrong predictions are inside this band. With a monitoring accuracy of 0.9783 and a period step of 0.0050, 26 wrong predictions are a measurable size. Because recall is 1.0000, all of the wrong predictions are false positives: subscribers who fell immediately above the decision threshold and are in reality low. The engineering counterpart of the right to contest is to open nine percent of the decisions to review and in return be able to reach all of the wrong ones — nearness to the boundary is not a coincidence, it is a structural property of every model with a threshold.

Retention Cuts Across the Other Three

The counterpart of the fourth class is unlike the other three, because it adds no field: the period is already recorded and the duration is one policy number. What the retention policy does is delete records, and the records it deletes are the ones the first three classes rely on. With a retention of three periods the log falls from 1200 decisions to 600, 8402 stored values drop to 4202 and the justification of 600 decisions can no longer be asked. The last lesson of the Introduction to Data Engineering course measured this on the data side — the retention policy leaves periods that cannot be audited retrospectively; on the decision side the same thing happens by the same mechanism.

The last column of the sweep turns the collision into a number. With a contest window and a retention period of three periods each, one period’s set of decisions is deleted while the right to contest is still open in every period: 600 decisions in total. At two periods it becomes 1400, at one 2400; at six it drops to zero. This number is not an implementation error, it is two obligation classes cutting each other: one asks for the record, the other deletes it, and both are in force in the same system at the same time. The measure of the trade-off is exactly this column — when the retention period is not longer than the contest window, the number of unmet obligations is greater than zero; equality is not enough.

The rule that follows is independent of the source of the classes: an obligation class counts as unmeasured unless its counterpart is written into a field in the decision record or into a record count.

Summary

• When the decision log holds the features at decision time, 1200 of 1200 decisions are recomputed exactly; when computed from today’s source the features of 239 decisions change and 36 decisions come out different. Nobody touched the source; the 80 late-arriving rows have settled into place.
• Record-keeping adds four fields per decision and raises the stored values from 3600 to 8402. It is the only one among the four classes that really asks for new fields.
• Explanation adds zero fields: the three contributions, the score and the distance to the threshold are derived from the fields of record-keeping. As long as the weights and the threshold are stored, the explanation is a computation.
• Contest adds three fields only on the 108 negative decisions within one point of the threshold; those 108 decisions are 0.0900 of 1200 and cover all 26 of the model’s 26 wrong predictions.
• Retention adds no fields, it deletes records: with three periods the log falls to 600 and the number of decisions deleted while the contest window is still open is 600. Two obligation classes cut each other in the same system.

Next Step

The counterparts of the obligations were written as fields and records; all of them either added something to what is stored or took something away from it. But no line of this lesson asked what keeping the model running itself costs. A model does two separate jobs: it is retrained period by period and it produces inference in every period. The two are written to the same budget and read as a single total. The next lesson splits that total in two and counts which item grows as the retraining frequency increases and which decision reading it as a single number hides.