Fairness Metrics

The previous lesson assumed that when one subgroup’s number came out lower than another’s it was a harm, and reported the gap directly. The question of which number’s gap was left open. Equalizing the rate of flagged subscribers in two subgroups, equalizing the rate of suspicious cases caught, and equalizing how much of what is flagged is actually suspicious are three separate demands. All three are reasonable, all three are measurable and all three cannot be met at the same time on the same model.

This lesson builds those three criteria and counts the conflict between them. The result is not a tuning error: as long as the base rates of the two groups differ, the definitions of the criteria exclude each other. This is the second claim of the course, and here it is shown with an identity.

• ER9. The set, split, model and subgroup definitions carry over unchanged from the previous lesson: a training set of 756 rows, a held-out set of 504 subscribers, a twelve-column linear model, thirteen subgroups.
• ER10. The two groups in which the conflict is measured are taken from the region axis: A is the east region, B is the remaining four regions. The reason for the choice is the base rate difference; the west region cannot be taken alone, because it has a single suspicious case in the held-out set.
• ER11. Demographic parity is the equality of the rates of flagged subscribers in the two groups.
• ER12. Equal opportunity is the equality of recall in the two groups: how much of the suspicious cases is flagged.
• ER13. Predictive parity is the equality of precision in the two groups: how much of what is flagged is actually suspicious.
• ER14. The means of satisfying a criterion is giving the group a separate threshold. The model, its weights and its scores do not change at all; only the score above which a subscriber is flagged changes.
• ER15. Threshold pairs are searched on a grid from 0.02 to 0.98 in steps of 0.01 and the total number of flagged subscribers is held fixed (63 subscribers, a shift of at most three). Without this constraint every criterion is satisfied by flagging no one and the comparison becomes meaningless.
• ER16. A gap is the absolute difference between the numbers of the two groups; rather than how close it gets to zero, what is read is what the other two gaps do.

Three Criteria, Three Different Equalities

The setup reproduces the previous lesson’s model, gives each subscriber in the held-out set a score and puts the five regions side by side at a single threshold.

# fairness.py — MODEL: The same FICTIONAL subscriber table, the same seed, the same split and
# the previous lesson's twelve-column linear model; nothing changes except the threshold.
import math
import statistics

SEED, RAW, M32 = 20260218, 1400, 0xFFFFFFFF
REGION = [("north", 0.28, 21, 0.00), ("south", 0.22, 17, -0.10),
          ("east", 0.18, 26, 0.30), ("west", 0.14, 14, -0.05),
          ("central", 0.18, 23, 0.05)]


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


SUBSCRIBERS, DATA = [], []
for i in range(RAW):
    r = rng(SEED + i)
    b = REGION[pick(r(), [x[1] for x in REGION])]
    household = 1 + pick(r(), [0.06, 0.24, 0.30, 0.24, 0.11, 0.05])
    rating = 1 + pick(r(), [0.08, 0.14, 0.27, 0.34, 0.17])
    if r() >= 0.046:
        SUBSCRIBERS.append({"id": 10001 + i, "region": b, "household": household,
                            "satisfaction": rating})
for k in SUBSCRIBERS:
    r, v = rng(SEED + 7000 + k["id"]), []
    for n in range(pick(r(), [0.05, 0.12, 0.21, 0.62])):
        v.append(0.0 if r() < 0.038 else math.floor(
            k["region"][2] * math.exp((r() + r() + r() - 1.5) * 0.62)
            * (1 - n * 0.05) * 100 + 0.5) / 100)
        r()
    if not v:
        continue
    mean, vol, r = sum(v) / len(v), max(v) - min(v), rng(SEED + 51000 + k["id"])
    z = (0.052 * (mean - 20) + 0.85 * (min(v) == 0.0) + 0.026 * vol
         + 0.24 * (k["household"] >= 5) - 0.20 * (k["satisfaction"] >= 4) + k["region"][3]
         + 0.9 * (k["id"] - 10001) / 1399 + (r() + r() + r() - 1.5) * 1.30)
    x = {"household": k["household"], "satisfaction": k["satisfaction"], "period": len(v),
         "mean_consumption": round(mean, 2), "volatility": round(vol, 2),
         "region_name": k["region"][0], "zero_readings": int(min(v) == 0.0),
         "suspicious": int(z > 1.35)}
    x["per_person"] = round(x["mean_consumption"] / x["household"], 3)
    for b in REGION:
        x["area_" + b[0]] = int(k["region"][0] == b[0])
    DATA.append(x)


def shuffle(data, seed):
    r, s = rng(seed), list(range(len(data)))
    for i in range(len(s) - 1, 0, -1):
        j = int(r() * (i + 1))
        s[i], s[j] = s[j], s[i]
    return [data[i] for i in s]


K = shuffle(DATA, SEED + 90000)
TRAIN, HELD_OUT = K[:756], K[756:]
FIELDS = ["mean_consumption", "volatility", "household", "satisfaction", "period",
          "zero_readings", "per_person", "area_north", "area_south", "area_east",
          "area_west", "area_central"]
MEAN = {a: statistics.fmean(x[a] for x in TRAIN) for a in FIELDS}
STD = {a: max(1e-9, statistics.pstdev([x[a] for x in TRAIN])) for a in FIELDS}


def scaled(x):
    return [1.0] + [(x[a] - MEAN[a]) / STD[a] for a in FIELDS]


def squash(z):
    if z >= 0:
        return 1.0 / (1.0 + math.exp(-z)) if z < 700 else 1.0
    e = math.exp(z) if z > -700 else 0.0
    return e / (1.0 + e)


def train_model(rows, label):            # MODEL: linear model, gradient descent
    X = [scaled(x) for x in rows]
    Y = [x[label] for x in rows]
    w = [0.0] * len(X[0])
    for _ in range(200):
        g = [0.0] * len(w)
        for i in range(len(X)):
            h = squash(sum(w[j] * X[i][j] for j in range(len(w)))) - Y[i]
            for j in range(len(w)):
                g[j] += h * X[i][j]
        for j in range(len(w)):
            w[j] -= 0.3 * g[j] / len(X)
    return w


def probability(w, x):
    return squash(sum(a * b for a, b in zip(w, scaled(x))))


def groups(x):
    t = 10 if x["mean_consumption"] <= 10 else 25 if x["mean_consumption"] <= 25 else 40
    return ["region " + x["region_name"], "tariff %d" % t,
            "household " + ("1-2" if x["household"] <= 2 else "3-4" if x["household"] <= 4 else "5+"),
            "readings " + ("few" if x["period"] <= 2 else "full")]


W = train_model(TRAIN, "suspicious")
SCORES = [(probability(W, x), x["suspicious"], x["region_name"]) for x in HELD_OUT]
A = [p for p in SCORES if p[2] == "east"]
B = [p for p in SCORES if p[2] != "east"]


def measure(s, e):                       # selection, recall, precision, correct, flagged
    susp = sum(y for _, y, _ in s)
    n = sum(1 for p, _, _ in s if p > e)
    caught = sum(y for p, y, _ in s if p > e)
    return (n / len(s), caught / susp, caught / n if n else float("nan"),
            sum(1 for p, y, _ in s if (p > e) == bool(y)), n)


print(f"train {len(TRAIN)}, held-out {len(HELD_OUT)}, subgroups examined "
      f"{len({g for x in HELD_OUT for g in groups(x)})}, baseline "
      f"{sum(x['suspicious'] == 0 for x in HELD_OUT) / len(HELD_OUT):.4f}")
print(f"{'region':<10}{'n':>5}{'suspicious':>11}{'base rate':>12}{'selection':>10}"
      f"{'recall':>12}{'precision':>10}")
for b in ("north", "south", "east", "west", "central"):
    s = [p for p in SCORES if p[2] == b]
    m = measure(s, 0.5)
    print(f"{b:<10}{len(s):>5}{sum(y for _, y, _ in s):>11}"
          f"{sum(y for _, y, _ in s) / len(s):>12.4f}{m[0]:>10.4f}{m[1]:>12.4f}{m[2]:>10.4f}")
base_a = sum(y for _, y, _ in A) / len(A)
base_b = sum(y for _, y, _ in B) / len(B)
a0, b0 = measure(A, 0.5), measure(B, 0.5)
print(f"\nA = east: n {len(A)}, base rate {base_a:.4f} | B = other four regions: "
      f"n {len(B)}, base rate {base_b:.4f}, ratio of rates {base_a / base_b:.4f}")
print(f"single threshold 0.50, flagged {a0[4] + b0[4]}: parity gap "
      f"{abs(a0[0] - b0[0]):.4f}, opportunity gap {abs(a0[1] - b0[1]):.4f}, "
      f"predictive gap {abs(a0[2] - b0[2]):.4f}")

train 756, held-out 504, subgroups examined 13, baseline 0.7857
region        n suspicious   base rate selection      recall precision
north       144         23      0.1597    0.0694      0.1739    0.4000
south       117         17      0.1453    0.0256      0.1176    0.6667
east         83         38      0.4578    0.4578      0.6842    0.6842
west         63          1      0.0159    0.0000      0.0000       nan
central      97         29      0.2990    0.1237      0.2759    0.6667

A = east: n 83, base rate 0.4578 | B = other four regions: n 421, base rate 0.1663, ratio of rates 2.7535
single threshold 0.50, flagged 63: parity gap 0.3984, opportunity gap 0.4842, predictive gap 0.1242

The column of the table that needs to be read is the base rate: the share of suspicious cases is 0.4578 in the east and 0.0159 in the west. This difference is not something the model produced but something that sits inside the fiction; the hidden score of the east region is 0.30 higher. The model sees this difference and flags 0.4578 of the subscribers in the east and 0.0256 in the south.

The bottom line is the starting point of the lesson. With a single threshold all three gaps are far from zero: parity gap 0.3984, opportunity gap 0.4842, predictive gap 0.1242. Which one to fix becomes a choice, because the three look from separate sides: demographic parity asks that a subscriber not be flagged more often than the neighbors, equal opportunity asks that in no region suspicious cases be missed more than in another, and predictive parity asks that being flagged mean the same thing in every region.

Fixing One Breaks the Others

The way to satisfy a criterion is to give the two groups separate thresholds. The search below, holding the total number of flagged subscribers fixed, pulls each criterion one at a time to the point nearest zero and writes the other two gaps next to it.

GRID = [i / 100 for i in range(2, 99)]
CA = {e: measure(A, e) for e in GRID}
CB = {e: measure(B, e) for e in GRID}
TARGET = a0[4] + b0[4]


def equalize(k):                         # k: 0 selection, 1 recall, 2 precision
    best = None
    for ea in GRID:
        u = CA[ea]
        for eb in GRID:
            v = CB[eb]
            if abs(u[4] + v[4] - TARGET) > 3 or v[2] != v[2] or u[2] != u[2]:
                continue
            gap, acc = abs(u[k] - v[k]), (u[3] + v[3]) / len(HELD_OUT)
            if best is None or gap < best[0] - 1e-12 or (abs(gap - best[0]) < 1e-12 and acc > best[3]):
                best = (gap, ea, eb, acc)
    return best


def show_row(name, ea, eb):
    u, v = CA[ea], CB[eb]
    print(f"{name:<22}{ea:>12.2f}{eb:>12.2f}{u[4] + v[4]:>12}"
          f"{abs(u[0] - v[0]):>12.4f}{abs(u[1] - v[1]):>16.4f}{abs(u[2] - v[2]):>15.4f}"
          f"{(u[3] + v[3]) / len(HELD_OUT):>10.4f}")


print(f"{'configuration':<22}{'threshold A':>12}{'threshold B':>12}{'flagged':>12}"
      f"{'parity gap':>12}{'opportunity gap':>16}{'predictive gap':>15}{'accuracy':>10}")
show_row("single threshold", 0.5, 0.5)
for k, name in ((0, "demographic parity"), (1, "equal opportunity"), (2, "predictive parity")):
    gap, ea, eb, acc = equalize(k)
    show_row(name, ea, eb)

configuration          threshold A threshold B     flagged  parity gap opportunity gap predictive gap  accuracy
single threshold              0.50        0.50          63      0.3984          0.4842         0.1242    0.8194
demographic parity            0.81        0.40          60      0.0017          0.0632         0.4800    0.7857
equal opportunity             0.76        0.40          63      0.0379          0.0105         0.4262    0.7877
predictive parity             0.34        0.65          66      0.6365          0.7038         0.0000    0.7976

In each row the number on the diagonal falls to zero and the others rise. If demographic parity is wanted, the east is given a threshold of 0.81 and the rest 0.40; the rates of flagged subscribers are equalized with a difference of 0.0017, but the predictive gap rises from 0.1242 to 0.4800 — 0.9000 of the subscribers flagged in the east are actually suspicious, while in the rest 0.4200 are. If equal opportunity is wanted, the thresholds become 0.76 and 0.40, the recalls meet with a difference of 0.0105, and the predictive gap stays at 0.4262. If predictive parity is wanted, the east is given 0.34 and the rest 0.65; the precisions are equalized exactly, but the opportunity gap shoots up to 0.7038, because 0.7895 of the suspicious cases are caught in the east and 0.0857 in the rest.

Part of the cost comes from accuracy: overall accuracy falls to 0.7857, 0.7877 and 0.7976 in the three configurations, and under demographic parity the model sits exactly on the baseline. But the real cost is not in accuracy but in the other two gaps; no configuration can shrink all three columns at once.

This Is Not a Tuning Error

The pattern in the table is not fixed by a better search or a finer grid, because its source is the definitions themselves. The three criteria are not independent; there is an identity among them. If in a group the selection rate is $s$, recall $d$, precision $k$ and base rate $t$, the number of correct flags can be written in two ways and

$$s = \frac{d \cdot t}{k}$$

follows. The block below first verifies this identity in the two groups, then tries to equalize recall and precision together.

print(f"\nidentity: selection = recall * base rate / precision   (threshold 0.50)")
for name, m, t in (("A", a0, base_a), ("B", b0, base_b)):
    print(f"  {name}: {m[1] * t / m[2]:.6f} = {m[0]:.6f}")


def jointly(tol):
    best = None
    for ea in GRID:
        u = CA[ea]
        for eb in GRID:
            v = CB[eb]
            if v[2] != v[2] or u[2] != u[2]:
                continue
            if tol and abs(u[4] + v[4] - TARGET) > tol:
                continue
            gap = max(abs(u[1] - v[1]), abs(u[2] - v[2]))
            if best is None or gap < best[0] - 1e-12:
                best = (gap, ea, eb)
    return best


SUSP = sum(y for _, y, _ in SCORES)
print(f"\nattempt to equalize recall and precision jointly ({SUSP} suspicious)")
print(f"{'capacity constraint':<20}{'threshold A':>12}{'threshold B':>12}{'largest gap':>18}"
      f"{'flagged':>10}{'caught':>11}{'selection ratio':>16}")
for tol, name in ((0, "none"), (3, "63 shift 3"), (15, "63 shift 15")):
    gap, ea, eb = jointly(tol)
    u, v = CA[ea], CB[eb]
    caught = round(u[1] * sum(y for _, y, _ in A)) + round(v[1] * sum(y for _, y, _ in B))
    print(f"{name:<20}{ea:>12.2f}{eb:>12.2f}{gap:>18.4f}{u[4] + v[4]:>10}{caught:>11}"
          f"{u[0] / v[0]:>16.4f}")
print(f"ratio of base rates {base_a / base_b:.4f}")

identity: selection = recall * base rate / precision   (threshold 0.50)
  A: 0.457831 = 0.457831
  B: 0.059382 = 0.059382

attempt to equalize recall and precision jointly (108 suspicious)
capacity constraint  threshold A threshold B       largest gap   flagged     caught selection ratio
none                        0.86        0.82            0.0120         2          2          5.0723
63 shift 3                  0.74        0.43            0.2727        60         33          1.8445
63 shift 15                 0.72        0.46            0.2486        48         29          2.7816
ratio of base rates 2.7535

The identity holds in both groups to the sixth decimal place. Its consequence is this: if recall and precision are equalized in the two groups, then $s_A / s_B = t_A / t_B$, that is, the ratio of selection rates locks to the ratio of base rates, here 2.7535. Demographic parity asks for this ratio to be 1. The two can be true together only if the base rates are equal or if no one is flagged.

The bottom table shows this by experiment. When the capacity constraint is removed, the search does find a threshold pair that makes all three gaps small at once — 0.86 and 0.82 — but at that point only two of the 504 subscribers are flagged and two of the 108 suspicious cases are caught. The equality has been achieved by doing nothing. When capacity is fixed at 63, the best attempt leaves a gap of 0.2727; under a looser constraint of 48 subscribers the gap falls to 0.2486 and the ratio of selection rates settles at 2.7816, next to the ratio of base rates, 2.7535 — where the identity predicts.

The rule that follows from this is not a recommendation but a limit: as long as the base rates differ, the three criteria cannot be satisfied at the same time. The work to be done is choosing a criterion, writing the choice down and reporting the other two gaps together with their numbers; there is no single number called “a fair model”.

Summary

• The share of suspicious cases is 0.4578 in the east region and 0.1663 in the remaining four regions; this base rate difference determines all the results of the lesson, and at a single threshold all three gaps are large: parity 0.3984, opportunity 0.4842, predictive 0.1242.
• When the number of flagged subscribers is held fixed, demographic parity raises the predictive gap to 0.4800, equal opportunity raises it to 0.4262 and predictive parity raises the opportunity gap to 0.7038; overall accuracy falls to between 0.7857 and 0.7976.
• The selection rate equals the product of recall and base rate divided by precision; when two criteria are equalized, the ratio of selection rates locks to the ratio of base rates, 2.7535.
• With capacity fixed there is no threshold pair that gives all three together: the best attempt leaves a gap of 0.2727, and when the constraint is removed the equality is achieved only by flagging two subscribers.

Next Step

In two lessons, what was measured was what the model did to whom. What the model says about the records it was trained on was never asked. How much does a single subscriber entering or leaving the training table move the output, and does deleting the subscriber ID really make that subscriber invisible? The next lesson counts these: the trace a single record leaves on the output is measured, what adding noise takes from accuracy is swept, and how many subscribers stay alone in their own field combination in a anonymized table is counted.