Diversity and the Echo Chamber

The previous lesson had each setup look at the log once. In a real system the decision repeats: the item shown produces an interaction, the interaction grows the log, and the growing log determines the next impression. This lesson runs that loop for forty-eight periods and counts where the coverage goes.

• RS57. The setup is the same block built in the first lesson; the starting log of the loop is the 3,002 impressions and 577 interactions of that setup.
• RS58. Each period the system shows each subscriber five items and estimates the score as the interaction rate per impression of that item within the subscriber’s group: (interactions + 0.5) / (impressions + 5). The group is the dominant domain in the subscriber’s past and is the only subscriber feature the system sees.
• RS59. Two setups are compared. The narrow setup gives all five slots to the estimate and is the baseline, since it contains no diversity rule. The wide setup gives four to the estimate and reserves the fifth for the item shown least until then.
• RS60. In the twelfth period the response rate of one item rises eightfold and hits the 0.95 ceiling; this is the moment the environment changes, and no setup is told.
• RS61. Three numbers are written each period: interactions per subscriber (higher is better), the number of items not shown at all that period (lower is better) and the share of impressions on the five most shown items (lower is better) — the last is the measure of popularity bias. Selection bias is in the Applied Statistics course.

The Setup Fed by Its Own Log

# COMMON -- FICTIONAL. The subscriber x service item interaction matrix is built inside the lesson.
# Impression, selection and score are generated separately: an item is first SHOWN, then
# it can be selected, and only if it is selected can it be scored.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


ITEMS = [  # (name, domain, channel, paid, commitment, appointment, base_share)
    ("auto_payment_instruction", "billing", "online", 0, 0, 0, 0.90),
    ("e_invoice_switch", "alert", "online", 0, 0, 0, 0.85),
    ("payment_reminder_sms", "alert", "online", 0, 0, 0, 0.78),
    ("installment_plan", "billing", "branch", 0, 1, 0, 0.62),
    ("remote_reading_application", "meter", "field", 0, 1, 1, 0.55),
    ("outage_alert_subscription", "alert", "online", 0, 0, 0, 0.50),
    ("debt_restructuring", "billing", "branch", 0, 1, 0, 0.44),
    ("address_update", "subscription", "online", 0, 0, 0, 0.40),
    ("meter_replacement", "meter", "field", 1, 0, 1, 0.36),
    ("consumption_report_subscription", "alert", "online", 0, 0, 0, 0.32),
    ("invoice_dispute_record", "billing", "branch", 0, 0, 0, 0.28),
    ("subscription_transfer", "subscription", "branch", 0, 0, 0, 0.25),
    ("illicit_use_check_appointment", "meter", "field", 0, 0, 1, 0.22),
    ("installation_maintenance_package", "technical", "field", 1, 1, 1, 0.19),
    ("second_meter_request", "meter", "field", 1, 0, 1, 0.16),
    ("business_tariff_switch", "subscription", "branch", 0, 1, 0, 0.14),
    ("pressure_regulator_mounting", "technical", "field", 1, 0, 1, 0.12),
    ("garden_irrigation_tariff", "subscription", "branch", 0, 1, 0, 0.10),
    ("meter_relocation", "meter", "field", 1, 0, 1, 0.09),
    ("common_area_sharing_record", "subscription", "branch", 0, 1, 0, 0.08),
    ("installation_compliance_report", "technical", "field", 1, 0, 1, 0.07),
    ("late_interest_waiver", "billing", "branch", 0, 0, 0, 0.06),
    ("mobile_access_permission", "alert", "online", 0, 0, 0, 0.05),
    ("extra_line_installation", "technical", "field", 1, 1, 1, 0.04),
]
SEGMENT = {  # propensity of the segment toward the domains; used only inside the generator
    "saver":    {"billing": 0.5, "meter": 1.7, "subscription": 0.6, "alert": 1.5, "technical": 0.6},
    "payer":    {"billing": 1.9, "meter": 0.5, "subscription": 0.6, "alert": 1.1, "technical": 0.3},
    "business": {"billing": 0.7, "meter": 0.9, "subscription": 1.6, "alert": 0.7, "technical": 1.9},
    "new":      {"billing": 0.8, "meter": 0.7, "subscription": 1.8, "alert": 1.3, "technical": 0.7},
}
SEGMENT_NAMES = list(SEGMENT)
rq = rng(SEED + 101)
QUALITY = [0.25 + 0.70 * rq() for _ in ITEMS]   # the item's own satisfaction

N_SUBSCRIBERS, N_ITEMS = 360, len(ITEMS)
SUBSCRIBERS, IMPRESSIONS, INTERACTIONS, SCORES, PROPENSITY = [], {}, {}, {}, {}
for i in range(N_SUBSCRIBERS):
    r = rng(SEED + 7919 * i)
    a, segment = 20001 + i, SEGMENT_NAMES[int(r() * 4)]
    activity = 0.35 + 1.15 * r() ** 1.6
    SUBSCRIBERS.append(a)
    for j, (name, domain, channel, paid, commitment, appointment, share) in enumerate(ITEMS):
        propensity = min(0.95, SEGMENT[segment][domain] * share * activity * 0.55)
        PROPENSITY[(a, j)] = propensity
        if r() < min(0.95, share ** 0.55 * 0.72):        # was it shown
            IMPRESSIONS[(a, j)] = 1
            if r() < propensity:                          # was it selected
                INTERACTIONS[(a, j)] = r()                # value: sequence stamp
                if r() < 0.32:                            # did it leave a score
                    raw = (0.16 + 0.50 * SEGMENT[segment][domain] / 1.9
                           + 0.42 * QUALITY[j] + 0.26 * (r() - 0.5))
                    SCORES[(a, j)] = max(1, min(5, int(round(raw * 5))))
        else:
            r(), r(), r()
DOMAIN_OF = [ITEMS[j][1] for j in range(N_ITEMS)]
COUNTS = {a: {} for a in SUBSCRIBERS}
for (a, j) in INTERACTIONS:                         # subscriber group: dominant domain in the past
    COUNTS[a][DOMAIN_OF[j]] = COUNTS[a].get(DOMAIN_OF[j], 0) + 1
GROUP_OF = {a: (max(sorted(counts), key=lambda x: counts[x]) if counts else "none")
            for a, counts in COUNTS.items()}
PERIODS, K, SHIFT, RISING = 48, 5, 12, 9            # in period 12 the response of item 9 changes


def run_loop(wide):
    """The system is fed by its own log: it measures what it shows, it shows what it measures."""
    shown, selected = {}, {}
    for (a, j) in IMPRESSIONS:
        shown[(GROUP_OF[a], j)] = shown.get((GROUP_OF[a], j), 0) + 1
    for (a, j) in INTERACTIONS:
        selected[(GROUP_OF[a], j)] = selected.get((GROUP_OF[a], j), 0) + 1
    total, rows = [0] * N_ITEMS, []
    for d in range(1, PERIODS + 1):
        period_shown, period_inter = [0] * N_ITEMS, 0
        for a in SUBSCRIBERS:
            g = GROUP_OF[a]
            order = sorted(range(N_ITEMS),
                           key=lambda j: (-(selected.get((g, j), 0) + 0.5)
                                          / (shown.get((g, j), 0) + 5.0), j))
            top = order[:K] if not wide else order[:K - 1] + [        # fifth slot to diversity
                min((j for j in range(N_ITEMS) if j not in order[:K - 1]),
                    key=lambda j: (total[j], (j + a + d) % N_ITEMS))]
            r = rng(SEED + 660000 + a * 53 + d * 1301)
            for j in top:
                period_shown[j] += 1
                total[j] += 1
                shown[(g, j)] = shown.get((g, j), 0) + 1
                e = PROPENSITY[(a, j)]
                if j == RISING and d >= SHIFT:
                    e = min(0.95, e * 8.0)
                if r() < e:
                    period_inter += 1
                    selected[(g, j)] = selected.get((g, j), 0) + 1
        top5 = sorted(range(N_ITEMS), key=lambda j: -period_shown[j])[:5]
        rows.append((period_inter / len(SUBSCRIBERS),
                     [j for j in range(N_ITEMS) if period_shown[j] == 0],
                     sum(period_shown[j] for j in top5) / sum(period_shown),
                     period_shown[RISING]))
    return rows


NARROW = run_loop(False)
print(f"narrow setup: top {K} every period, {PERIODS} periods, {len(SUBSCRIBERS)} subscribers")
print(f"in period {SHIFT} the response rate of {ITEMS[RISING][0]} rises eightfold")
print("period  interactions/subscriber  items never shown that period  top-five impression share")
for d in (1, 8, 24, 48):
    e, never, p, n = NARROW[d - 1]
    print(f"  {d:>2}        {e:.4f}                        {len(never):>2}/{N_ITEMS}                    {p:.4f}")
LOCKED = set(range(N_ITEMS))
for x in NARROW[PERIODS // 2:]:
    LOCKED &= set(x[1])
print(f"items not shown in ALL of the last {PERIODS - PERIODS // 2} periods: {len(LOCKED)}/{N_ITEMS}")

narrow setup: top 5 every period, 48 periods, 360 subscribers
in period 12 the response rate of consumption_report_subscription rises eightfold
period  interactions/subscriber  items never shown that period  top-five impression share
   1        1.2250                         0/24                    0.4711
   8        1.3917                         8/24                    0.5461
  24        1.4583                         8/24                    0.5461
  48        1.3194                         8/24                    0.5461
items not shown in ALL of the last 24 periods: 6/24

In the first period all 24 items are shown. In the eighth period eight items drop out; the number freezes at 8/24 there and does not move until the forty-eighth period. Six items are shown in none of the last twenty-four periods. Popularity bias is the other face of the same movement: the impression share of the top five items rises from 0.4711 to 0.5461 and stays there.

The loop closes on itself. No new information is born about an item that is not shown, its estimate stays low, and the low estimate leaves it out again in the next period: the item stays out not because it is bad but because it is not measured. In the twelfth period the environment changes but the narrow setup cannot see it, because the only way to tell the system is for that item to be shown.

Giving the Fifth Slot to Diversity

WIDE = run_loop(True)
WIDE_LOCKED = set(range(N_ITEMS))
for x in WIDE[PERIODS // 2:]:
    WIDE_LOCKED &= set(x[1])
print(f"wide setup: top {K - 1} by model, fifth slot to the least shown item")
print("period  interactions/subscriber  items never shown that period  top-five impression share  shifted item")
for d in (1, 8, 24, 48):
    e, never, p, n = WIDE[d - 1]
    print(f"  {d:>2}        {e:.4f}                        {len(never):>2}/{N_ITEMS}                    "
          f"{p:.4f}            {n:>3}")
cost = sum(WIDE[d][0] - NARROW[d][0] for d in range(SHIFT - 1))
gain = sum(WIDE[d][0] - NARROW[d][0] for d in range(SHIFT - 1, PERIODS))
equal = [d + 1 for d in range(PERIODS)
         if sum(x[0] for x in WIDE[:d + 1]) > sum(x[0] for x in NARROW[:d + 1])]
print(f"short-term cost (total of periods 1-{SHIFT - 1}): per subscriber {cost:+.4f}")
print(f"long-term gain (total of periods {SHIFT}-{PERIODS}): per subscriber {gain:+.4f}")
print(f"cumulative break-even period: {equal[0] if equal else 'none'}; at period {PERIODS} the "
      f"cumulative difference is {sum(x[0] for x in WIDE) - sum(x[0] for x in NARROW):+.4f}")
print(f"the shifted item at period {PERIODS}: narrow setup {NARROW[-1][3]} impressions, "
      f"wide setup {WIDE[-1][3]}")
print(f"data collected -- narrow: {len(LOCKED)} items never shown, top-five impression share "
      f"{NARROW[-1][2]:.4f}; wide: {len(WIDE_LOCKED)} items, {WIDE[-1][2]:.4f}")

wide setup: top 4 by model, fifth slot to the least shown item
period  interactions/subscriber  items never shown that period  top-five impression share  shifted item
   1        1.1833                         0/24                    0.4611             37
   8        1.2722                         0/24                    0.5011             81
  24        1.4806                         0/24                    0.4989             97
  48        1.3889                         0/24                    0.4989             97
short-term cost (total of periods 1-11): per subscriber -1.0000
long-term gain (total of periods 12-48): per subscriber +2.2250
cumulative break-even period: 27; at period 48 the cumulative difference is +1.2250
the shifted item at period 48: narrow setup 17 impressions, wide setup 97
data collected -- narrow: 6 items never shown, top-five impression share 0.5461; wide: 0 items, 0.4989

The cost is paid in the first eleven periods: the wide setup finishes 1.0000 interactions per subscriber behind, because one of the five slots goes not to the best estimate but to the least known item. In return coverage never narrows and the share of the top five items stays at 0.4989.

In the twelfth period the account turns. Since the wide setup already shows the item whose response changed in its rotating slot, it notices the change in the same period. In the forty-eighth period that item receives 97 impressions in the wide setup and 17 in the narrow setup. The total gain between the twelfth and the forty-eighth is +2.2250 per subscriber; together with the −1.0000 cost of the first eleven periods, the cumulative difference reaches zero in the twenty-seventh period and becomes +1.2250 in the forty-eighth. Diversity is not a matter of taste but a measurement budget: a system that never measures wins as long as the environment stays constant and loses every time it changes.

“Echo chamber” is a metaphor that describes this table and has no number of its own; its counterpart is the number of items not shown and the impression share of the top five. The place where it misleads is this: in the setup no subscriber’s propensity changes. What narrows is not the interest of the subscriber but the domain the system measures, and an item that is not measured being counted as bad is not a belief but a gap in the record.

Summary

• After the eighth period the narrow setup does not show 8 of 24 items in any period, 6 items appear in none of the last twenty-four periods, and the impression share of the top five rises from 0.4711 to 0.5461.
• The short-term cost of diversity is −1.0000 per subscriber in the first eleven periods and its long-term gain is +2.2250 between the twelfth and forty-eighth; the cumulative difference breaks even in the twenty-seventh period.
• The echo chamber is a metaphor; in this setup what narrows is not the interest of the subscriber but the domain the system measures.

Course Wrap-Up

The account of fourteen lessons is in a single table, and each number was read from its own lesson. The rows do not carry a common unit; the last column writes the metric and its direction. The difference is given in a single sign convention: the method’s number minus the baseline and is in parentheses in the third column. The fourth column is the course’s own number: the unseen state, the untried action, the item not shown.

The rule of the course is this: the measure of a system that makes decisions cannot be read without accounting for the fact that those decisions change its own data; a system that does not write down the data it collected counts as unmeasured.

The fourth column reduces to a single number: the total of the unseen pairs, the cells not updated, the subscribers left unvisited and the items not shown in the twelve lessons that print a count is 3,203 units. The units are not the same and the total is a count, not a mean: behind every performance number measured in this course stand more than three thousand units about which the method learned nothing. Reinforcement 02 and 06 print no count and do not enter the sum.

The second claim of the course is read from the same table: a number measured offline does not tell online behavior. In four lessons the two numbers diverged. In Recommender 06 the setup that was first offline with 0.6886 was last with 1.4563 when it made its own impression decisions, and fell below the baseline. In Recommender 07 the narrow setup that looked good on the log finished forty-eight periods 1.2250 behind. In Recommender 05 the hit rate read from the log fell from 0.5752 to 0.5487 once questions were asked. In Reinforcement 03 the divergence appeared in the opposite direction: the behavior of the agent with exploration rate 1.00 was bad but the data it collected was good, and the frozen table gave 23.000. There is a single source: the log the number is read from is a log whose decisions were made by another setup.

In fifty-two lessons the baseline was written beside every number, and in the last fourteen it was counted that the system changes its own data. The only thing never asked is for whom it is measured. Every number in the table is the mean over all subscribers: if the same number says two different things in two subgroups, which one is reported; what is the basis on which the system shows an item to one person and not to another, and can it be read; and with which number is the harm written that a system with a good measure does to a group that never enters the measure. The next course, Model Evaluation, Interpretability and Ethics, opens with these three questions.