--- title: 'Diversity and the Echo Chamber' source: 'https://academia.sh/en/courses/reinforcement-and-recommender/diversity-and-echo-chamber' course: 'Reinforcement Learning and Recommender Systems' language: en updated: '2026-10-21T05:03:28+00:00' license: 'CC BY-SA 4.0' objectives: - 'Runs the feedback loop for forty-eight periods and counts the periods in which coverage narrows and popularity bias grows' - 'Compares the short-term cost and the long-term gain of giving the fifth recommendation slot to diversity on the same measure' - 'Distinguishes which number the echo chamber metaphor corresponds to in this setup and which it does not' --- # Diversity and the Echo Chamber The narrowing of coverage and the growth of popularity bias when the system is fed by its own recommendations: over forty-eight periods the narrow setup after the eighth period never shows 8 of 24 items in any period, six items appear in none of the last twenty-four periods, and the impression share of the top five items rises from 0.4711 to 0.5461. The setup that gives the fifth slot to diversity loses 1.0000 interactions per subscriber in the first eleven periods, finds the item whose conditions changed in the twelfth period and moves ahead, breaks even cumulatively in the twenty-seventh period and finishes the forty-eighth period 1.2250 ahead. The previous lesson had each setup look at the log once. In a real system the decision repeats: the item shown produces an interaction, the interaction grows the log, and the growing log determines the next impression. This lesson runs that loop for forty-eight periods and counts where the coverage goes. - **RS57. The setup is the same block built in the first lesson**; the starting log of the loop is the 3,002 impressions and 577 interactions of that setup. - **RS58. Each period the system shows each subscriber five items** and estimates the score as the interaction rate per impression of that item within the subscriber's group: `(interactions + 0.5) / (impressions + 5)`. The group is the dominant domain in the subscriber's past and is the only subscriber feature the system sees. - **RS59. Two setups are compared.** The narrow setup gives all five slots to the estimate and is **the baseline**, since it contains no diversity rule. The wide setup gives four to the estimate and reserves the fifth for the item shown least until then. - **RS60. In the twelfth period the response rate of one item rises eightfold** and hits the 0.95 ceiling; this is the moment the environment changes, and no setup is told. - **RS61. Three numbers are written each period:** interactions per subscriber (higher is better), the number of items not shown at all that period (lower is better) and the share of impressions on the five most shown items (lower is better) — the last is the measure of **popularity bias**. Selection bias is in the Applied Statistics course. ## The Setup Fed by Its Own Log ```python # COMMON -- FICTIONAL. The subscriber x service item interaction matrix is built inside the lesson. # Impression, selection and score are generated separately: an item is first SHOWN, then # it can be selected, and only if it is selected can it be scored. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value ITEMS = [ # (name, domain, channel, paid, commitment, appointment, base_share) ("auto_payment_instruction", "billing", "online", 0, 0, 0, 0.90), ("e_invoice_switch", "alert", "online", 0, 0, 0, 0.85), ("payment_reminder_sms", "alert", "online", 0, 0, 0, 0.78), ("installment_plan", "billing", "branch", 0, 1, 0, 0.62), ("remote_reading_application", "meter", "field", 0, 1, 1, 0.55), ("outage_alert_subscription", "alert", "online", 0, 0, 0, 0.50), ("debt_restructuring", "billing", "branch", 0, 1, 0, 0.44), ("address_update", "subscription", "online", 0, 0, 0, 0.40), ("meter_replacement", "meter", "field", 1, 0, 1, 0.36), ("consumption_report_subscription", "alert", "online", 0, 0, 0, 0.32), ("invoice_dispute_record", "billing", "branch", 0, 0, 0, 0.28), ("subscription_transfer", "subscription", "branch", 0, 0, 0, 0.25), ("illicit_use_check_appointment", "meter", "field", 0, 0, 1, 0.22), ("installation_maintenance_package", "technical", "field", 1, 1, 1, 0.19), ("second_meter_request", "meter", "field", 1, 0, 1, 0.16), ("business_tariff_switch", "subscription", "branch", 0, 1, 0, 0.14), ("pressure_regulator_mounting", "technical", "field", 1, 0, 1, 0.12), ("garden_irrigation_tariff", "subscription", "branch", 0, 1, 0, 0.10), ("meter_relocation", "meter", "field", 1, 0, 1, 0.09), ("common_area_sharing_record", "subscription", "branch", 0, 1, 0, 0.08), ("installation_compliance_report", "technical", "field", 1, 0, 1, 0.07), ("late_interest_waiver", "billing", "branch", 0, 0, 0, 0.06), ("mobile_access_permission", "alert", "online", 0, 0, 0, 0.05), ("extra_line_installation", "technical", "field", 1, 1, 1, 0.04), ] SEGMENT = { # propensity of the segment toward the domains; used only inside the generator "saver": {"billing": 0.5, "meter": 1.7, "subscription": 0.6, "alert": 1.5, "technical": 0.6}, "payer": {"billing": 1.9, "meter": 0.5, "subscription": 0.6, "alert": 1.1, "technical": 0.3}, "business": {"billing": 0.7, "meter": 0.9, "subscription": 1.6, "alert": 0.7, "technical": 1.9}, "new": {"billing": 0.8, "meter": 0.7, "subscription": 1.8, "alert": 1.3, "technical": 0.7}, } SEGMENT_NAMES = list(SEGMENT) rq = rng(SEED + 101) QUALITY = [0.25 + 0.70 * rq() for _ in ITEMS] # the item's own satisfaction N_SUBSCRIBERS, N_ITEMS = 360, len(ITEMS) SUBSCRIBERS, IMPRESSIONS, INTERACTIONS, SCORES, PROPENSITY = [], {}, {}, {}, {} for i in range(N_SUBSCRIBERS): r = rng(SEED + 7919 * i) a, segment = 20001 + i, SEGMENT_NAMES[int(r() * 4)] activity = 0.35 + 1.15 * r() ** 1.6 SUBSCRIBERS.append(a) for j, (name, domain, channel, paid, commitment, appointment, share) in enumerate(ITEMS): propensity = min(0.95, SEGMENT[segment][domain] * share * activity * 0.55) PROPENSITY[(a, j)] = propensity if r() < min(0.95, share ** 0.55 * 0.72): # was it shown IMPRESSIONS[(a, j)] = 1 if r() < propensity: # was it selected INTERACTIONS[(a, j)] = r() # value: sequence stamp if r() < 0.32: # did it leave a score raw = (0.16 + 0.50 * SEGMENT[segment][domain] / 1.9 + 0.42 * QUALITY[j] + 0.26 * (r() - 0.5)) SCORES[(a, j)] = max(1, min(5, int(round(raw * 5)))) else: r(), r(), r() DOMAIN_OF = [ITEMS[j][1] for j in range(N_ITEMS)] COUNTS = {a: {} for a in SUBSCRIBERS} for (a, j) in INTERACTIONS: # subscriber group: dominant domain in the past COUNTS[a][DOMAIN_OF[j]] = COUNTS[a].get(DOMAIN_OF[j], 0) + 1 GROUP_OF = {a: (max(sorted(counts), key=lambda x: counts[x]) if counts else "none") for a, counts in COUNTS.items()} PERIODS, K, SHIFT, RISING = 48, 5, 12, 9 # in period 12 the response of item 9 changes def run_loop(wide): """The system is fed by its own log: it measures what it shows, it shows what it measures.""" shown, selected = {}, {} for (a, j) in IMPRESSIONS: shown[(GROUP_OF[a], j)] = shown.get((GROUP_OF[a], j), 0) + 1 for (a, j) in INTERACTIONS: selected[(GROUP_OF[a], j)] = selected.get((GROUP_OF[a], j), 0) + 1 total, rows = [0] * N_ITEMS, [] for d in range(1, PERIODS + 1): period_shown, period_inter = [0] * N_ITEMS, 0 for a in SUBSCRIBERS: g = GROUP_OF[a] order = sorted(range(N_ITEMS), key=lambda j: (-(selected.get((g, j), 0) + 0.5) / (shown.get((g, j), 0) + 5.0), j)) top = order[:K] if not wide else order[:K - 1] + [ # fifth slot to diversity min((j for j in range(N_ITEMS) if j not in order[:K - 1]), key=lambda j: (total[j], (j + a + d) % N_ITEMS))] r = rng(SEED + 660000 + a * 53 + d * 1301) for j in top: period_shown[j] += 1 total[j] += 1 shown[(g, j)] = shown.get((g, j), 0) + 1 e = PROPENSITY[(a, j)] if j == RISING and d >= SHIFT: e = min(0.95, e * 8.0) if r() < e: period_inter += 1 selected[(g, j)] = selected.get((g, j), 0) + 1 top5 = sorted(range(N_ITEMS), key=lambda j: -period_shown[j])[:5] rows.append((period_inter / len(SUBSCRIBERS), [j for j in range(N_ITEMS) if period_shown[j] == 0], sum(period_shown[j] for j in top5) / sum(period_shown), period_shown[RISING])) return rows NARROW = run_loop(False) print(f"narrow setup: top {K} every period, {PERIODS} periods, {len(SUBSCRIBERS)} subscribers") print(f"in period {SHIFT} the response rate of {ITEMS[RISING][0]} rises eightfold") print("period interactions/subscriber items never shown that period top-five impression share") for d in (1, 8, 24, 48): e, never, p, n = NARROW[d - 1] print(f" {d:>2} {e:.4f} {len(never):>2}/{N_ITEMS} {p:.4f}") LOCKED = set(range(N_ITEMS)) for x in NARROW[PERIODS // 2:]: LOCKED &= set(x[1]) print(f"items not shown in ALL of the last {PERIODS - PERIODS // 2} periods: {len(LOCKED)}/{N_ITEMS}") ``` ``` narrow setup: top 5 every period, 48 periods, 360 subscribers in period 12 the response rate of consumption_report_subscription rises eightfold period interactions/subscriber items never shown that period top-five impression share 1 1.2250 0/24 0.4711 8 1.3917 8/24 0.5461 24 1.4583 8/24 0.5461 48 1.3194 8/24 0.5461 items not shown in ALL of the last 24 periods: 6/24 ``` In the first period all 24 items are shown. In the eighth period eight items drop out; the number freezes at **8/24** there and does not move until the forty-eighth period. Six items are shown in **none** of the last twenty-four periods. Popularity bias is the other face of the same movement: the impression share of the top five items rises from 0.4711 to **0.5461** and stays there. The loop closes on itself. No new information is born about an item that is not shown, its estimate stays low, and the low estimate leaves it out again in the next period: the item stays out not because it is bad but because it **is not measured**. In the twelfth period the environment changes but the narrow setup cannot see it, because the only way to tell the system is for that item to be shown. ## Giving the Fifth Slot to Diversity ```python WIDE = run_loop(True) WIDE_LOCKED = set(range(N_ITEMS)) for x in WIDE[PERIODS // 2:]: WIDE_LOCKED &= set(x[1]) print(f"wide setup: top {K - 1} by model, fifth slot to the least shown item") print("period interactions/subscriber items never shown that period top-five impression share shifted item") for d in (1, 8, 24, 48): e, never, p, n = WIDE[d - 1] print(f" {d:>2} {e:.4f} {len(never):>2}/{N_ITEMS} " f"{p:.4f} {n:>3}") cost = sum(WIDE[d][0] - NARROW[d][0] for d in range(SHIFT - 1)) gain = sum(WIDE[d][0] - NARROW[d][0] for d in range(SHIFT - 1, PERIODS)) equal = [d + 1 for d in range(PERIODS) if sum(x[0] for x in WIDE[:d + 1]) > sum(x[0] for x in NARROW[:d + 1])] print(f"short-term cost (total of periods 1-{SHIFT - 1}): per subscriber {cost:+.4f}") print(f"long-term gain (total of periods {SHIFT}-{PERIODS}): per subscriber {gain:+.4f}") print(f"cumulative break-even period: {equal[0] if equal else 'none'}; at period {PERIODS} the " f"cumulative difference is {sum(x[0] for x in WIDE) - sum(x[0] for x in NARROW):+.4f}") print(f"the shifted item at period {PERIODS}: narrow setup {NARROW[-1][3]} impressions, " f"wide setup {WIDE[-1][3]}") print(f"data collected -- narrow: {len(LOCKED)} items never shown, top-five impression share " f"{NARROW[-1][2]:.4f}; wide: {len(WIDE_LOCKED)} items, {WIDE[-1][2]:.4f}") ``` ``` wide setup: top 4 by model, fifth slot to the least shown item period interactions/subscriber items never shown that period top-five impression share shifted item 1 1.1833 0/24 0.4611 37 8 1.2722 0/24 0.5011 81 24 1.4806 0/24 0.4989 97 48 1.3889 0/24 0.4989 97 short-term cost (total of periods 1-11): per subscriber -1.0000 long-term gain (total of periods 12-48): per subscriber +2.2250 cumulative break-even period: 27; at period 48 the cumulative difference is +1.2250 the shifted item at period 48: narrow setup 17 impressions, wide setup 97 data collected -- narrow: 6 items never shown, top-five impression share 0.5461; wide: 0 items, 0.4989 ``` The cost is paid in the first eleven periods: the wide setup finishes **1.0000** interactions per subscriber behind, because one of the five slots goes not to the best estimate but to the least known item. In return coverage never narrows and the share of the top five items stays at **0.4989**. In the twelfth period the account turns. Since the wide setup already shows the item whose response changed in its rotating slot, it notices the change in the **same period**. In the forty-eighth period that item receives **97** impressions in the wide setup and **17** in the narrow setup. The total gain between the twelfth and the forty-eighth is **+2.2250** per subscriber; together with the **−1.0000** cost of the first eleven periods, the cumulative difference reaches zero in the **twenty-seventh period** and becomes **+1.2250** in the forty-eighth. Diversity is not a matter of taste but **a measurement budget**: a system that never measures wins as long as the environment stays constant and loses every time it changes. "Echo chamber" is a **metaphor** that describes this table and has no number of its own; its counterpart is the number of items not shown and the impression share of the top five. The place where it misleads is this: in the setup no subscriber's propensity changes. What narrows is not the interest of the subscriber but **the domain the system measures**, and an item that is not measured being counted as bad is not a belief but a gap in the record. ## Summary - After the eighth period the narrow setup does not show 8 of 24 items in any period, 6 items appear in none of the last twenty-four periods, and the impression share of the top five rises from 0.4711 to 0.5461. - The short-term cost of diversity is −1.0000 per subscriber in the first eleven periods and its long-term gain is +2.2250 between the twelfth and forty-eighth; the cumulative difference breaks even in the twenty-seventh period. - The echo chamber is a metaphor; in this setup what narrows is not the interest of the subscriber but the domain the system measures. ## Course Wrap-Up The account of fourteen lessons is in a single table, and each number was read from its own lesson. The rows do not carry a common unit; the last column writes the metric and its direction. **The difference is given in a single sign convention: the method's number minus the baseline** and is in parentheses in the third column. The fourth column is the course's own number: the unseen state, the untried action, the item not shown. | Lesson | Baseline | Method's number | Data collected | Metric | |---|---|---|---|---| | Reinforcement 01 | random action 10.432 | hand-written rule 20.621 (+10.189) | 1,355 of 1,458 pairs unseen | return, higher is better | | Reinforcement 02 | random action 10.432 | rule with history field 23.148 (+12.716) | state space from 243 to 1,215; no count | return, higher is better | | Reinforcement 03 | agent without exploration −16.492 | exploration rate 0.20 gives 25.659 (+42.151) | 880 of 1,458 pairs untried | return, higher is better | | Reinforcement 04 | rule policy 20.621 | table with exploration 24.419 (+3.798) | 221 of 1,458 cells not updated | return, higher is better | | Reinforcement 05 | random action 10.432 | replay buffer 26.233 / 25.302 (+15.801) | 678 of 1,458 cells not updated | return, higher is better | | Reinforcement 06 | zero weights 10.177 | baseline subtraction 22.866 / 22.190 (+12.689) | agreement 0.290; no count | return, higher is better | | Reinforcement 07 | raw return 14.207 / 17.218 | advantage 23.401 / 23.755 (+9.194) | 0 of 5 subscribers left unvisited | return, higher is better | | Recommender 01 | random order 0.2036 | most chosen item 0.6527 (+0.4491) | 15 of 24 items never recommended | top-five hit rate, higher is better | | Recommender 02 | popularity 0.6527 | content-based 0.4431 (−0.2096) | 0 of 24 items not shown | top-five hit rate, higher is better | | Recommender 03 | popularity 0.6527 | user-based 0.6946 (+0.0419) | 8 of 24 items not shown | top-five hit rate, higher is better | | Recommender 04 | popularity 0.6527 | skipping setup 0.7006 (+0.0479) | 15 of 24 items not shown | top-five hit rate, higher is better | | Recommender 05 | popularity fallback 0.5752 | three-question intake 0.5487 (−0.0265) | 11 of 24 items not shown | cold-start top-five hit rate, higher is better | | Recommender 06 | online baseline 1.4581 | light setup 1.4788 (+0.0207) | 12 of 24 items not shown | interactions per subscriber, higher is better | | Recommender 07 | narrow setup, 48 periods | wide setup (+1.2250) | 8 of 24 items not shown | cumulative interactions, higher is better | **The rule of the course is this: the measure of a system that makes decisions cannot be read without accounting for the fact that those decisions change its own data; a system that does not write down the data it collected counts as unmeasured.** The fourth column reduces to a single number: the total of the unseen pairs, the cells not updated, the subscribers left unvisited and the items not shown in the twelve lessons that print a count is **3,203 units**. The units are not the same and the total is a count, not a mean: behind every performance number measured in this course stand more than three thousand units about which the method learned nothing. Reinforcement 02 and 06 print no count and do not enter the sum. The second claim of the course is read from the same table: **a number measured offline does not tell online behavior.** In four lessons the two numbers diverged. In Recommender 06 the setup that was first offline with 0.6886 was last with 1.4563 when it made its own impression decisions, and fell below the baseline. In Recommender 07 the narrow setup that looked good on the log finished forty-eight periods 1.2250 behind. In Recommender 05 the hit rate read from the log fell from 0.5752 to 0.5487 once questions were asked. In Reinforcement 03 the divergence appeared in the opposite direction: the **behavior** of the agent with exploration rate 1.00 was bad but **the data it collected** was good, and the frozen table gave 23.000. There is a single source: the log the number is read from is a log whose decisions were made by another setup. In fifty-two lessons the baseline was written beside every number, and in the last fourteen it was counted that the system changes its own data. The only thing never asked is **for whom** it is measured. Every number in the table is the mean over all subscribers: if the same number says two different things in two subgroups, which one is reported; what is the **basis** on which the system shows an item to one person and not to another, and can it be read; and with which number is the harm written that a system with a good measure does to a group that never enters the measure. The next course, Model Evaluation, Interpretability and Ethics, opens with these three questions.