Building Narrative with Data

The previous three lessons fixed the number: the layout was split to support the decision, the definition of the metric was made ungameable, and the data cut-off and the refresh lag were written next to the number. What the reader holds is still a number. For a number to turn into a decision it has to sit in a chain, and if the links of the chain are missing, the reader completes them on their own.

The measure of this lesson is the gap that a skipped step leaves: the number of inferences the reader is forced to make when a link is removed. A second number is written next to it: how many of the defaults of these inferences are wrong with respect to the dataset, and how many of those wrong defaults strengthen the recommendation.

• RP19. The dataset is inherited from the Data Collection and Preparation course and is fictional: a reading table of 3,199 rows, 1,329 subscribers, three periods, five regions. The seed is written into the code.
• RP20. The finding comes from the Statistical Analysis and Exploratory Data Analysis course: the east mean is 24.674 m³ (557 readings), the north 20.210 m³ (921 readings), the gap 4.463 m³. The test computation is not repeated here; the numbers are read from there.
• RP21. Both numbers of the margin are read from the Statistical Analysis and Exploratory Data Analysis course: when the region labels were reshuffled 2,000 times, the observed gap was not exceeded in any run; the spread of the four defensible decisions on mass, sample, zero readings and the number of periods is 0.206 m³.
• RP22. The narrative chain is fictional: five steps, sixteen premises, fifteen of them needed to defend the recommendation.
• RP23. When a step is removed, the reader is assumed to complete the premises of that step on their own. The default of each premise and the direction of the default are written in the lesson.
• RP24. The recommendation is also fictional: a review of the tariff tier boundary in the east region.

The Chain

# narrative.py — MODEL: the fictional dataset from K03 is generated with the same seed; on top
# of it a five-step narrative chain that runs from finding to recommendation is built.
import math

SEED, RAW, M32 = 20260218, 1400, 0xFFFFFFFF
REGION = [("north", 0.28, 21), ("south", 0.22, 17), ("east", 0.18, 26),
          ("west", 0.14, 14), ("central", 0.18, 23)]


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


SUBSCRIBERS, DISCARDED = [], []
for i in range(RAW):
    r = rng(SEED + i)
    b = REGION[pick(r(), [x[1] for x in REGION])]
    pick(r(), [0.06, 0.24, 0.30, 0.24, 0.11, 0.05])
    pick(r(), [0.08, 0.14, 0.27, 0.34, 0.17])
    (DISCARDED if r() < 0.046 else SUBSCRIBERS).append(
        {"subscriber_id": 10001 + i, "region": b[0], "base": b[2]})
VALID = {k["subscriber_id"] for k in SUBSCRIBERS}
READINGS = {}
for k in SUBSCRIBERS + DISCARDED:
    r = rng(SEED + 7000 + k["subscriber_id"])
    for d in range(pick(r(), [0.05, 0.12, 0.21, 0.62])):
        t = 0.0 if r() < 0.038 else math.floor(
            k["base"] * math.exp((r() + r() + r() - 1.5) * 0.62) * (1 - d * 0.05) * 100
            + 0.5) / 100
        if k["subscriber_id"] in VALID:
            READINGS.setdefault(k["subscriber_id"], {})[d + 1] = t
        r()

MEANS = {}
for b, _, _ in REGION:
    v = [t for k in SUBSCRIBERS if k["region"] == b
         for t in READINGS.get(k["subscriber_id"], {}).values()]
    MEANS[b] = (sum(v) / len(v), len(v))
GAP = MEANS["east"][0] - MEANS["north"][0]
WIDEST = max(MEANS.values())[0] - min(MEANS.values())[0]
ORDERED = sorted(MEANS, key=lambda b: -MEANS[b][0])

# Narrative chain: each step carries a set of premises. Premise fields: code, step,
# content, the default the reader falls to if the step is missing, whether the default is wrong
# for the dataset, whether it is needed to defend the recommendation, whether the default
# strengthens the recommendation (+1/-1).
PREMISES = [
    ("O1", "finding", f"east mean {MEANS['east'][0]:.3f} m3",
     "number unknown", False, True, 0),
    ("O2", "finding", f"north mean {MEANS['north'][0]:.3f} m3",
     "number unknown", False, True, 0),
    ("O3", "finding", f"numbers rest on {MEANS['east'][1]} and {MEANS['north'][1]} readings",
     "observation count is large", True, True, 1),
    ("O4", "finding", "the number in the finding was written with its margin",
     "number is exact", True, True, 1),
    ("O5", "comparison", f"gap {GAP:.3f} m3", "gap unknown", False, True, 0),
    ("O6", "comparison", f"gap is {GAP / MEANS['north'][0] * 100:.1f} percent of north",
     "gap is small", True, True, -1),
    ("O7", "comparison",
     f"widest gap of the five regions {WIDEST:.3f} m3 ({ORDERED[0]}-{ORDERED[-1]})",
     "gap is the only important gap", True, True, 1),
    ("O8", "candidate cause", "candidate: east base consumption is higher",
     "no cause exists", False, True, 0),
    ("O9", "candidate cause", "rival candidate: reading coverage gap",
     "a single cause exists", True, True, 1),
    ("O10", "candidate cause", "rival candidate: tariff tier distribution",
     "a single cause exists", True, False, 1),
    ("O11", "uncertainty", "K05: not exceeded in 2,000 reshuffles", "margin not measured", False, True, 0),
    ("O12", "uncertainty", "K05: spread of four decisions 0.206 m3", "margin is zero", True, True, 1),
    ("O13", "uncertainty", "what the margin does not cover was named",
     "margin covers everything", True, True, 1),
    ("O14", "recommendation", "recommended action was written", "action is the reader's choice", True, True, -1),
    ("O15", "recommendation", "reversibility of the action was written",
     "action is reversible", True, True, 1),
    ("O16", "recommendation", "which metric the action will be measured by", "no measurement will be done", True, True, 1),
]
STEPS = ["finding", "comparison", "candidate cause", "uncertainty", "recommendation"]
print(f"fictional dataset (seed {SEED}): {len(SUBSCRIBERS)} subscribers, "
      f"{sum(len(v) for v in READINGS.values())} readings")
print(f"finding: east {MEANS['east'][0]:.3f} m3 ({MEANS['east'][1]} readings), "
      f"north {MEANS['north'][0]:.3f} m3 ({MEANS['north'][1]} readings), gap {GAP:.3f} m3")
print(f"chain: {len(STEPS)} steps, {len(PREMISES)} premises, "
      f"{sum(1 for o in PREMISES if o[5])} of them needed to defend the recommendation")
for a in STEPS:
    print(f"  {a:<15} " + ", ".join(o[0] for o in PREMISES if o[1] == a))

fictional dataset (seed 20260218): 1329 subscribers, 3199 readings
finding: east 24.674 m3 (557 readings), north 20.210 m3 (921 readings), gap 4.463 m3
chain: 5 steps, 16 premises, 15 of them needed to defend the recommendation
  finding         O1, O2, O3, O4
  comparison      O5, O6, O7
  candidate cause O8, O9, O10
  uncertainty     O11, O12, O13
  recommendation  O14, O15, O16

Here the narrative is not a style but a set of premises. If the recommendation is to be defensible, the premises standing behind it must have been given to the reader; every premise that is not given is produced by the reader. What the model does is count this production.

The name of the third step is also deliberately not “cause” but candidate cause. The report writes that the base consumption of the east region is higher as a candidate, puts the two rival candidates beside it, and does not decide among them; there is no mechanism in the dataset that would make this decision. Writing the candidates is not an evasion; it is the only way to keep the reader from falling onto a single candidate.

The Skipped Step

# One step is removed from the chain and the premises the reader is forced to complete on
# their own are counted as needed. Wrong default: a completion that is not true for the dataset.
def inference_gap(removed=None, dropped=frozenset()):
    missing = [o for o in PREMISES if o[5] and (o[1] == removed or o[0] in dropped)]
    wrong = [o for o in missing if o[4]]
    return missing, wrong, sum(1 for o in wrong if o[6] > 0)


print(f"{'removed step':<22}{'reader inference':>20}{'wrong default':>19}"
      f"{'strengthening':>21}")
for a in [None] + STEPS:
    e, y, g = inference_gap(a)
    print(f"{a or 'none (full chain)':<22}{len(e):>20}{len(y):>19}{g:>21}")
print(f"\nall wrong defaults in the whole chain and their directions")
for o in PREMISES:
    if o[4]:
        print(f"  {o[0]:<4} {o[1]:<16} the reader falls to this default: {o[3]:<30}"
              f"{'strengthens the recommendation' if o[6] > 0 else 'weakens the recommendation'}")
every_wrong = [o for o in PREMISES if o[4]]
print(f"of {len(every_wrong)} wrong defaults in total, {sum(1 for o in every_wrong if o[6] > 0)} "
      f"strengthen the recommendation, {sum(1 for o in every_wrong if o[6] < 0)} weaken it")

removed step              reader inference      wrong default        strengthening
none (full chain)                        0                  0                    0
finding                                  4                  2                    2
comparison                               3                  2                    1
candidate cause                          2                  1                    1
uncertainty                              3                  2                    2
recommendation                           3                  3                    2

all wrong defaults in the whole chain and their directions
  O3   finding          the reader falls to this default: observation count is large    strengthens the recommendation
  O4   finding          the reader falls to this default: number is exact               strengthens the recommendation
  O6   comparison       the reader falls to this default: gap is small                  weakens the recommendation
  O7   comparison       the reader falls to this default: gap is the only important gap strengthens the recommendation
  O9   candidate cause  the reader falls to this default: a single cause exists         strengthens the recommendation
  O10  candidate cause  the reader falls to this default: a single cause exists         strengthens the recommendation
  O12  uncertainty      the reader falls to this default: margin is zero                strengthens the recommendation
  O13  uncertainty      the reader falls to this default: margin covers everything      strengthens the recommendation
  O14  recommendation   the reader falls to this default: action is the reader's choice weakens the recommendation
  O15  recommendation   the reader falls to this default: action is reversible          strengthens the recommendation
  O16  recommendation   the reader falls to this default: no measurement will be done   strengthens the recommendation
of 11 wrong defaults in total, 9 strengthen the recommendation, 2 weaken it

The first row of the table confirms the definition of the chain: in the full chain no inference is left to the reader, because all fifteen of the fifteen premises needed to defend the recommendation have been written. This row is not a measure of success but the measuring instrument’s own zero point; the remaining five rows are read against it.

None of the five steps is free. When the finding is removed, four inferences are left to the reader; when the uncertainty or the recommendation is removed, three each; when the comparison is removed, three; when the candidate cause is removed, two. The numbers are close to each other and, read alone, do not say which step is more expensive to skip.

The second and the third columns make the distinction. Of the eleven wrong defaults, nine strengthen the recommendation and only two weaken it. The gaps are not filled at random: when the reader cannot see a margin next to a number, the reader counts the margin as zero; when the reader cannot see a rival candidate cause, the reader assumes there is a single cause; when the number of observations is not written, the reader accepts that the number comes from a large dataset. All three make the recommendation look sturdier than it is.

The result is a methodological rule. Skipping a narrative step makes the recommendation easier to defend, and this ease is independent of the author’s intent. The two weakening defaults do not break this rule; on the contrary, they explain it: when the reader does not see the size of the gap, the reader counts it as small, and when the recommended action is not written, the reader chooses the action. These two are not skipped, because they do not serve the author; the steps that are skipped are the steps that serve once they are skipped.

The Place of the Margin

At this point the place of the margin of variation of the Statistical Analysis and Exploratory Data Analysis course in the chain can be settled. Three placements are tried.

# The place of the margin in the narrative is tried with three placements. The margin was
# measured in K05 and is not recomputed here; what is asked is at which step of the chain it stands.
PLACEMENT = [("margin not written at all", {"O4", "O11", "O12", "O13"}),
             ("margin only at the end, in a limitations paragraph", {"O4"}),
             ("margin next to the finding and in the uncertainty step", set())]
print(f"{'placement of the margin':<58}{'inference':>9}{'wrong':>8}{'strengthening':>15}")
for name, dropped in PLACEMENT:
    e, y, g = inference_gap(dropped=dropped)
    print(f"{name:<58}{len(e):>9}{len(y):>8}{g:>15}")

print("\n--- chain of the third placement ---------------------------------")
for a in STEPS:
    print(f"  [{a}]")
    for o in PREMISES:
        if o[1] == a:
            print(f"     {o[0]:<4} {o[2]}{'' if o[5] else '   (not needed)'}")

placement of the margin                                   inference   wrong  strengthening
margin not written at all                                         4       3              3
margin only at the end, in a limitations paragraph                1       1              1
margin next to the finding and in the uncertainty step            0       0              0

--- chain of the third placement ---------------------------------
  [finding]
     O1   east mean 24.674 m3
     O2   north mean 20.210 m3
     O3   numbers rest on 557 and 921 readings
     O4   the number in the finding was written with its margin
  [comparison]
     O5   gap 4.463 m3
     O6   gap is 22.1 percent of north
     O7   widest gap of the five regions 10.896 m3 (east-west)
  [candidate cause]
     O8   candidate: east base consumption is higher
     O9   rival candidate: reading coverage gap
     O10  rival candidate: tariff tier distribution   (not needed)
  [uncertainty]
     O11  K05: not exceeded in 2,000 reshuffles
     O12  K05: spread of four decisions 0.206 m3
     O13  what the margin does not cover was named
  [recommendation]
     O14  recommended action was written
     O15  reversibility of the action was written
     O16  which metric the action will be measured by

The second placement is the common one, and it leaves one inference open. The margin is written, but it is written at the end: while reading the finding the reader saw no interval next to the number and had already formed the “the number is exact” default. The limitations paragraph that comes at the end does not take this default back; it only clears the author’s record. The margin has not one place but two places, and the first is next to the number in the finding.

The second is the uncertainty step, and there the margin says two different things. The gap that was not exceeded in the 2,000 reshuffles of the Statistical Analysis and Exploratory Data Analysis course closes a single question: would this gap come out if the region labels were distributed at random. The 0.206 m³ spread of the four defensible decisions is an entirely different uncertainty, and the reshuffling margin does not measure it at all. This is why a third premise stands in the chain: what the margin does not cover is named. When it is not named, the default the reader falls to is “the margin covers everything”, and this default too is among those that strengthen the recommendation.

The last three premises of the chain tie the recommendation to an action: the action is written, whether it is reversible is written, the metric by which it will be measured is written. The last one ties the previous two lessons to this point. If the metric is not written, the recommendation is never tested; if the metric is written, its definition has to be tight, otherwise the number that evaluates the recommendation can be moved without moving what it measures.

Summary

• The narrative is not a style but a five-step chain of premises: finding, comparison, candidate cause, uncertainty, recommendation. The chain carries sixteen premises and fifteen of them are needed to defend the recommendation.
• When a step is removed, the number of inferences left to the reader is between 2 and 4; this number alone does not say which step is more expensive.
• Of the chain’s eleven wrong defaults, nine strengthen the recommendation and two weaken it; skipping a step makes the recommendation easier to defend, and this ease is independent of the author’s intent.
• The margin of variation has not one place but two: next to the number in the finding and in the uncertainty step. A separate limitations paragraph at the end leaves 1 inference open, and that inference is the “the number is exact” default.
• The uncertainty step also names what the margin does not cover: the 2,000-reshuffle margin of the Statistical Analysis and Exploratory Data Analysis course closes only the question of random distribution of the labels, and does not measure the 0.206 m³ spread of the four decisions.
• The third step writes not the cause but the candidate causes; the last three premises of the chain tie the recommendation to an action, to the reversibility of the action and to the metric that will test it.

Next Step

The report now stands on a chain: the numbers are in a layout that can be read back, the definitions of the metrics are tight, the cut-off of the source is written, the links of the narrative are complete. All of this concerns producing a single report once. Whether the same report can be produced again three months later, by another person, so that it gives the same numbers, was never asked. The next lesson counts this: how many items have to be fixed to produce the same report a second time, how much the number moves when one is missing, and how many items the report adds on top of the items counted by the Data Collection and Preparation course and the Data Analysis with Python course.