Categorical Visualization

The previous lesson encoded the distribution of a continuous value with three displays. Some of the questions in the report are not continuous but categorical: the number of readings of five regions in four tariff tiers, the share of those numbers within the region, the order of a tier among the regions. Categorical data sits in a crosstab, and that table can be displayed in two different ways — the slices are either stacked on top of each other or placed side by side.

The difference between these two forms is not a matter of style but a difference of channel. In the stacked display only the bottom slice starts from the zero of the axis; the others float above the axis, and the reader has no starting point to measure them from. The numbers of the lesson are the counterpart of this difference in observations.

• CD14. The stacked display has a single baseline. The bottom slice starts from the zero of the axis and is read from the position channel; the lower edge of the upper slices is not on the axis but at the point where the previous slice ends.
• CD15. The floating slice is read from the length channel. The reader does not read two positions and subtract; the reader looks at the length of the slice. The resolutions of the first lesson continue: position 100, length 40 steps.
• CD16. The axis span comes from the display. The stacked axis has to carry the total, the grouped axis only the largest slice. The same number of steps means a step of a different size in the two displays.
• CD17. A slice smaller than half an axis step is read back as zero. A slice that sits in the source leaves the display entirely and reaches the reader as “none”.
• CD18. The comparison of a categorical question is between categories. The measure is how many of the region pairs that must be separated are separated, in the same tier.
• CD19. The stacked display gives the total with a single reading. The upper edge of the stack is measured from the zero of the axis; the number that edge carries is the total, not the parts.

The Crosstab and the Two Axes

# categorical.py — MODEL: the measurement network of K03-K05 is
# generated with the same seed. The crosstab of five regions and four tariff
# tiers is the input; this table will be encoded with two separate displays
# and read back.
import math

SEED, RAW, M32 = 20260218, 1400, 0xFFFFFFFF
PERIODS = ["2026-01", "2026-02", "2026-03"]
REGION = [("north", 0.28, 21), ("south", 0.22, 17), ("east", 0.18, 26),
          ("west", 0.14, 14), ("central", 0.18, 23)]


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


_kept, _dropped = [], []
for i in range(RAW):
    r = rng(SEED + i)
    b = REGION[pick(r(), [x[1] for x in REGION])]
    pick(r(), [0.06, 0.24, 0.30, 0.24, 0.11, 0.05])
    pick(r(), [0.08, 0.14, 0.27, 0.34, 0.17])
    (_dropped if r() < 0.046 else _kept).append(
        {"subscriber_id": 10001 + i, "region": b[0], "base": b[2]})
VALID = {a["subscriber_id"] for a in _kept}

READINGS = []
for a in _kept + _dropped:
    r = rng(SEED + 7000 + a["subscriber_id"])
    for d in range(pick(r(), [0.05, 0.12, 0.21, 0.62])):
        t = 0.0 if r() < 0.038 else math.floor(
            a["base"] * math.exp((r() + r() + r() - 1.5) * 0.62)
            * (1 - d * 0.05) * 100 + 0.5) / 100
        if a["subscriber_id"] not in VALID:
            r()
            continue
        READINGS.append({"subscriber_id": a["subscriber_id"],
                         "region": a["region"], "period": PERIODS[d],
                         "consumption_m3": t})
        r()

TARIFF = [10, 25, 40]
NAME = [b[0] for b in REGION]


def tier(x):
    return (1 if x < TARIFF[0] else 2 if x < TARIFF[1]
            else 3 if x < TARIFF[2] else 4)


K = len(TARIFF) + 1
CROSS = {a: [0] * K for a in NAME}
for o in READINGS:
    CROSS[o["region"]][tier(o["consumption_m3"]) - 1] += 1
TOTAL = {a: sum(CROSS[a]) for a in NAME}

# The axis span changes with the display: the stacked axis has to carry the
# total, the grouped axis only the largest slice. The upper limit is rounded up
# to the next hundred, because the axis label sits on round numbers.
RES_POSITION = 100


def round_hundred(x):
    return (int(x) // 100 + 1) * 100


AXIS_STACKED = round_hundred(max(TOTAL.values()))
AXIS_GROUPED = round_hundred(max(max(CROSS[a]) for a in NAME))


def grid(v, axis, n):
    return axis * round(n * v / axis) / n


print(f"fictional crosstab: {len(READINGS)} readings, tariff tiers "
      f"{TARIFF[0]}/{TARIFF[1]}/{TARIFF[2]} m3")
print(f"{'region':8s}" + "".join(f"{'tier ' + str(i + 1):>12s}" for i in range(K))
      + f"{'total':>9s}")
for a in NAME:
    print(f"{a:8s}" + "".join(f"{c:12d}" for c in CROSS[a]) + f"{TOTAL[a]:9d}")
print(f"stacked axis 0-{AXIS_STACKED}, one step {AXIS_STACKED / RES_POSITION:.1f} observations; "
      f"grouped axis 0-{AXIS_GROUPED}, one step {AXIS_GROUPED / RES_POSITION:.1f} observations")

fictional crosstab: 3199 readings, tariff tiers 10/25/40 m3
region        tier 1      tier 2      tier 3      tier 4    total
north             42         661         210           8      921
south             59         538          48           0      645
east              21         281         225          30      557
west              88         377          11           0      476
central           30         352         198          20      600
stacked axis 0-1000, one step 10.0 observations; grouped axis 0-700, one step 7.0 observations

The crosstab is the source itself and carries twenty numbers. The magnitudes of the numbers are very different: the second tier varies between 281 and 661, the fourth tier between 0 and 30. The difficulty of categorical display starts here, because a single axis has to carry both magnitudes at the same time.

The axis spans of the two displays are different, and the difference is not a preference but a necessity. In the stacked display the bar of a region extends as far as the total of all slices, so the axis has to carry the largest total, 921, and ends at the next round number, 1000. In the grouped display the bars stand side by side and the axis carries only the largest slice, 661; the upper limit is 700. The same position grid of a hundred steps means a step of 10 observations in one and a step of 7 observations in the other.

Stacked: The Slice Without a Baseline

# stacked.py — the slices are placed on top of each other. The bottom slice
# starts from the zero of the axis and is read from the position channel; the
# upper slices float above the axis, and instead of subtracting two positions
# the reader looks at the length of the slice and reads it with the 40 steps
# of the length channel (first lesson).
STEPS, RES_LENGTH = 40, 40


def read_stacked(c):
    return [grid(c[0], AXIS_STACKED, RES_POSITION)] + [
        grid(x, AXIS_STACKED, RES_LENGTH) for x in c[1:]]


def read_grouped(c):
    return [grid(x, AXIS_GROUPED, RES_POSITION) for x in c]


def bar(c):
    s, t = "", 0.0
    for i, x in enumerate(c):
        t += x
        s += str(i + 1) * (round(STEPS * t / AXIS_STACKED) - len(s))
    return s


print(f"stacked display, {STEPS}-step axis (what the reader sees):")
for a in NAME:
    print(f"{a:8s}|{bar(CROSS[a]):<40s}| total {TOTAL[a]}")

print(f"\n{'region':8s}" + "".join(f"{'slice ' + str(i + 1):>14s}" for i in range(K)))
abs_err = [[] for _ in range(K)]
rel_err = [[] for _ in range(K)]
lost = 0
for a in NAME:
    g = read_stacked(CROSS[a])
    for i in range(K):
        abs_err[i].append(abs(g[i] - CROSS[a][i]))
        if CROSS[a][i] > 0:
            rel_err[i].append(100 * abs(g[i] - CROSS[a][i]) / CROSS[a][i])
            lost += g[i] == 0
    print(f"{a:8s}" + "".join(f"{c:>7d}/{x:<6.0f}" for c, x in zip(CROSS[a], g)))
print(f"\n{'slice':6s}{'channel':>10s}{'|error|':>9s}{'relative':>10s}")
for i in range(K):
    print(f"{i + 1:<6d}{'position' if i == 0 else 'length':>10s}"
          f"{sum(abs_err[i]) / len(NAME):9.1f}{sum(rel_err[i]) / len(rel_err[i]):9.1f}%")
print(f"slices read back as zero though above zero in the source: {lost}")
_t = grid(max(TOTAL.values()), AXIS_STACKED, RES_POSITION)
print(f"the total comes with a single reading in the same display: north {TOTAL['north']} "
      f"-> {_t:.0f}")

stacked display, 40-step axis (what the reader sees):
north   |1122222222222222222222222222333333333   | total 921
south   |11222222222222222222222233              | total 645
east    |1222222222223333333334                  | total 557
west    |1111222222222222222                     | total 476
central |122222222222222333333334                | total 600

region         slice 1       slice 2       slice 3       slice 4
north        42/40        661/650       210/200         8/0     
south        59/60        538/550        48/50          0/0     
east         21/20        281/275       225/225        30/25    
west         88/90        377/375        11/0           0/0     
central      30/30        352/350       198/200        20/25    

slice    channel  |error|  relative
1       position      1.2      2.7%
2         length      6.6      1.4%
3         length      5.0     22.0%
4         length      3.6     47.2%
slices read back as zero though above zero in the source: 2
the total comes with a single reading in the same display: north 921 -> 920

The bars themselves are the clearest evidence of the lesson. North’s fourth slice, that is, the eight subscribers with readings above 40 m³, does not show up at all in the bar of forty steps; west’s third slice does not either. These are not numbers the display erases but numbers the display cannot carry: on an axis of 1000 units, in a bar divided into 40 steps, one step is worth 25 observations, and 11 observations are below half a step.

The error table is read by slice order. The bottom slice passes through the position channel and its relative error stays at 2.7%. The upper slices pass through the length channel, and the relative error rises to 22.0% in the third slice and 47.2% in the fourth. The 1.4% error of the second slice seems to break this pattern but does not: the second slice is the largest slice of the set and has the highest absolute error, 6.6 observations — it is because it is large that its relative error stays small. Two things determine the error: whether the slice has a baseline, and the size of the slice relative to the axis.

The one thing the same display does not lose is the total. The upper edge of the stack is measured from the zero of the axis, comes with a single position reading, and north’s 921 observations are read back as 920. This is why the stacked display exists: it gives the total and the size order of the totals at a single glance. It manages to give the parts only for the bottom slice.

An objection stands here: a careful reader can read both edges of a slice on the axis and subtract, and get a better number. True, but that reader is no longer looking at the chart; the reader is doing arithmetic on top of the chart. If a number has to be obtained by subtraction, a table takes the place of the display.

Grouped: A Baseline for Every Slice

# grouped.py — the same numbers are placed side by side: every slice starts
# from its own baseline, all are read from the position channel, and the axis
# carries only the largest slice. The question is between regions: which
# region is higher in the same tier.
from itertools import combinations

print(f"{'tier':8s}{'source':>26s}{'stacked':>26s}{'grouped':>26s}")
SERIES = []
for i in range(K):
    s = ([CROSS[a][i] for a in NAME], [read_stacked(CROSS[a])[i] for a in NAME],
         [read_grouped(CROSS[a])[i] for a in NAME])
    SERIES.append(s)
    print(f"{i + 1:<8d}" + "".join(f"{x:>26s}" for x in
          ["/".join(f"{v:.0f}" for v in t) for t in s]))

print(f"\n{'tier':8s}{'pairs to separate':>24s}{'stacked':>10s}"
      f"{'grouped':>8s}{'reversed':>13s}")
tally = [0, 0, 0, 0]
for i, (k, y, g) in enumerate(SERIES):
    c = [(p, q) for p, q in combinations(range(len(NAME)), 2) if k[p] != k[q]]
    sep_y = sum(1 for p, q in c if y[p] != y[q])
    sep_g = sum(1 for p, q in c if g[p] != g[q])
    rev = sum(1 for p, q in c if (k[p] - k[q]) * (y[p] - y[q]) < 0)
    for j, v in enumerate((len(c), sep_y, sep_g, rev)):
        tally[j] += v
    print(f"{i + 1:<8d}{len(c):>24d}{sep_y:>10d}{sep_g:>8d}{rev:>13d}")
print(f"{'total':8s}{tally[0]:>24d}{tally[1]:>10d}{tally[2]:>8d}{tally[3]:>13d}")

tier                        source                   stacked                   grouped
1                   42/59/21/88/30            40/60/20/90/30            42/56/21/91/28
2              661/538/281/377/352       650/550/275/375/350       658/539/280/378/350
3                210/48/225/11/198          200/50/225/0/200         210/49/224/14/196
4                      8/0/30/0/20               0/0/25/0/25               7/0/28/0/21

tier           pairs to separate   stacked grouped     reversed
1                             10        10      10            0
2                             10        10      10            0
3                             10         9      10            0
4                              9         6       9            0
total                         39        35      39            0

The comparison question is between categories, and the measure is how many of the region pairs that differ in the source are still read as different after the display. In the fourth tier nine of the ten pairs are different in the source — south and west really are the same, zero. The stacked display separates six of these nine, the grouped display all nine.

The three pairs the stacked display loses are clear. East’s 30 observations and central’s 20 observations are both read as 25, and north’s 8 observations drop to zero and become the same as the two regions that are really zero. In the third tier west’s 11 observations also drop to zero but collide with no one; the only loss there is that north’s 210 and central’s 198 are both read as 200, and nine of the ten pairs are separated instead of nine. Out of the 39 pairs that must be separated, the stacked display separates 35 and the grouped display 39.

The last column is the subtle finding of this lesson: no pair is read reversed. The stacked display never shows a small region as big; the only thing it does is erase differences. This comes from the error having no direction — rounding to the grid brings two regions closer to each other, but does not swap them. The sentence to write into the report comes from here: the ranking read from the stacked display is not wrong, it is incomplete. How many steps of the ranking can be read is a separate number, and that number is 35 of 39.

The price of the grouped display is not in the read-back but in the space: four slices in five regions mean twenty bars, and when one more slice is added it becomes twenty-five. The stacked display fits the same numbers into five bars. The choice between the two displays depends on whether the question asked is about the total or about the parts, and that question is the question of the report, not of the data.

Summary

• In the stacked display only the bottom slice has a baseline; that slice is read from the position channel, the floating slices from the length channel, and since the axis carries the total, one step rises to 25 observations.
• The relative read-back error by slice order is 2.7% · 1.4% · 22.0% · 47.2%; two things determine the error: whether the slice has a baseline, and the size of the slice relative to the axis.
• North’s fourth slice of 8 readings and west’s third slice of 11 readings are read back as zero, that is, two slices that sit in the source leave the display entirely.
• Of the 39 region pairs that must be separated, the stacked display separates 35 and the grouped display 39; no pair is read reversed, what is lost is not the order but the distinction.
• The stacked display gives the total with a single reading — north’s 921 observations are read back as 920 — and what it loses is only the parts.

Next Step

In three lessons the categories were counted and the distributions were divided into intervals, and each time there was a label on one side of the axis and a number on the other. Some of the questions in the report ask for two numbers at once: whether the tier also rises as consumption rises in a region, whether the readings of two periods move together. The first lesson had shown that this question can be answered only with the position channel, because the second independent copy of the channel exists only there. The next lesson uses those two copies and counts what happens as the number of points grows: how much overplotting there is, where the unreadable region starts, and how many separate levels are read back when the same relationship is put on the color steps of a heat map.