Vectorized Operations

The previous lesson showed that one of the three measures came out contrary to expectation: the line count fell eleven to one and the allocated bytes fell six to one, but the pass count did not fall at all. This lesson takes up that number, and it starts with worse news — in a vectorized chain the pass count does not fall, it rises.

The job itself comes from the derived variable step of the previous course: the share of each cell that remains above the second tariff tier is to be computed. 25 m³ is subtracted from the reading, negative results are clipped at zero, and the remainder is converted to liters. Three operations, one result. It is written in two ways: fused in a single loop, and as a chain of three vectorized calls.

• NC5. A vectorized call saves you from writing the loop, not from the pass. Every link in the chain scans the data once and holds its result in a new array; the gain is in the line, the cost is in the pass and in the allocated bytes.
• NC6. A vectorized operation chooses the dtype itself. This choice moves the number in three ways: by promoting the type it enlarges the bytes, by overflowing in a narrow type it shrinks the result, and by turning a sentinel value into an ordinary number it makes the missing cell indistinguishable.

One Job, Two Paths

# vectorized.py — MODEL: a vectorized operation is written as a call that does not walk the
# elements one by one. Each call scans the data once (one pass) and holds its result in a new
# array (one intermediate array). The dataset is the fictional one prepared by the Data Collection and Preparation course.
import inspect
import sys
from array import array

def rng(seed):
    s = seed & 0xFFFFFFFF
    def next_value():
        nonlocal s
        s = (1664525 * s + 1013904223) & 0xFFFFFFFF
        return s / 4294967296
    return next_value

SUBSCRIBERS, PERIODS = 1260, 3
r = rng(20260218)
RAW = [round(4 + r() * 56, 2) for _ in range(SUBSCRIBERS * PERIODS)]
r2 = rng(9110723)
p = [r2() for _ in range(SUBSCRIBERS * PERIODS)]
EMPTY = set(sorted(range(len(p)), key=lambda j: p[j])[:581])
READINGS = [0.0 if j in EMPTY else RAW[j] for j in range(SUBSCRIBERS * PERIODS)]

PASSES = 0   # one full scan over the data from start to finish
ALLOC = 0    # bytes allocated by the result and the intermediate arrays

class Array:
    def __init__(self, data, dtype="d"):
        self.d, self.dtype = array(dtype, data), dtype

def apply_op(a, f, dtype=None):
    global PASSES, ALLOC
    PASSES += 1
    c = Array([f(x) for x in a.d], dtype or a.dtype)
    ALLOC += sys.getsizeof(c.d)
    return c

UPPER = 25.0      # upper bound of the second tariff tier, m3
LITER = 1000      # m3 -> liter

def by_hand(data):
    global PASSES, ALLOC
    output = []
    for x in data:
        f = x - UPPER
        output.append((f if f > 0.0 else 0.0) * LITER)
    PASSES += 1
    c = Array(output)
    ALLOC += sys.getsizeof(c.d)
    return c

subtract = lambda a, s: apply_op(a, lambda x: x - s)
clip = lambda a, lower: apply_op(a, lambda x: x if x > lower else lower)
multiply = lambda a, s: apply_op(a, lambda x: x * s)

def chain(a):
    return multiply(clip(subtract(a, UPPER), 0.0), LITER)

A = Array(READINGS)
PASSES, ALLOC = 0, 0
E = by_hand(A.d); pass_hand, alloc_hand = PASSES, ALLOC
PASSES, ALLOC = 0, 0
V = chain(A); pass_chain, alloc_chain = PASSES, ALLOC
lines = lambda f: len(inspect.getsource(f).strip().splitlines())
row_fmt = lambda w: lambda h: "".join(str(v).ljust(w[0]) if j == 0 else str(v).rjust(w[j])
                                      for j, v in enumerate(h))
y = row_fmt([22, 18, 20, 10])
print(f"job: (reading - {UPPER} m3) is clipped at zero and converted to liters; {len(READINGS)} elements\n")
print(y(["measure", "single hand loop", "vectorized", "ratio"]))
print(y(["line count", lines(by_hand), lines(chain), f"{lines(by_hand)/lines(chain):.1f}x"]))
print(y(["pass count", pass_hand, pass_chain, f"{pass_chain/pass_hand:.1f}x"]))
print(y(["bytes allocated", alloc_hand, alloc_chain, f"{alloc_chain/alloc_hand:.1f}x"]))
same = sum(1 for x, z in zip(E.d, V.d) if abs(x - z) < 1e-9)
print(f"\nsame result: {same}/{len(READINGS)} elements are the same below the 1e-9 liter threshold "
      f"(floating-point comparison is not done with ==; the threshold is below the liter resolution)")
print(f"excess total {sum(V.d)/LITER:.2f} m3")

job: (reading - 25.0 m3) is clipped at zero and converted to liters; 3780 elements

measure                 single hand loop          vectorized     ratio
line count                            10                   2      5.0x
pass count                             1                   3      3.0x
bytes allocated                    30320               90960      3.0x

same result: 3780/3780 elements are the same below the 1e-9 liter threshold (floating-point comparison is not done with ==; the threshold is below the liter resolution)
excess total 34707.55 m3

Both paths produce the same 3,780 numbers. The comparison was made with a threshold far below the liter resolution, 1e-9 liters; two floating-point values are not compared with ==, because two calculations that reach the same mathematical result in a different order can part ways in the last digit. The threshold here does not hide a meaningful difference: 1e-9 liters is a billionth of the smallest value a meter can read.

The gain is in the line, and it is real. The hand-written version takes 10 lines, the chain 2. The difference is not just length: in the hand-written version the loop bound, the accumulation list and the clipping condition are nested in the same place, and changing one of the three operations requires touching the body of the loop. In the chain the three links are independent of each other, and subtract, clip and multiply are written once and used everywhere.

The cost is in the next two rows, and it is mostly left out of the account. Because the chain consists of three links, the data is scanned three times; the hand-written loop scans once. Because each link holds its result in a new array, three arrays of 3,780 elements are allocated: 90,960 bytes. Of this, 60,640 bytes are intermediate arrays, that is, two temporary results thrown away after use. The hand-written version allocates only the output array: 30,320 bytes. On small datasets this difference does not show; when the scale grows, the length of the chain is written directly onto the memory peak, because each of the intermediate arrays takes as much space as the input.

From here comes a reading that ties into the standing rule of the course: the number of links in the chain is a decision made by whoever writes the call, and usually without noticing. A chain of three links means three passes, a chain of five links means five passes, and the line of code stays the same in both. The length of the chain is visible in the code; its cost is not.

Fusing the Chain

# fused.py — MODEL: the third path. The three links of the chain are fused into a single
# vectorized call; the loop is still not written but the data is scanned once.
def fused(a):
    return apply_op(a, lambda x: (x - UPPER if x > UPPER else 0.0) * LITER)

PASSES, ALLOC = 0, 0
K = fused(A); pass_fused, alloc_fused = PASSES, ALLOC
w = row_fmt([20, 10, 12, 12, 18])
print(w(["measure", "by hand", "chain", "fused", "lowest"]))
print(w(["line count", lines(by_hand), lines(chain), lines(fused), "chain / fused"]))
print(w(["pass count", pass_hand, pass_chain, pass_fused, "by hand / fused"]))
print(w(["bytes allocated", alloc_hand, alloc_chain, alloc_fused, "by hand / fused"]))
print(w(["reusable parts", 0, 3, 0, "chain"]))
deviating = sum(1 for x, z in zip(V.d, K.d) if abs(x - z) > 1e-9)
print(f"\nall three paths give the same result: {deviating} elements deviate above the 1e-9 liter threshold")
print(f"the fused path is at the same line count as the chain and at the same pass count and bytes as the by-hand version; its cost is "
      f"that three decisions go into a single expression and the number of reusable parts drops from 3 to 0")

measure                by hand       chain       fused            lowest
line count                  10           2           2     chain / fused
pass count                   1           3           1   by hand / fused
bytes allocated          30320       90960       30320   by hand / fused
reusable parts               0           3           0             chain

all three paths give the same result: 0 elements deviate above the 1e-9 liter threshold
the fused path is at the same line count as the chain and at the same pass count and bytes as the by-hand version; its cost is that three decisions go into a single expression and the number of reusable parts drops from 3 to 0

The third path shows that the cost of the chain is not inevitable. When the work of the three links is fused into a single vectorized call, the line count stays the same as in the chain, and the pass count drops to 1 as in the hand-written loop and the allocated bytes to 30,320. This is the lowest of all three measures.

Where the cost lies stands in the last row. In the chain, subtract, clip and multiply are three independent parts, and they are reused in another calculation in another order. In the fused expression the same three decisions enter a single body and cannot be taken out from the outside; the number of reusable parts falls from 3 to 0. If the tariff threshold moves by one step, in the chain only a parameter of a call changes; in the fused expression the body is rewritten.

This is the form the measure takes in the course: the gain of an abstraction does not show in all three numbers at once, and “writing vectorized” is not a single thing. How many links are written is a decision, and that decision determines the pass count and the memory peak together.

Three Decisions the Call Makes

# hidden.py — MODEL: three decisions a vectorized call makes in your place.
LITER_VALUES = [int(round(x * LITER)) for x in READINGS]

# 1. Type promotion. The first link of the chain is an integer operation, the second multiplies by a float constant.
INTS = Array(LITER_VALUES, "i")
PASSES, ALLOC = 0, 0
remaining = apply_op(INTS, lambda x: x - 25000, "i")
int_bytes = ALLOC + sys.getsizeof(INTS.d)
PASSES, ALLOC = 0, 0
charge = apply_op(apply_op(INTS, lambda x: x - 25000, "i"), lambda x: x * 0.0042, "d")
float_bytes = ALLOC + sys.getsizeof(INTS.d)

# 2. Overflow. If the dtype is chosen as a 16-bit integer to save memory, the liter values
# do not fit the limit. The wrapping is written out explicitly; it is a model of the dtype's behavior.
LIMIT = 2 ** 15
wrap = lambda v: ((v + LIMIT) % (2 * LIMIT)) - LIMIT
overflowing = [v for v in LITER_VALUES if v > LIMIT - 1]
wrapped = [wrap(v) for v in LITER_VALUES]

# 3. Loss of the sentinel value. The 581 empty cells had been marked with -1.
FLAGGED = Array([-1.0 if j in EMPTY else RAW[j] for j in range(SUBSCRIBERS * PERIODS)])
final = multiply(clip(subtract(FLAGGED, UPPER), 0.0), LITER)
distinct = sum(1 for j in EMPTY if abs(final.d[j]) > 1e-9)
zeros = sum(1 for x in final.d if abs(x) < 1e-9)

z = row_fmt([26, 22, 30])
print(z(["hidden default", "written assumption", "measured result"]))
print(z(["type promotion", "stays integer", f"{int_bytes} -> {float_bytes} bytes"]))
print(z(["16-bit overflow", "no overflow", f"{len(overflowing)}/{len(LITER_VALUES)} elements shifted"]))
print(z(["sentinel cell", "581 cells marked", f"{distinct}/581 distinguishable"]))
print(f"\ntype promotion: a single float constant carries the rest of the chain to 64 bits, "
      f"allocated bytes grow by {float_bytes - int_bytes}; charge total {sum(charge.d):.2f}")
print(f"overflow: {len(overflowing)} elements exceed the 32,767-liter limit; the total "
      f"comes out {sum(wrapped)/LITER:.2f} m3 instead of {sum(LITER_VALUES)/LITER:.2f} m3")
print(f"sentinel: after the chain, the cells whose excess looks zero number {zeros}; of these 581 are "
      f"not real zeros but marked empty cells")

hidden default                written assumption               measured result
type promotion                     stays integer          30400 -> 60720 bytes
16-bit overflow                      no overflow    1538/3780 elements shifted
sentinel cell                   581 cells marked         0/581 distinguishable

type promotion: a single float constant carries the rest of the chain to 64 bits, allocated bytes grow by 30320; charge total 31643.72
overflow: 1538 elements exceed the 32,767-liter limit; the total comes out 1239.85 m3 instead of 102034.22 m3
sentinel: after the chain, the cells whose excess looks zero number 1783; of these 581 are not real zeros but marked empty cells

Type promotion is the quietest item of the chain. The array was built as a 32-bit integer and the first link stayed integer; when the second link multiplies by a fractional unit price, the result cannot be represented in an integer and the whole array is carried to the 64-bit float. The allocated bytes rise from 30,400 to 60,720. In the written code there is no sign at all of the growth; what decides the growth is that the constant 0.0042 is fractional. In long chains this means that every link after the first fractional link takes twice the space.

Overflow goes the opposite way. Narrowing the dtype saves memory: a 16-bit integer holds the 3,780 elements in 7,560 bytes. But 1,538 of the readings in liters exceed the 32,767 limit, and these values shift to the negative side. The total of the dataset comes out 1,239.85 m³ instead of 102,034.22 m³ — a loss of 98.8%. The result arrives not with an error message but as a small number that looks reasonable; a check that only asks “is the total positive” lets it through. The wrapping behavior was written out explicitly here, because what a dtype does when its limit is exceeded is the type’s own decision, and the model makes it visible.

The sentinel value is the third decision, and it continues from where the previous lesson left off. Marking empty cells with -1 is the known way of holding a missing value in an array, but the marker lives only until the first operation: subtracting 25 from -1 gives -26, the clip pulls it to zero, and the multiplication leaves zero. At the end of the chain none of the 581 cells can be told apart. Of the 1,783 cells whose excess looks zero, 581 are in fact cells that were never read, and no trace is left to show the difference between the two. The clip operation here reduces two different meanings to a single value: “stayed below the threshold” and “was not measured”.

What the three items have in common is this: none of them produces an error. Type promotion gives the correct result and only enlarges the memory, overflow gives a number that looks reasonable, and the clipping of the sentinel gives a valid excess value. These are exactly the decisions an abstraction hides — the decisions made in cases where the call runs and produces a result. For this reason the rule of the course looks not at whether a call gives the correct result but at whether its default was written down.

Summary

• The vectorized chain lowers the line count from 10 to 2, raises the pass count from 1 to 3 and the allocated bytes from 30,320 to 90,960; 60,640 of these bytes are intermediate arrays.
• The number of links in the chain is written one to one onto the pass count; the line of code stays constant, and the cost is not visible in the code.
• A fractional constant carries the rest of the chain to 64 bits and raises the allocated bytes from 30,400 to 60,720.
• A 16-bit dtype shifts 1,538 of the 3,780 elements and shows the total of 102,034.22 m³ as 1,239.85 m³; the result is not an error but a number that looks reasonable.
• Of the 581 empty cells marked with a sentinel value, 0 stay distinguishable after the first operation; the count of cells whose excess looks zero rises to 1,783, and “below the threshold” and “was not measured” fall to a single value.
• When three links are fused into a single call, the pass count falls to 1, the bytes to 30,320 and the line count does not change; the cost is that the number of reusable parts falls from 3 to 0.

Next Step

In this lesson all the operations were done on a single shape: between an array of 3,780 elements and one number. In real calculations two arrays meet, and their shapes are most often not the same — a base per period, a household count per subscriber, a unit price per tariff tier. The next lesson measures how different shapes are counted as compatible and why two incompatible shapes, instead of giving an error, produce an unexpected array.