Native Extensions

The previous two lessons framed the hot path with two separate numbers: under the CPU-bound load, is_cpu_step is called 322 times, and depending on the shape holding the 80-step structure, it occupies between 1 and 172 objects. Both point to the same place — the work’s weight is in the CPU steps.

At this point there is a commonly used route: move the hot path out of the language, to a lower level. Its gain is usually told in terms of time, and that telling is not accepted in this course. This lesson asks the same question in a single form: when the hot path’s step count drops, what happens to total ticks?

What Is a Native Extension

A native extension is a component that holds code already translated into machine instructions rather than having the interpreter execute it step by step, and that appears to the language like a module. The rest of the program uses it with an ordinary function call; the call crosses a boundary, arguments are converted to the form the lower level expects, the work is done there, and the result is converted back.

This lesson does not measure how to write an extension, it measures what it earns. No tool, compiler, or package is named; what is measured is the approach itself, not a particular mechanism that realizes it. Two properties defining the approach are enough: the work is done outside the language’s execution steps, and every entry and exit crosses a boundary. Whatever tool provides these two properties, the measurement stays the same; the next table is the direct counterpart of these two properties.

A real extension is not compiled either, and the reason is the same as the course’s own measure. A compiled extension’s gain depends on the environment, the compilation options, and the kind of work; it gives two different numbers on two machines. The step measured in the model, though, comes out the same on every run. A number that cannot be reproduced cannot carry a gain claim.

Three Effects in the Model

The native extension enters the model with three separate effects, and measuring the three separately is this lesson’s real work.

The first is step count. Several consecutive CPU steps on the hot path are handed off to the extension with a single call; the step count the interpreter has to see drops. Four steps collapsing into one call means one step in place of those four.

The second is boundary cost. The call does not cross for free: converting arguments, transferring data to the other side of the boundary, and building the return value all do work. In the model this is a fixed number of extra steps per call.

The third is the lock. Code running at the lower level can release the global interpreter lock while it does its work. Its counterpart in the scheduler is direct: the step that releases the lock does not want a CPU slot, so it can overlap with other tasks’ steps. This is not new behavior; the shared definition’s scheduler already treats every step that does not want a slot this way.

The three effects can be turned on and off independently, and the table does exactly that: in some rows only the step count drops, in some boundary cost is added, in some the lock is released.

What the model leaves out also has to be said, because a model cannot be read without saying what it does not cover. Boundary cost here is fixed per call; in reality it can depend on the size of the data transferred, and a fixed number misleads on a call that copies large data. Which side of the boundary owns the memory, how an error at the lower level is carried back into the language, and the extension’s portability are not in the model either. These are not part of the gain question, they are part of the cost question; the table only measures gain and keeps this limit explicit.

The measurement’s assumptions:

• PE13 — In the model, the native extension is represented only by these three effects: step merging, boundary cost, releasing the lock. No other effect is modeled.
• PE14 — Only consecutive and complete groups merge. Steps that cannot fill a group keep being interpreted; an extension only takes over where it can take over the whole hot path.
• PE15 — Boundary cost is a fixed, whole number of steps per call. Argument conversion, data transfer, and building the return value are all rolled into this one number.
• PE16 — A step releasing the lock is not a CPU step for the scheduler: it does not want a slot and can overlap. The behavior of tasks and run is not changed; only a new step kind is added.
• PE17 — A real extension is not compiled, time is not measured. Gain is written only as steps and ticks.
• PE18 — The comparison baseline is the same task set’s uncondensed run. The two loads are measured separately and nothing changes but the load; both runs are in the eight-worker, single-slot regime.

Measurement

"""Native extension model: when the hot path shortens, what happens to total tick."""

SEED = 20260817
CPU, IO, NATIVE = "cpu", "io", "native"


def make_rng(seed):
    state = seed % 2147483646 + 1

    def draw(n):
        nonlocal state
        state = (state * 48271) % 2147483647
        return state % n
    return draw


def tasks(count=8, steps=10, io_share=7, seed=SEED):
    draw, result = make_rng(seed), []
    for i in range(count):
        result.append([IO if draw(10) < io_share else CPU for _ in range(steps)])
    return result


def run(jobs, workers, cpu_slots):
    remaining = [list(j) for j in jobs]
    tick = overlap = cpu_steps = io_steps = 0
    while any(remaining):
        active = [i for i, j in enumerate(remaining) if j][:workers]
        if not active:
            break
        slots_left, advanced = cpu_slots, 0
        for i in active:
            step = remaining[i][0]
            if step == CPU:
                if slots_left <= 0:
                    continue
                slots_left -= 1
                cpu_steps += 1
            else:
                io_steps += 1
            remaining[i].pop(0)
            advanced += 1
        tick += 1
        overlap += max(0, advanced - 1)
    return tick, overlap, cpu_steps, io_steps


def condense(job, group_size, boundary=0, kind=CPU):
    """Reduces group_size consecutive CPU steps to one step; adds boundary steps."""
    result, accumulated = [], 0
    for step in job:
        if step == CPU:
            accumulated += 1
            if accumulated == group_size:
                result.append(kind)
                result.extend([CPU] * boundary)
                accumulated = 0
        else:
            result.extend([CPU] * accumulated)
            accumulated = 0
            result.append(step)
    result.extend([CPU] * accumulated)
    return result


REGIMES = (
    ("baseline: every step interpreted", 1, 0, CPU),
    ("four steps in one call", 4, 0, CPU),
    ("four steps, boundary 1 step", 4, 1, CPU),
    ("four steps, lock released", 4, 0, NATIVE),
    ("four steps, lock + boundary 1", 4, 1, NATIVE),
    ("two steps, lock + boundary 1", 2, 1, NATIVE),
    ("ten steps, lock released", 10, 0, NATIVE),
)

for load, share in (("CPU-bound", 1), ("I/O-bound", 7)):
    G = tasks(io_share=share)
    baseline = run(G, 8, 1)[0]
    print(f"{load:<30s} {'steps':>5s} {'tick':>4s} {'overlap':>8s} {'gain':>7s}")
    for name, group_size, boundary, kind in REGIMES:
        Y = [condense(g, group_size, boundary, kind) for g in G]
        tick, overlap, _, _ = run(Y, 8, 1)
        print(f"  {name:<28s} {sum(len(g) for g in Y):5d} {tick:4d} {overlap:8d}"
              f" {baseline - tick:7d}")
    print()

G = tasks(io_share=1)
D = [condense(g, 4, 0, NATIVE) for g in G]
print(f"under the CPU-bound load, steps before condensing {sum(len(g) for g in G)}, "
      f"after {sum(len(g) for g in D)}")
print(f"groups of four reduced to one call {sum(g.count(NATIVE) for g in D)}, "
      f"cpu steps still interpreted {sum(g.count(CPU) for g in D)}")
H = tasks(io_share=7)
E = [condense(g, 4, 0, NATIVE) for g in H]
print(f"under the I/O-bound load, groups of four {sum(g.count(NATIVE) for g in E)}, "
      f"steps {sum(len(g) for g in H)} -> {sum(len(g) for g in E)}")

CPU-bound                      steps tick  overlap    gain
  baseline: every step interpreted    80   73        7       0
  four steps in one call          44   37        7      36
  four steps, boundary 1 step     56   49        7      24
  four steps, lock released       44   25       19      48
  four steps, lock + boundary 1    56   37       19      36
  two steps, lock + boundary 1    80   44       36      29
  ten steps, lock released        53   44        9      29

I/O-bound                      steps tick  overlap    gain
  baseline: every step interpreted    80   31       49       0
  four steps in one call          80   31       49       0
  four steps, boundary 1 step     80   31       49       0
  four steps, lock released       80   31       49       0
  four steps, lock + boundary 1    80   31       49       0
  two steps, lock + boundary 1    80   24       56       7
  ten steps, lock released        80   31       49       0

under the CPU-bound load, steps before condensing 80, after 44
groups of four reduced to one call 12, cpu steps still interpreted 25
under the I/O-bound load, groups of four 0, steps 80 -> 80

The Drop in Steps

Under the CPU-bound load, the baseline is 80 steps, 73 ticks. Reducing groups of four to one call pulls steps to 44, ticks to 37; gain 36 ticks. The block below shows where this comes from: 12 groups of four merged, 48 steps reduced to 12 steps, leaving 25 CPU steps still to be interpreted.

What draws attention is that the overlap column stays at 7. Step count dropped, ticks dropped, but overlap never rose. The reason is the single slot: a merged step is still a CPU step, and it still wants a slot. Less work was done, but the work was still done in sequence.

When the lock is released, the table changes: the same 44 steps finish in 25 ticks, and overlap rises from 7 to 19. Gain from 36 to 48. The 12 extra ticks come entirely from those 12 merged steps ceasing to ask for a slot. The same step reduction gives two-thirds of the gain without releasing the lock, and the full gain with it.

What the Boundary Costs

Boundary cost sits in the table as two rows and is easy to read.

Without the lock, a one-step-per-call boundary pushes steps from 44 to 56 — 12 calls, 12 extra steps. Ticks from 37 to 49, gain drops from 36 to 24. The boundary took back a third of the gain.

With the lock, the same boundary pulls gain from 48 down to 36. This number is interesting: 36 is exactly equal to the gain of the row where the lock is never released but the boundary is never paid either. So boundary cost eats exactly the gain that releasing the lock brought. Two different mechanisms, the same total. If a measurement showed only the gain number for these two rows, no difference between them would show at all.

This is the most fragile point of an extension decision: gain grows with group size, boundary cost grows with call count. Small groups mean many calls, and many calls mean many boundaries.

Gain Without a Step Drop

The sixth row shows this at the extreme. Reducing two steps to one call and paying a one-step boundary per call keeps total steps at 80 — two steps left, one call and one boundary step arrived; the arithmetic came back to where it started. Yet ticks drop from 73 to 44 and overlap rises from 7 to 36.

The work done never shrank at all; the gain is 29 ticks. All of it comes from releasing the lock. This row is this lesson’s share of the course’s measurement axis: an optimization’s gain does not have to come from reducing the work done; it can also come from letting that work overlap.

The Limit of Growing the Group

The last row is where intuition runs backward. When the group is made ten instead of four — handing more steps off to a single call, expecting a bigger gain — steps come out to 53 instead of 44, ticks to 44 instead of 25.

The reason is in the shared definition’s setup. In ten-step tasks, exactly ten consecutive CPU steps arriving in a row is rare; I/O steps cut in and break up the group. Steps that cannot fill the group do not merge, they are interpreted one by one. When group size exceeds the hot path’s real length, gain does not grow, it shrinks.

Placing this row next to the one above it teaches a second thing. The group-of-ten gives 44 ticks at 53 steps; the group-of-four, together with boundary cost, gives 37 ticks at 56 steps. So the regime with fewer steps spends more ticks. Total step count alone does not decide tick count; what decides is which kind those steps are. In the group-of-ten, only a few steps can release the lock, so overlap stays at 9; in the group-of-four, it rises to 19. Counting steps is not enough; you have to count which steps can overlap too.

If the Hot Path Is Not There

The second table is this lesson’s harshest result. Under the I/O-bound load, five of the seven rows are exactly identical to the baseline: 80 steps, 31 ticks, 49 overlapping steps, gain 0.

The block below gives the reason in a single number: under this load, the count of groups of four consecutive CPU steps is 0. There is nothing to merge. The extension was written, the boundary was built, the lock was released — and the table never moved.

The one row with a gain is the pair-group: 31 to 24 ticks, gain 7. Under this load, a run of two consecutive CPU steps can be found, and again the gain comes not from steps but from overlap; steps stay at 80.

The conclusion is this: moving the hot path to a lower level only earns something if the hot path is actually there. Both tables measure the same mechanism, the same group size, and the same boundary cost; one earns 48 ticks, the other none. The difference does not come from the code, it comes from the composition of the load — and the way to know that composition is to measure, as the previous two lessons did.

The first lesson’s call breakdown already showed this distinction. is_cpu_step was called 136 times under the I/O-bound load, 322 under the CPU-bound load; in the first, CPU steps did not pile up at the slot, in the second they did. The information needed for the extension decision was sitting there already. Moving to an extension without profiling first is the same thing as writing the second table’s five rows: work done, measured gain zero.

Summary

• A native extension is represented in the model with three effects: the hot path’s step count dropping, a fixed boundary cost per call, and a step ceasing to want a slot because the lock is released.
• Under the CPU-bound load, 12 groups of four drop 80 steps to 44 and 73 ticks to 37; if the lock is also released, ticks drop to 25 and overlap rises from 7 to 19.
• A one-step-per-call boundary cost pulls gain from 48 down to 36 — eating the entire gain that releasing the lock brought. Gain grows with group size, cost grows with call count.
• Gain can happen without total steps ever dropping: at pair groups, steps stay at 80 while ticks drop from 73 to 44, and all the gain comes from overlap.
• Gain shrinks when group size exceeds the hot path’s real length: the group-of-ten gives 44 ticks while the group-of-four gives 25.
• The same mechanism earns nothing under the I/O-bound load, because the count of consecutive groups of four is 0 under that load.

Next Step

This lesson did the same thing across a table: on every row, what was counted was written into the row’s name — how many steps merged, whether a boundary was paid, whether the lock was released. If the names were stripped away, only numbers would remain, and 37 would sit next to 37 and claim to be measuring the same thing; yet one was the locked regime paying a boundary, the other the unlocked regime not paying one. The course’s last lesson measures this danger on itself: same work, same regime, the only difference is what is included in the measurement — and how much difference that makes between two numbers.