--- title: 'Native Extensions' source: 'https://academia.sh/en/courses/python-concurrency/native-extensions' course: 'Concurrency and Performance' language: en updated: '2026-08-17T18:10:24+00:00' license: 'CC BY-SA 4.0' --- # Native Extensions Moving the hot path to a lower level is measured in the model with three effects; under the CPU-bound load, twelve groups of four drop 80 steps to 44 and 73 ticks to 25, boundary cost pulls the gain from 48 down to 36, and the same mechanism earns nothing under the I/O-bound load. The previous two lessons framed the hot path with two separate numbers: under the CPU-bound load, `is_cpu_step` is called 322 times, and depending on the shape holding the 80-step structure, it occupies between 1 and 172 objects. Both point to the same place — the work's weight is in the CPU steps. At this point there is a commonly used route: move the hot path out of the language, to a lower level. Its gain is usually told in terms of time, and that telling is not accepted in this course. This lesson asks the same question in a single form: **when the hot path's step count drops, what happens to total ticks?** ## What Is a Native Extension A **native extension** is a component that holds code already translated into machine instructions rather than having the interpreter execute it step by step, and that appears to the language like a module. The rest of the program uses it with an ordinary function call; the call crosses a **boundary**, arguments are converted to the form the lower level expects, the work is done there, and the result is converted back. This lesson does not measure how to write an extension, it measures **what it earns**. No tool, compiler, or package is named; what is measured is the approach itself, not a particular mechanism that realizes it. Two properties defining the approach are enough: **the work is done outside the language's execution steps**, and **every entry and exit crosses a boundary.** Whatever tool provides these two properties, the measurement stays the same; the next table is the direct counterpart of these two properties. A real extension is not compiled either, and the reason is the same as the course's own measure. A compiled extension's gain depends on the environment, the compilation options, and the kind of work; it gives two different numbers on two machines. The step measured in the model, though, comes out the same on every run. **A number that cannot be reproduced cannot carry a gain claim.** ## Three Effects in the Model The native extension enters the model with three separate effects, and measuring the three separately is this lesson's real work. **The first is step count.** Several consecutive CPU steps on the hot path are handed off to the extension with a single call; the step count the interpreter has to see drops. Four steps collapsing into one call means one step in place of those four. **The second is boundary cost.** The call does not cross for free: converting arguments, transferring data to the other side of the boundary, and building the return value all do work. In the model this is a fixed number of extra steps per call. **The third is the lock.** Code running at the lower level can release the global interpreter lock while it does its work. Its counterpart in the scheduler is direct: the step that releases the lock does not want a CPU slot, so it can **overlap** with other tasks' steps. This is not new behavior; the shared definition's scheduler already treats every step that does not want a slot this way. The three effects can be turned on and off independently, and the table does exactly that: in some rows only the step count drops, in some boundary cost is added, in some the lock is released. What the model leaves out also has to be said, because a model cannot be read without saying what it does not cover. Boundary cost here is fixed per call; in reality it can depend on the size of the data transferred, and a fixed number misleads on a call that copies large data. Which side of the boundary owns the memory, how an error at the lower level is carried back into the language, and the extension's portability are not in the model either. These are not part of the gain question, they are part of the **cost question**; the table only measures gain and keeps this limit explicit. The measurement's assumptions: - **PE13** — In the model, the native extension is represented only by these three effects: step merging, boundary cost, releasing the lock. No other effect is modeled. - **PE14** — Only **consecutive and complete** groups merge. Steps that cannot fill a group keep being interpreted; an extension only takes over where it can take over the whole hot path. - **PE15** — Boundary cost is a fixed, whole number of steps per call. Argument conversion, data transfer, and building the return value are all rolled into this one number. - **PE16** — A step releasing the lock is not a CPU step for the scheduler: it does not want a slot and can overlap. The behavior of `tasks` and `run` is **not changed**; only a new step kind is added. - **PE17** — A real extension is not compiled, time is not measured. Gain is written only as **steps and ticks.** - **PE18** — The comparison baseline is the same task set's uncondensed run. The two loads are measured separately and nothing changes but the load; both runs are in the eight-worker, single-slot regime. ## Measurement ```python """Native extension model: when the hot path shortens, what happens to total tick.""" SEED = 20260817 CPU, IO, NATIVE = "cpu", "io", "native" def make_rng(seed): state = seed % 2147483646 + 1 def draw(n): nonlocal state state = (state * 48271) % 2147483647 return state % n return draw def tasks(count=8, steps=10, io_share=7, seed=SEED): draw, result = make_rng(seed), [] for i in range(count): result.append([IO if draw(10) < io_share else CPU for _ in range(steps)]) return result def run(jobs, workers, cpu_slots): remaining = [list(j) for j in jobs] tick = overlap = cpu_steps = io_steps = 0 while any(remaining): active = [i for i, j in enumerate(remaining) if j][:workers] if not active: break slots_left, advanced = cpu_slots, 0 for i in active: step = remaining[i][0] if step == CPU: if slots_left <= 0: continue slots_left -= 1 cpu_steps += 1 else: io_steps += 1 remaining[i].pop(0) advanced += 1 tick += 1 overlap += max(0, advanced - 1) return tick, overlap, cpu_steps, io_steps def condense(job, group_size, boundary=0, kind=CPU): """Reduces group_size consecutive CPU steps to one step; adds boundary steps.""" result, accumulated = [], 0 for step in job: if step == CPU: accumulated += 1 if accumulated == group_size: result.append(kind) result.extend([CPU] * boundary) accumulated = 0 else: result.extend([CPU] * accumulated) accumulated = 0 result.append(step) result.extend([CPU] * accumulated) return result REGIMES = ( ("baseline: every step interpreted", 1, 0, CPU), ("four steps in one call", 4, 0, CPU), ("four steps, boundary 1 step", 4, 1, CPU), ("four steps, lock released", 4, 0, NATIVE), ("four steps, lock + boundary 1", 4, 1, NATIVE), ("two steps, lock + boundary 1", 2, 1, NATIVE), ("ten steps, lock released", 10, 0, NATIVE), ) for load, share in (("CPU-bound", 1), ("I/O-bound", 7)): G = tasks(io_share=share) baseline = run(G, 8, 1)[0] print(f"{load:<30s} {'steps':>5s} {'tick':>4s} {'overlap':>8s} {'gain':>7s}") for name, group_size, boundary, kind in REGIMES: Y = [condense(g, group_size, boundary, kind) for g in G] tick, overlap, _, _ = run(Y, 8, 1) print(f" {name:<28s} {sum(len(g) for g in Y):5d} {tick:4d} {overlap:8d}" f" {baseline - tick:7d}") print() G = tasks(io_share=1) D = [condense(g, 4, 0, NATIVE) for g in G] print(f"under the CPU-bound load, steps before condensing {sum(len(g) for g in G)}, " f"after {sum(len(g) for g in D)}") print(f"groups of four reduced to one call {sum(g.count(NATIVE) for g in D)}, " f"cpu steps still interpreted {sum(g.count(CPU) for g in D)}") H = tasks(io_share=7) E = [condense(g, 4, 0, NATIVE) for g in H] print(f"under the I/O-bound load, groups of four {sum(g.count(NATIVE) for g in E)}, " f"steps {sum(len(g) for g in H)} -> {sum(len(g) for g in E)}") ``` ``` CPU-bound steps tick overlap gain baseline: every step interpreted 80 73 7 0 four steps in one call 44 37 7 36 four steps, boundary 1 step 56 49 7 24 four steps, lock released 44 25 19 48 four steps, lock + boundary 1 56 37 19 36 two steps, lock + boundary 1 80 44 36 29 ten steps, lock released 53 44 9 29 I/O-bound steps tick overlap gain baseline: every step interpreted 80 31 49 0 four steps in one call 80 31 49 0 four steps, boundary 1 step 80 31 49 0 four steps, lock released 80 31 49 0 four steps, lock + boundary 1 80 31 49 0 two steps, lock + boundary 1 80 24 56 7 ten steps, lock released 80 31 49 0 under the CPU-bound load, steps before condensing 80, after 44 groups of four reduced to one call 12, cpu steps still interpreted 25 under the I/O-bound load, groups of four 0, steps 80 -> 80 ``` ## The Drop in Steps Under the CPU-bound load, the baseline is **80 steps, 73 ticks**. Reducing groups of four to one call pulls steps to **44**, ticks to **37**; gain **36 ticks**. The block below shows where this comes from: **12** groups of four merged, 48 steps reduced to 12 steps, leaving **25** CPU steps still to be interpreted. What draws attention is that the overlap column stays at **7**. Step count dropped, ticks dropped, but overlap never rose. The reason is the single slot: a merged step is still a CPU step, and it still wants a slot. **Less work was done, but the work was still done in sequence.** When the lock is released, the table changes: the same 44 steps finish in **25 ticks**, and overlap rises from **7** to **19**. Gain from **36** to **48**. The 12 extra ticks come entirely from those 12 merged steps ceasing to ask for a slot. **The same step reduction gives two-thirds of the gain without releasing the lock, and the full gain with it.** ## What the Boundary Costs Boundary cost sits in the table as two rows and is easy to read. Without the lock, a one-step-per-call boundary pushes steps from **44** to **56** — 12 calls, 12 extra steps. Ticks from **37** to **49**, gain drops from **36** to **24**. The boundary took back **a third** of the gain. With the lock, the same boundary pulls gain from **48** down to **36**. This number is interesting: **36** is exactly equal to the gain of the row where the lock is never released but the boundary is never paid either. So boundary cost eats exactly the gain that releasing the lock brought. Two different mechanisms, the same total. If a measurement showed only the gain number for these two rows, no difference between them would show at all. This is the most fragile point of an extension decision: **gain grows with group size, boundary cost grows with call count.** Small groups mean many calls, and many calls mean many boundaries. ## Gain Without a Step Drop The sixth row shows this at the extreme. Reducing two steps to one call and paying a one-step boundary per call keeps total steps at **80** — two steps left, one call and one boundary step arrived; the arithmetic came back to where it started. Yet ticks drop from **73** to **44** and overlap rises from **7** to **36**. The work done never shrank at all; the gain is **29 ticks**. All of it comes from releasing the lock. This row is this lesson's share of the course's measurement axis: **an optimization's gain does not have to come from reducing the work done; it can also come from letting that work overlap.** ## The Limit of Growing the Group The last row is where intuition runs backward. When the group is made ten instead of four — handing more steps off to a single call, expecting a bigger gain — steps come out to **53** instead of **44**, ticks to **44** instead of **25**. The reason is in the shared definition's setup. In ten-step tasks, exactly ten consecutive CPU steps arriving in a row is rare; I/O steps cut in and break up the group. Steps that cannot fill the group do not merge, they are interpreted one by one. **When group size exceeds the hot path's real length, gain does not grow, it shrinks.** Placing this row next to the one above it teaches a second thing. The group-of-ten gives **44** ticks at **53** steps; the group-of-four, together with boundary cost, gives **37** ticks at **56** steps. So the regime with **fewer** steps spends **more** ticks. Total step count alone does not decide tick count; what decides is **which kind** those steps are. In the group-of-ten, only a few steps can release the lock, so overlap stays at **9**; in the group-of-four, it rises to **19**. **Counting steps is not enough; you have to count which steps can overlap too.** ## If the Hot Path Is Not There The second table is this lesson's harshest result. Under the I/O-bound load, five of the seven rows are **exactly identical** to the baseline: 80 steps, 31 ticks, 49 overlapping steps, gain **0**. The block below gives the reason in a single number: under this load, the count of groups of four consecutive CPU steps is **0**. There is nothing to merge. The extension was written, the boundary was built, the lock was released — and the table never moved. The one row with a gain is the pair-group: **31** to **24** ticks, gain **7**. Under this load, a run of two consecutive CPU steps can be found, and again the gain comes not from steps but from overlap; steps stay at **80**. The conclusion is this: **moving the hot path to a lower level only earns something if the hot path is actually there.** Both tables measure the same mechanism, the same group size, and the same boundary cost; one earns 48 ticks, the other none. The difference does not come from the code, it comes from **the composition of the load** — and the way to know that composition is to measure, as the previous two lessons did. The first lesson's call breakdown already showed this distinction. `is_cpu_step` was called **136** times under the I/O-bound load, **322** under the CPU-bound load; in the first, CPU steps did not pile up at the slot, in the second they did. The information needed for the extension decision was sitting there already. Moving to an extension without profiling first is the same thing as writing the second table's five rows: work done, measured gain zero. ## Summary - A native extension is represented in the model with three effects: the hot path's step count dropping, a fixed boundary cost per call, and a step ceasing to want a slot because the lock is released. - Under the CPU-bound load, **12** groups of four drop 80 steps to **44** and 73 ticks to **37**; if the lock is also released, ticks drop to **25** and overlap rises from **7** to **19**. - A one-step-per-call boundary cost pulls gain from **48** down to **36** — eating the entire gain that releasing the lock brought. Gain grows with group size, cost grows with call count. - Gain can happen without total steps ever dropping: at pair groups, steps stay at **80** while ticks drop from **73** to **44**, and all the gain comes from overlap. - Gain shrinks when group size exceeds the hot path's real length: the group-of-ten gives **44** ticks while the group-of-four gives **25**. - The same mechanism earns nothing under the I/O-bound load, because the count of consecutive groups of four is **0** under that load. ## Next Step This lesson did the same thing across a table: on every row, what was counted was written into the row's name — how many steps merged, whether a boundary was paid, whether the lock was released. If the names were stripped away, only numbers would remain, and **37** would sit next to **37** and claim to be measuring the same thing; yet one was the locked regime paying a boundary, the other the unlocked regime not paying one. The course's last lesson measures this danger on itself: same work, same regime, the only difference is what is included in the measurement — and how much difference that makes between two numbers.