--- title: Threads source: 'https://academia.sh/en/courses/python-concurrency/threads' course: 'Concurrency and Performance' language: en updated: '2026-08-17T18:10:24+00:00' license: 'CC BY-SA 4.0' --- # Threads Eight workers and a single slot finish eighty steps in 31 ticks on the I/O-bound load (gain 49, ratio 0.6125), and only 73 ticks on the CPU-bound load (gain 7); same mechanism, a sevenfold difference, and the gain is bounded in every load by the number of steps that can overlap. The previous lesson looked inside the tick and never wrote the total tick count. In the single-slot regime's first twelve ticks, the CPU column was always **1**, while the I/O column climbed as high as four. From that came a lower bound: 26 CPU steps have to spread across at least 26 separate ticks. This lesson asks for the number itself. Queueing eight tasks at once — opening a **thread** — finishes eighty steps in how many ticks? The answer is not one number. The same mechanism gives two entirely different numbers depending on load composition, and what this lesson measures is the size of that difference. ## A Thread Adds a Worker, Not a Slot A thread is a separate execution flow within the same process. Process, thread, scheduling, and context switching were built as mechanisms in the Operating System Concepts course and are not repeated here. This lesson's question is not mechanism, it is a **number**: which setting of the model does opening a thread change? The answer is written in the previous lesson's two settings. Eight threads means eight tasks can enter the queue at once; that is, worker count rises to eight. But all eight threads are still inside the same interpreter, and that interpreter's lock is single. The slot stays at one. The direct consequence: adding a thread lets I/O steps overlap; it does not let CPU steps overlap. A thread releases the lock while waiting on a network reply, and another thread advances. If both want to execute bytecode at the same instant, one waits. This distinction corrects both of two common claims about threads. "A thread makes work parallel" is true only for I/O steps; CPU steps have no parallelism, only order. "Threads are useless because of the lock" misses the I/O steps. Both share the same gap: **not saying which step is meant.** The measurement below counts exactly this distinction. ## The Regime Is Modeled, Not Run This lesson does not start real threads, and that is not a convenience, it is the measurement's condition. A program that actually opens eight threads leaves which thread advances at which moment to the operating system's scheduler. The same program run twice produces two different orders; the written output will not reproduce on the reader's machine. What is meant to be measured — which step overlaps — stays buried under that noise. Instead, scheduling gets **modeled**: the ordering rule is written down, the seed is fixed, and the run gives the same number everywhere. The measurement's assumptions: - **CM13** — The core is the same as the previous lesson: eight tasks, ten steps per task, eighty steps total; the seed is fixed and the setup is deterministic. - **CM14** — The thread regime's counterpart in the model is worker 8, slot 1. - **CM15** — No real thread is started; if one were, the output would depend on the scheduler and would not reproduce. - **CM16** — The comparison baseline is the single-thread regime (worker 1, slot 1), and it gives 80 ticks on all three loads. - **CM17** — Gain is the regime's tick count subtracted from the single-thread tick count; ratio is gain divided by the single-thread tick count. **This is not a speedup factor**, it is the difference between two tick counts. - **CM18** — The lower bound is the rule built in the previous lesson: in a single slot, tick count cannot drop below CPU step count. - **CM19** — The worker sweep uses the same task sequence; only worker count changes. - **CM20** — Context-switch cost is zero in the model. A real system has this cost, and it is outside the model; what is measured is not the size of the cost, it is the bound on overlap. - **CM21** — Tasks are independent of each other; shared state, locking, and races are outside this measurement. - **CM22** — Duration is never measured. The counted unit is steps, ticks, and overlapping steps. - **CM23** — The measurement is a single run; all three loads are generated from the same core. ## The Measurement ```python """Threads: same mechanism, two loads, two separate gains.""" SEED = 20260817 CPU, IO = "cpu", "io" def make_rng(seed): state = seed % 2147483646 + 1 def draw(n): nonlocal state state = (state * 48271) % 2147483647 return state % n return draw def tasks(count=8, steps=10, io_share=7, seed=SEED): """Each task is a step sequence; io_share/10 fraction are I/O steps.""" draw, result = make_rng(seed), [] for i in range(count): result.append([IO if draw(10) < io_share else CPU for _ in range(steps)]) return result def run(jobs, workers, cpu_slots): """Returns: tick, overlapping steps, cpu steps, io steps.""" remaining = [list(j) for j in jobs] tick = overlap = cpu_steps = io_steps = 0 while any(remaining): active = [i for i, j in enumerate(remaining) if j][:workers] if not active: break slots_left, advanced = cpu_slots, 0 for i in active: step = remaining[i][0] if step == CPU: if slots_left <= 0: continue slots_left -= 1 cpu_steps += 1 else: io_steps += 1 remaining[i].pop(0) advanced += 1 tick += 1 overlap += max(0, advanced - 1) return tick, overlap, cpu_steps, io_steps LOADS = (("I/O bound", 7), ("balanced", 5), ("CPU bound", 1)) print("thread regime: eight workers, single slot") print(f"{'load':<22s} {'steps':>5s} {'single-thread':>13s} {'thread':>9s} " f"{'overlap':>8s} {'gain':>7s} {'ratio':>7s}") for label, share in LOADS: j = tasks(io_share=share) single = run(j, 1, 1)[0] tick, overlap, _, _ = run(j, 8, 1) print(f"{label:<22s} {sum(len(t) for t in j):5d} {single:13d} {tick:9d} " f"{overlap:8d} {single - tick:7d} {(single - tick) / single:7.4f}") print() print("lower bound versus reached tick") print(f"{'load':<22s} {'cpu steps':>9s} {'lower bound':>11s} {'reached':>9s} " f"{'diff':>5s}") for label, share in LOADS: j = tasks(io_share=share) tick, _, cpu_steps, _ = run(j, 8, 1) print(f"{label:<22s} {cpu_steps:9d} {cpu_steps:11d} {tick:9d} " f"{tick - cpu_steps:5d}") print() print("as worker count grows (slot stays one)") print(f"{'worker':>6s} " + " ".join(f"{label:>13s}" for label, _ in LOADS)) print(f"{'':6s} " + " ".join(f"{'tick / overlap':>13s}" for _ in LOADS)) for workers in (1, 2, 4, 8): cells = [] for label, share in LOADS: tick, overlap, _, _ = run(tasks(io_share=share), workers, 1) cells.append(f"{f'{tick} / {overlap}':>13s}") print(f"{workers:6d} " + " ".join(cells)) print() j_io, j_cpu = tasks(io_share=7), tasks(io_share=1) g_io = run(j_io, 1, 1)[0] - run(j_io, 8, 1)[0] g_cpu = run(j_cpu, 1, 1)[0] - run(j_cpu, 8, 1)[0] print(f"same mechanism, two loads: gain {g_io} ticks versus {g_cpu} ticks, " f"ratio {g_io / g_cpu:.1f}") print(f"CPU-bound load io steps {run(j_cpu, 8, 1)[3]}, " f"gain {g_cpu}; I/O-bound load io steps " f"{run(j_io, 8, 1)[3]}, gain {g_io}") ``` ``` thread regime: eight workers, single slot load steps single-thread thread overlap gain ratio I/O bound 80 80 31 49 49 0.6125 balanced 80 80 42 38 38 0.4750 CPU bound 80 80 73 7 7 0.0875 lower bound versus reached tick load cpu steps lower bound reached diff I/O bound 26 26 31 5 balanced 40 40 42 2 CPU bound 73 73 73 0 as worker count grows (slot stays one) worker I/O bound balanced CPU bound tick / overlap tick / overlap tick / overlap 1 80 / 0 80 / 0 80 / 0 2 44 / 36 52 / 28 73 / 7 4 34 / 46 44 / 36 73 / 7 8 31 / 49 42 / 38 73 / 7 same mechanism, two loads: gain 49 ticks versus 7 ticks, ratio 7.0 CPU-bound load io steps 7, gain 7; I/O-bound load io steps 54, gain 49 ``` ## Same Mechanism, a Sevenfold Difference The upper table's two ends carry this lesson's entire claim. On the **I/O-bound load**, eight workers and a single slot finish eighty steps in **31** ticks. Gain **49 ticks**, ratio **0.6125**. Overlapping steps are also **49**: the identity that tick count is found by subtracting overlapping steps from total steps shows up here exactly. On the **CPU-bound load**, the same mechanism finishes eighty steps in only **73** ticks. Gain **7 ticks**, ratio **0.0875**. The only thing that changes is step type. Task count is the same, step count is the same, worker count is the same, slot is the same, scheduler is the same. The last line's ratio is **7.0** — the difference between the gains is exactly **sevenfold**. The **balanced** load sitting in between shows both ends lie on the same curve: 80 ticks drops to **42**, gain **38**, ratio **0.4750**. As I/O share falls from seven in ten to five in ten, gain drops from 49 to 38; at one in ten, to 7. Gain behaves like a step function of I/O share, and all three measurement points point the same direction. The reading is this: **concurrency is not a speedup mechanism.** The same regime overlaps more than half the work on one load, and cannot get close to a tenth on the other. When a thread is added to a program and no gain shows up, what is missing may not be thread count, but **overlappable steps in that program**. ## The Gain's Upper Bound Is Written in the Code The last line also says where the gain comes from, placing two numbers side by side. On the CPU-bound load, I/O steps are **7**, gain is **7**. The two numbers are equal. On the I/O-bound load, I/O steps are **54**, gain is **49** — gain sits a bit below the number of overlappable steps, but never above it. The rule: **gain can never exceed the number of overlappable steps.** Since CPU steps have to be taken one at a time in a single-slot regime, the only thing that can be shaved off tick count is I/O steps. On the CPU-bound load, only seven steps could overlap; not eight, and certainly not eighty. The middle table shows the same bound from the other end. The lower bound was the rule built in the previous lesson: in a single slot, tick count cannot drop below CPU step count. On the I/O-bound load, lower bound is **26**, reached is **31**; diff **5**. On the CPU-bound load, lower bound is **73**, reached is **73**; diff **0**. The last row matters: on the CPU-bound load, the regime sits **exactly** on the lower bound. Nothing better is possible, because each of the 73 CPU steps demands its own tick. Anyone looking for improvement on this load gets nothing by playing with thread count; what needs to change is **slot count**, and that is not a number a thread can change. The **5**-tick gap on the I/O-bound load is meaningful too: **26** of the 31 ticks carry a CPU step, the remaining **5** carry only I/O steps. In those five ticks, the slot sits empty, because no live task's next step is a CPU step at that moment. The empty slot looks like a loss, but it is not fixable: which task wants a CPU step at which tick is written into the step sequence, not something the scheduler can choose. ## Where Adding Workers Stops The lower table raises worker count from 1 to 8, giving three different behaviors on the three loads. On the **CPU-bound load**, the second worker drops 80 ticks to **73**; the fourth and eighth workers add nothing. Tick count freezes at **73**, overlapping steps at **7**. Seven steps could overlap, and all seven overlapped with the second worker; nothing was left for a third. **Raising thread count has no countable payoff on this load.** On the **I/O-bound load**, the curve is not flat but it is declining: the second worker earns 36 ticks, the third and fourth together earn 10, the fifth through eighth together earn 3. Gain does not grow linearly with worker count; the first few workers already claim most of the overlappable steps. On the **balanced** load, the same shape flattens sooner: from 80 to 52, then 44, then 42. Going to eight workers over four only earns two more ticks. The table's first row carries information too: with a single worker, all three loads give **80 ticks / 0 overlapping steps**. Running eight tasks with one worker is single-threaded no matter what the regime is called. When a program's thread pool width is dropped to one, what is measured is not concurrency, only order. A comparison that does not write down pool width is unreadable for this reason. The common lesson across the three columns: **adding workers earns until overlappable steps run out, then stops.** What decides where it stops is not worker count, it is load composition. And nowhere in this table does the CPU-bound column drop below 73 — there is no way past that bound in this lesson's regime. ## What the Model Leaves Out A measurement is unreadable unless it states what it counts, so two things this model **does not** count need naming. **Context-switch cost counts as zero.** In a real system, switching from one thread to another is not free; there is state to save. The model never counts this cost, so the gain numbers above are written on the **optimistic side**. This does not invalidate the results: adding the cost only shrinks the gain, it does not flip its sign. The CPU-bound load's 7 ticks were already small; adding the cost shrinks them further, and reinforces that threads are pointless on that load. **Shared state is ignored.** The eight tasks are independent of each other; none reads what another writes. In a real program, writing to a shared dictionary from two threads requires a lock in between, and waiting on that lock adds a new kind of wait. This falls outside what the model measures, but its direction is clear: **waiting on a lock reduces overlapping steps**, because a waiting task does not advance in that tick. Both together mean this: the measured **49** and **7** are the upper bound on the gain that can be expected from threads. A real program can only approach these numbers. If a gain claim sits above this bound, something is uncounted in the measurement. A separate warning follows from this: a thread is a mechanism for **waiting** steps. If a program is computing instead of waiting, it does not fail to gain because of the lock — it fails because there is nothing to overlap. ## Summary - Opening a thread grows worker count in the model, not slot count; all eight threads still share the same interpreter's single lock. - Eight workers and a single slot drop 80 ticks to 31 on the I/O-bound load — gain 49 ticks, ratio 0.6125. The same mechanism drops 80 to only 73 on the CPU-bound load — gain 7 ticks. The difference is exactly sevenfold. - Gain can never exceed the number of overlappable steps: on the CPU-bound load, I/O steps are 7 and gain is 7. - On the CPU-bound load, the regime sits exactly on the lower bound (73 against 73); on the I/O-bound load, a 5-tick gap remains, and the slot sits empty in those five ticks. - Adding workers earns until overlappable steps run out: on the CPU-bound load, gain freezes at the second worker; on the I/O-bound load, it keeps earning, diminishingly, through the eighth. ## Next Step This lesson's last number shows a boundary: on the CPU-bound load, no matter how many workers get added, tick count does not drop below 73, because the setting that does not change is the slot. The next lesson changes that setting. Starting a separate process means a separate interpreter, and a separate interpreter means a separate slot. Its question: how far does four slots drop eighty steps on the CPU-bound load, and once slot count equals task count, does load composition still say anything?