Multiprocessing

The previous lesson hit a wall. On the CPU-bound load, raising worker count from two to eight left tick count at 73; gain froze at the second worker and never moved again. The reason was written in the table: the setting that does not change is the slot. Seventy-three CPU steps demand seventy-three ticks as long as only one CPU step can be taken per tick.

A thread cannot change that setting, because all eight threads sit inside the same interpreter. What can change it is a separate interpreter. This lesson measures that path and asks two questions: how far does four slots drop eighty steps, and once slot count equals task count, does load composition still say anything?

Separate Process, Separate Interpreter

A process is a unit of execution with its own address space. The difference between process and thread was built as mechanism in the Operating System Concepts course and is not repeated here. This lesson’s question is again a number: which setting of the model does starting a separate process change?

A separate process means a separate interpreter. The separate interpreter has its own global interpreter lock, and that lock has nothing to do with another’s. Four processes, four independent locks — that is, four slots. Its counterpart in the model is direct: the CPU slot count rises to four.

This is what a thread cannot do. A thread adds a worker; a process adds a slot. The previous lesson’s wall was not a worker wall, it was a slot wall; this is why what clears it is a process.

What Processes Pay Is Countable

A separate address space has a cost, and that cost can be written in the course’s measurement unit.

Two threads in the same process see the same object; one reads what the other writes. Two separate processes do not. Handing a task to a process means transferring its input to that process; getting the result back means transferring it back. What gets transferred is converted to a transferable representation, then rebuilt at the other end.

This work is not a duration, it is a step sequence. The model counts it exactly that way: a number of CPU steps added to both ends of every task. A transfer step is a CPU step too, and it uses the slot. The measurement’s last section asks at what size the cost eats the gain.

The measurement’s assumptions:

• CM24 — The core is the same as the previous two lessons: eight tasks, ten steps per task, eighty steps total; the seed is fixed.
• CM25 — The process regime’s counterpart in the model is the slot growing; worker stays fixed at eight so that slot is seen as the only variable.
• CM26 — No real process is started. If one were, which process advances when would depend on the operating system, and the output would not reproduce.
• CM27 — Four slots means four separate interpreters; eight slots means as many interpreters as tasks.
• CM28 — The lower-bound rule continues: tick count cannot drop below CPU step count divided by slot count, rounded up.
• CM29 — Transfer cost is the number of CPU steps added to both ends of every task: incoming data and returning result. The cost is an integer and is the same for every task.
• CM30 — A transfer step uses the slot; it has no separate resource of its own.
• CM31 — Runs with and without transfer cost are generated from the same task sequence; the comparison baseline is the single-thread regime that pays no cost (80 ticks).
• CM32 — Starting a process itself is free in the model; only data transfer is counted. In a real system, starting is costly too, and this only shrinks the measured gain.
• CM33 — Tasks are independent; shared state and coordination between processes are outside this measurement.
• CM34 — Duration is never measured; the counted unit is steps, ticks, and overlapping steps.
• CM35 — The measurement is a single run, and all three loads are generated from the same core.

The Measurement

"""Multiprocessing: adding slots, and the slot count reaching task count."""

SEED = 20260817
CPU, IO = "cpu", "io"


def make_rng(seed):
    state = seed % 2147483646 + 1

    def draw(n):
        nonlocal state
        state = (state * 48271) % 2147483647
        return state % n
    return draw


def tasks(count=8, steps=10, io_share=7, seed=SEED):
    """Each task is a step sequence; io_share/10 fraction are I/O steps."""
    draw, result = make_rng(seed), []
    for i in range(count):
        result.append([IO if draw(10) < io_share else CPU for _ in range(steps)])
    return result


def run(jobs, workers, cpu_slots):
    """Returns: tick, overlapping steps, cpu steps, io steps."""
    remaining = [list(j) for j in jobs]
    tick = overlap = cpu_steps = io_steps = 0
    while any(remaining):
        active = [i for i, j in enumerate(remaining) if j][:workers]
        if not active:
            break
        slots_left, advanced = cpu_slots, 0
        for i in active:
            step = remaining[i][0]
            if step == CPU:
                if slots_left <= 0:
                    continue
                slots_left -= 1
                cpu_steps += 1
            else:
                io_steps += 1
            remaining[i].pop(0)
            advanced += 1
        tick += 1
        overlap += max(0, advanced - 1)
    return tick, overlap, cpu_steps, io_steps


def with_transfer(jobs, cost):
    """Adds 'cost' cpu steps to both ends of every task: data transfer."""
    return [[CPU] * cost + list(t) + [CPU] * cost for t in jobs]


LOADS = (("I/O bound", 7), ("balanced", 5), ("CPU bound", 1))

print("as slot count grows (worker stays eight)")
print(f"{'slot':>5s} " + " ".join(f"{label:>13s}" for label, _ in LOADS))
print(f"{'':5s} " + " ".join(f"{'tick / overlap':>13s}" for _ in LOADS))
for slot in range(1, 9):
    cells = []
    for label, share in LOADS:
        tick, overlap, _, _ = run(tasks(io_share=share), 8, slot)
        cells.append(f"{f'{tick} / {overlap}':>13s}")
    print(f"{slot:5d} " + " ".join(cells))

j = tasks(io_share=1)
single = run(j, 1, 1)[0]
print()
print(f"CPU-bound load, single thread {single} ticks")
for slot in (1, 4, 8):
    tick, overlap, cpu_steps, _ = run(j, 8, slot)
    print(f"  {slot} slot: {tick:3d} ticks, {overlap:3d} overlapping steps, "
          f"gain {single - tick:3d} ticks, lower bound "
          f"{-(-cpu_steps // slot):3d}")

print()
print("transfer cost: 'cost' cpu steps added to each task at both ends")
print(f"{'cost':>6s} {'steps':>5s} {'four slots':>10s} {'overlap':>8s} "
      f"{'single-thread':>13s} {'gain':>7s}")
for cost in (0, 1, 2, 4, 8, 15):
    jj = with_transfer(j, cost)
    tick, overlap, _, _ = run(jj, 8, 4)
    print(f"{cost:6d} {sum(len(t) for t in jj):5d} {tick:10d} {overlap:8d} "
          f"{single:13d} {single - tick:7d}")

as slot count grows (worker stays eight)
 slot     I/O bound      balanced     CPU bound
      tick / overlap tick / overlap tick / overlap
    1       31 / 49       42 / 38        73 / 7
    2       16 / 64       23 / 57       38 / 42
    3       11 / 69       18 / 62       29 / 51
    4       10 / 70       16 / 64       20 / 60
    5       10 / 70       12 / 68       19 / 61
    6       10 / 70       10 / 70       19 / 61
    7       10 / 70       10 / 70       16 / 64
    8       10 / 70       10 / 70       10 / 70

CPU-bound load, single thread 80 ticks
  1 slot:  73 ticks,   7 overlapping steps, gain   7 ticks, lower bound  73
  4 slot:  20 ticks,  60 overlapping steps, gain  60 ticks, lower bound  19
  8 slot:  10 ticks,  70 overlapping steps, gain  70 ticks, lower bound  10

transfer cost: 'cost' cpu steps added to each task at both ends
  cost steps four slots  overlap single-thread    gain
     0    80         20       60            80      60
     1    96         24       72            80      56
     2   112         28       84            80      52
     4   144         36      108            80      44
     8   208         52      156            80      28
    15   320         80      240            80       0

Four Slots: From Eighty to Twenty

The middle block breaks down the previous lesson’s wall. On the CPU-bound load, a single slot gave 73 ticks; four slots finish the same eighty steps in 20 ticks. Gain 60 ticks, overlapping steps 60.

The comparison carries this lesson’s entire point. Threads earned 7 ticks on this load; a process earns 60. What changes is not task count, not worker count, not the scheduler — only the slot.

The lower-bound column falls into place too: 73 CPU steps divided across four slots gives a lower bound of 19, reached is 20. A one-tick gap remains, because toward the run’s end not enough live tasks are left to fill the slots. At eight slots, the lower bound is 10 and reached is also 10 — this time it sits exactly.

This gives a practical reading: on a CPU-bound load, gain comes from dividing by slot count. CPU step count is fixed; how many slots they get spread across decides tick count. A thread cannot perform this division.

When Slots Reach Task Count, the Load Disappears

The upper table’s last row is this lesson’s second measurement, and the course’s fourth reading: at eight slots, all three loads give 10 ticks and 70 overlapping steps.

Across the seven rows above, the three columns stayed apart. At a single slot, there were three separate numbers, 31, 42, and 73; at four slots, 10, 16, and 20. In the eighth row, all three land in the same place, and every difference between them disappears.

The reason is in the definition: once slot count equals task count, no task waits for lack of a slot. Every task alive at a tick takes a step. Since each of the eight tasks carries ten steps, ten ticks are required, and since eight tasks advance every tick, seven overlapping steps accumulate per tick: 10 ticks, 70 overlapping steps. Step type no longer decides anything.

This is the measurement’s most commonly misread spot. “I/O-bound work wants this regime, CPU-bound work wants that regime” is true, but conditional: it is true while the constraint holds. Once the constraint lifts, load composition produces no distinction. Asking about load composition is meaningful where slot count is less than task count — which, on a real machine, is always the case.

A Process on the I/O-Bound Load

The upper table’s first column answers a separate question: what does a process do on a load where threads already work well?

On the I/O-bound load, a single slot gave 31 ticks; four slots drop it to 10. There is a gain, but two things stand out. First, on this load there is no gain past the fourth slot — four, five, six, seven, and eight slots all give the same 10 / 70 row. The constraint is no longer the slot, it is the ten steps per task.

Second, and more important, this column does not include transfer cost. On this load, CPU steps are only 26; the data to be transferred is the same as in the thread regime. Against the 21 ticks four slots earn, adding even three transfer steps to each task’s two ends is enough to flip the account. Opening a process for waiting work means trading the wait for a transfer cost.

What can be written as a general rule: what adding a slot pays back is decided by CPU step count, and what transfer costs is decided by the size of the transferred data. The two vary independently within the same load. A process earns when CPU steps are many and transferred data is small; when the two reverse, it is the most expensive regime.

Spreading concurrency across multiple machines is the subject of the System Design and Distributed Systems curriculum and is not repeated here. Everything this course measures happens on one machine, inside one program; transfer cost is a cost within that boundary too. The difference between distributing to separate processes and distributing to separate machines collapses in the model to one number: the size of the transfer cost. The direction is the same; the scale is not.

Adding Slots Does Not Earn Linearly

The upper table’s CPU-bound column deserves reading on its own: 73, 38, 29, 20, 19, 19, 16, 10.

The second slot earns 35 ticks, the third 9, the fourth 9, the fifth 1, the sixth 0, the seventh 3, the eighth 6. The curve is neither linear nor smoothly declining. The sixth slot earns nothing; the seventh and eighth earn again.

The source of this is in the setup’s detail: gain depends on how many tasks want a CPU step at that tick. With five slots, if most ticks already have fewer than five tasks wanting one, the sixth slot sits empty. Toward the end, once tasks even out, a new slot finds work again.

The general lesson: doubling slot count does not halve tick count. Gain depends on the step sequences’ alignment at that moment, and is known only by measuring. What decides where to stop when raising slot count is not a formula, it is the difference between two consecutive measurements.

Transfer Cost Eats the Gain

The lower table writes out the process regime’s bill. At zero cost, four slots gave 20 ticks and gain was 60. Adding one transfer step to each task at both ends raises total steps from 80 to 96, tick count becomes 24, gain drops to 56. At two steps, gain is 52; at four steps, 44; at eight steps, 28.

At fifteen steps, gain is 0: 320 steps of work finish in 80 ticks on four slots — exactly where the single thread paying no cost finished. Past this point, the process regime starts losing.

The reading of this number: the gain from adding a process races against the step count of the data transfer. For a ten-step-per-task job, a thirty-step transfer cost wipes out the gain entirely. This ratio depends on the setup and is not a general threshold; what is general is the relationship itself. Distributing small tasks to processes means the transfer becomes bigger than the work.

The table’s overlapping-step column shows the trap here too. As cost rises, overlapping steps climb from 60 to 240 — the number grows while the gain shrinks. Overlapping steps mislead when read on their own: the added steps overlap too. Whatever a measurement’s denominator is, that is what needs reading; here the denominator is the single thread’s 80 ticks, and the comparison is against that.

Summary

• Adding a process grows slot count in the model: a separate process is a separate interpreter, and a separate interpreter is a separate lock. This is what a thread cannot do.
• On the CPU-bound load, four slots drop eighty steps to 20 ticks; gain 60 ticks, overlapping steps 60. On the same load, a single slot’s gain was 7 ticks.
• At eight slots, all three loads give 10 ticks / 70 overlapping steps. Once slots reach task count, CPU steps stop being the constraint, and load composition does not decide the outcome.
• The gain from adding slots is not linear: on the CPU-bound load, the second slot earns 35 ticks, the fifth 1, the sixth 0. Where to stop is found by measuring.
• Adding 15 transfer steps to each task at both ends drops four slots’ gain from 60 ticks to 0: 320 steps of work finishes in 80 ticks, the same place as the cost-free single thread.

Next Step

Up to here, both regimes got their overlap from outside: threads asked the operating system for workers, processes asked a separate interpreter for slots. In both, what decided when a task advances sat outside the code. The next lesson measures the third path: code that does not ask the operating system for overlap, and instead gives up control on its own terms. Its question: how much overlap results from how often a coroutine gives up control, and what happens if it never does?