Asynchronous Programming

The previous two lessons always got their overlap from outside. Threads asked the operating system for workers, processes asked a separate interpreter for slots. In both, what decided when a task advances sat outside the code; the program only stated how many flows it wanted.

A third path exists, and this lesson measures it. In asynchronous programming, the source of overlap is neither the operating system nor a second interpreter: it is the code itself. A coroutine writes, on its own, the point where it gives up control. This lesson’s question is narrow and countable for that reason: how much overlap comes from how often a coroutine gives up control, and what happens if it never does?

A Coroutine Is a Resumable Body

A coroutine is a function whose body does not run when called; the call returns an object instead, and the body advances only as the coroutine is resumed. It is defined with async def. The await inside its body is not a marker waiting for a result, it is the point where control gets given up: the coroutine stops there, returns to whatever resumed it, and lets something else advance.

A pausing body is not new to this course. The Data Structures and Functional Tools course measured the generator and showed that the body stops at yield and keeps its local names; that measurement is not repeated here. The difference between the two can be written in one sentence: a generator stops to give a value, a coroutine stops to give up control — one’s stop adds an item to a stream, the other’s stop hands the turn to something else.

The event loop is the party that manages these stops. It holds coroutines waiting to be resumed; it resumes one, and once that one reaches a yield point and returns, it resumes the next. Registering a coroutine with the loop and asking it to advance independently makes it a task.

The Event Loop Adds No New Slot

In the model’s language: the asynchronous regime asks the operating system for neither worker count nor slot count. The event loop runs in a single flow; only one body advances at a time, and the slot is one.

Where does overlap come from, then? From exactly one place: another body being able to advance when one gives up control. In this course’s measure, without a yield point, there is no overlapping step either. This gives a direct upper bound, and the measurement is about to test it: overlapping steps cannot exceed the number of yields.

The reverse also holds, and gets measured at the end of the lesson: yielding is necessary for overlap but not sufficient. There also has to be a second task, ready and waiting at that moment, to take the control that gets given up.

The Measurement Writes Its Own Loop

The measurement’s scheduler here is a hand-written event loop; the coroutines, though, are real. Bodies defined with async def and containing real await produce real coroutine objects, resumed with send. The only thing changed is who does the resuming: a loop with a written-down order, instead of the standard event loop. The reason is the measurement’s condition — overlapping steps cannot be counted unless which task gets resumed at which tick is fixed.

The short block at the lesson’s end runs the real event loop, and await itself fixes the order there.

The measurement’s assumptions:

• CM36 — The task setup is the same as the previous lessons: eight tasks, ten steps per task, eighty steps total, I/O-bound load.
• CM37 — The coroutines are real; they are defined with async def and stop at await. The loop that resumes them is written within the lesson.
• CM38 — A yield point is the number of times a coroutine returns to the event loop; every await is a yield.
• CM39 — The three regimes differ only in where the yield points sit: at every step, only at I/O steps, never. The step sequences are the same in all three.
• CM40 — The event loop is single-flow: the CPU slot is one and does not change.
• CM41 — If a coroutine takes k steps without giving up control, the loop stays busy for those k steps; the first is written to the tick it was resumed on, the remaining k-1 to separate ticks where no one advances.
• CM42 — Resume order follows task index and does not change through the run.
• CM43 — The comparison baseline is the previous lessons’ scheduler: the same tasks, eight workers, a single slot.
• CM44 — The real event loop is run only with sequential await; output order is fixed by await itself, not left to a scheduler.
• CM45 — sleep(0) is not a wait in this measurement, it is a yield marker; what is counted is not duration, it is how many times control gets given up.
• CM46 — Duration is never measured; the counted unit is steps, ticks, overlapping steps, and yields.
• CM47 — The measurement is a single run, and the seed is fixed.

The Measurement

"""Coroutines: who gives up control, and how often."""

import asyncio

SEED = 20260817
CPU, IO = "cpu", "io"


def make_rng(seed):
    state = seed % 2147483646 + 1

    def draw(n):
        nonlocal state
        state = (state * 48271) % 2147483647
        return state % n
    return draw


def tasks(count=8, steps=10, io_share=7, seed=SEED):
    """Each task is a step sequence; io_share/10 fraction are I/O steps."""
    draw, result = make_rng(seed), []
    for i in range(count):
        result.append([IO if draw(10) < io_share else CPU for _ in range(steps)])
    return result


def run(jobs, workers, cpu_slots):
    """The previous lessons' scheduler; comparison baseline."""
    remaining = [list(j) for j in jobs]
    tick = overlap = 0
    while any(remaining):
        active = [i for i, j in enumerate(remaining) if j][:workers]
        if not active:
            break
        slots_left, advanced = cpu_slots, 0
        for i in active:
            if remaining[i][0] == CPU:
                if slots_left <= 0:
                    continue
                slots_left -= 1
            remaining[i].pop(0)
            advanced += 1
        tick += 1
        overlap += max(0, advanced - 1)
    return tick, overlap


class Suspend:
    """The point where a coroutine hands control back to the event loop."""

    def __init__(self, kind):
        self.kind = kind

    def __await__(self):
        return (yield self.kind)


async def every_step(steps, log):
    """Gives up control before every step: yield points = step count."""
    for s in steps:
        await Suspend(s)
        log.append(s)


async def io_step_only(steps, log):
    """Gives up control only at I/O steps; runs CPU steps back to back."""
    for s in steps:
        if s == IO:
            await Suspend(s)
        log.append(s)


async def never(steps, log):
    """Never gives up control."""
    for s in steps:
        log.append(s)


def loop(factory, jobs, cpu_slots=1):
    """Hand-written deterministic event loop. Returns: tick, overlap, yields.

    If a coroutine takes k steps without giving up control, the loop stays
    busy for those k steps: k-1 more ticks are written as debt, and no
    task advances in those ticks.
    """
    log = [[] for _ in jobs]
    coro = [factory(j, log[i]) for i, j in enumerate(jobs)]
    pending = [None] * len(coro)
    alive = [True] * len(coro)
    debt = [0] * len(coro)
    yields = 0

    def resume(i, prepaid=0):
        nonlocal yields
        before = len(log[i])
        try:
            pending[i] = coro[i].send(None)
            yields += 1
        except StopIteration:
            pending[i] = None
            alive[i] = False
        debt[i] += max(0, len(log[i]) - before - prepaid)

    for i in range(len(coro)):
        resume(i)                       # up to the first yield point
    tick = overlap = 0
    while any(alive) or any(debt):
        busy = [i for i in range(len(coro)) if debt[i] > 0]
        if busy:
            debt[busy[0]] -= 1
            tick += 1
            continue
        slots_left, advanced = cpu_slots, 0
        for i in range(len(coro)):
            if not alive[i] or pending[i] is None:
                continue
            if pending[i] == CPU:
                if slots_left <= 0:
                    continue
                slots_left -= 1
            resume(i, 1)
            advanced += 1
        if advanced == 0:
            break
        tick += 1
        overlap += max(0, advanced - 1)
    return tick, overlap, yields


j = tasks(io_share=7)
print("I/O bound load, eighty steps, single slot")
print(f"{'yield point':<24s} {'yields':>7s} {'tick':>5s} {'overlap':>8s}")
for label, factory in (("every step", every_step),
                        ("I/O step only", io_step_only),
                        ("never", never)):
    tick, overlap, yields = loop(factory, j)
    print(f"{label:<24s} {yields:7d} {tick:5d} {overlap:8d}")

tick, overlap = run(j, 8, 1)
print(f"{'previous lesson schedule':<24s} {'':>7s} {tick:5d} {overlap:8d}")


async def stage(label, times, log):
    """Gives up control to the event loop on every iteration."""
    for i in range(times):
        await asyncio.sleep(0)
        log.append(f"{label}{i}")


async def sequential():
    """Sequential await: the second coroutine does not start until the first finishes."""
    log = []
    for label in ("a", "b"):
        await stage(label, 3, log)
    return log


print()
log = asyncio.run(sequential())
print("real event loop, sequential await:", " ".join(log))
print(f"yield points {len(log)}, overlapping steps 0 — "
      f"one yields, none receive")

I/O bound load, eighty steps, single slot
yield point               yields  tick  overlap
every step                    80    31       49
I/O step only                 54    36       44
never                          0    80        0
previous lesson schedule            31       49

real event loop, sequential await: a0 a1 a2 b0 b1 b2
yield points 6, overlapping steps 0 — one yields, none receive

Yield Frequency Decides Overlap

The table changes exactly one variable across three rows: where await gets written. The step sequences are the same in all three, task count is the same, the slot is the same.

The coroutine that yields at every step gives up control 80 times and finishes eighty steps in 31 ticks; overlapping steps 49.

The coroutine that yields only at I/O steps gives up control 54 times — exactly the I/O step count. Ticks rise to 36, overlapping steps drop to 44. The five-tick loss comes from CPU steps being taken back to back: if a coroutine carries three CPU steps between two I/O steps, it does those three in a single resumption, and the loop cannot hand the turn to anyone else during that stretch.

The coroutine that never yields gives up control 0 times, and the result is 80 ticks, 0 overlapping steps — exactly the single-thread regime itself. Being written with async def changes nothing.

The order the three rows give: yields 80 → 54 → 0, overlapping steps 49 → 44 → 0. The relationship runs one direction, and the upper bound holds in every row: overlapping steps never exceed the number of yields.

This gives a direct design consequence: what decides overlap in an asynchronous program is not how many tasks get started, it is where the bodies stop. The word async produces no yield point; await does.

The second row’s five-tick loss deserves reading on its own, because most real bodies resemble that row. The loss grows with the length of the CPU steps: the more steps between two yield points, the longer the loop stays busy. This is the countable reason long computations get moved elsewhere in asynchronous code — left in place, they make every task wait at once.

Same Schedule, Different Source

The table’s last row carries a verification, and this lesson’s most notable number sits there.

The previous lessons’ scheduler, running the same tasks with eight workers and a single slot, gave 31 ticks / 49 overlapping steps. The coroutine regime that yields at every step gives 31 ticks / 49 overlapping steps too. The two numbers match exactly.

The match is not a coincidence, it is a consequence of the definition. In the thread regime, once a task advanced, the turn passed to another; the scheduler made that call. In the coroutine regime, too, a task takes one step and stops, and the turn passes to another; the code makes that call. The schedule observed is the same, the party that writes the schedule is different.

This also settles what asynchronous programming actually earns. It does not earn more overlap — the top row shows this, the number is the same. What it earns is a different source of overlap: getting the same schedule without asking the operating system for a thread, in a single flow, by writing the yield points into the code. The cost sits in the same place: a body that forgets to write its yield points falls to the third row.

One Yields, None Receive

The last two lines show that a yield point is not enough.

Two coroutines run on the real event loop; each gives up control three times, for 6 yield points total. The output is a0 a1 a2 b0 b1 b2. The second coroutine does not advance at all until the first finishes, and overlapping steps are 0.

The reason is what await does: the sequential body waits on stage("a", ...), and until that finishes, stage("b", ...) is not started at all. Control gets given up, but there is no ready task to take it; the loop resumes the same coroutine again.

This gives a two-part rule. Overlap requires both that the body gives up control and that a second task ready to advance exists at that moment. The first is provided by await; the second, by registering coroutines with the loop as tasks. If either is missing, the result is the table’s third row: no matter how many yields there are, overlapping steps stay zero.

This distinction also explains the most common mistake in asynchronous code: the body is written async, await is in place, and nothing overlaps. The number that needs measuring is not the count of await, it is the count of tasks ready to advance at the same time.

The Cost of Yielding the Model Does Not Count

The table’s first row gives the most yields and the most overlap together; from this, “the more await, the better” might be concluded. The model confirms this, but the model does not count one thing, and it needs naming.

Every yield is bookkeeping: the coroutine’s state gets saved, put back in the loop’s queue, then resumed again. In the model, this work’s cost counts as zero; the only thing counted is which tick a step gets taken on. In a real program, yielding is not free, and a body that yields at every step pays back part of its earned overlap to that bookkeeping.

Its effect on the measurement has a direction and can be read: 31 ticks is the upper bound on the gain that can be expected from the asynchronous regime. Adding the cost only grows this number. The 36 in the second row sits between the two costs: it writes fewer yields, but loses five ticks by taking CPU steps back to back.

Bodies written in practice resemble the second row, because await gets written in front of waiting calls, not scattered among computing lines. What the measurement says: the difference between the second and third row — 44 overlapping steps against 0 — does not depend on the length of the computing lines, it depends on whether the waiting call in between gives up control or not. A long CPU stretch shrinks overlap somewhat; a single call that never yields zeroes it out.

Summary

• A coroutine is a body that does not run when called and advances only as it is resumed; await is not a wait, it is the yield point where control gets given up.
• The event loop adds neither worker nor slot; it runs in a single flow. The only source of overlap is bodies giving up control, and overlapping steps can never exceed the number of yields.
• The same eighty steps give 80 yields / 31 ticks / 49 overlapping steps when yielding is written at every step, 54 / 36 / 44 when written only at I/O steps, and 0 / 80 / 0 when never written.
• The coroutine regime that yields at every step gives exactly the same numbers as the eight-worker, single-slot schedule; what changes is not the schedule, it is who writes it.
• A yield point is necessary but not sufficient for overlap: two coroutines running with sequential await give up control 6 times and give 0 overlapping steps.

Next Step

The table’s third row is this lesson’s result, but the next lesson’s question. A body never giving up control is not always the writer’s choice: if a called function does not know how to give up control, the body waits on it, and the loop stalls. The next lesson measures this — how far overlapping steps drop when one call in the flow does not yield, what is left when no call yields, and how many steps it costs to bring a call that cannot yield back into the flow.