Built-in Functions

The previous lesson measured the four levels of name lookup and showed the names at the last rung only as something to be shadowed. Yet the names sitting at that level are the language’s most heavily used, and they connect directly to every protocol measured so far.

This lesson’s question is not what those names do — it is whether they do any work themselves. Does len compute the length itself, or does it ask the object? Does sorted know how to sort itself, or does it leave comparison to the object? The course’s measurement axis suggests an answer: if syntax is a shorthand, built-ins can be a shorthand too. The measurement tests this.

A Built-in Is a Shell

The four lessons up to here measured syntax forms: if, for, while, call notation. Built-ins are not syntax; they are ordinary functions called with a bare name. But the measurement axis is the same for both — a built-in’s number is also which special method it calls, and how many times.

The contract is this: a built-in does not do the work itself. It looks for a special method of a given name on the object, calls it if found, and gives back what it returns. len does not count the length, it asks __len__; str does not build the text, it asks __str__; iter does not produce an iterator, it requests one from __iter__. A built-in’s contribution is not the work, it is a single call name: the same name is used no matter what the object’s type is.

The consequence is that built-ins operate not by type but by protocol. What decides whether an object can be measured with len is not its class’s name, it is whether it defines __len__.

The practical payoff: a new class needs nothing to inherit or register anywhere to work with built-ins. It only defines a method carrying the expected name; the built-in finds it. The shared reference’s fourth reading gets paid a second time here — what an object can do is decided not by its type’s name, but by the special methods it defines. All a built-in knows about an object is whether the method it is looking for is there.

Collecting Built-ins

A second cluster is not satisfied with calling a single method. list, sum, and sorted want all of the object’s items; to do this they run the iteration protocol start to finish. The expected sequence has to be the same one the second lesson measured: one __iter__, followed by one more __next__ than the item count.

One consequence: collecting items is always work proportional to item count. The built-in cannot know how many items there are in advance, because the iteration protocol announces no count. Collecting ends only when StopIteration arrives — it continues until items run out.

sorted stands apart within this cluster, because after collecting it also has to order the items. Collecting is one protocol, ordering another; the second does comparisons between items. This is where the assumption that a built-in maps to a single special method gets tested.

When There Is No Protocol

The third measure is the case where the method cannot be found. The shared reference’s fourth reading already said this: no protocol, no syntax either. Does the same hold for built-ins?

It is clear from the start that the answer is not uniform, because there is a case already measured in the first lesson: the truthiness trial did not error when it found no method at all, it counted the object true. So at least one built-in has a default behavior. How many do, how many do not — the measurement separates this.

The measurement’s assumptions:

• CF31 — The oracle is the rig itself: it logs the name every time a special method is called, and which method a built-in calls is read from this log.
• CF32 — Two classes are used. Flow defines only the iteration and comparison methods; Tracker adds length and text methods to it. Collecting built-ins are measured on Flow — so the measured sequence comes only from the iteration protocol.
• CF33 — Silent defines no special method at all and is the measure of the “no protocol” state.
• CF34 — Results are not printed as values, only “produced a value” or the exception’s class is written. The text form of an object defining no method carries environment-dependent information; it does not enter the measurement.
• CF35 — Tracker.__repr__‘s body calls __str__; this is the shared reference’s behavior and is the reason for the two calls.
• CF36 — In the sorting measurement, not the call count but the set of protocols called is written. Comparison count is a detail of the sort order and is not part of the measure.
• CF37 — The total call count is specific to these eight built-ins and a three-item object.
• CF38 — In the key-function measurement, the len built-in is given as key, so every place the key is called is logged and how many times per item can be counted.

Measurement

"""The built-ins' contract: which special method does each one call."""

LOG = []


def record(name):
    LOG.append(name)


class Flow:
    """Only iteration and ordering protocol; does not define __len__."""

    def __init__(self, items=(1, 2, 3)):
        self.items = list(items)

    def __iter__(self):
        record("__iter__")
        self._i = 0
        return self

    def __next__(self):
        record("__next__")
        if self._i >= len(self.items):
            raise StopIteration
        value = self.items[self._i]
        self._i += 1
        return value

    def __lt__(self, other):
        record("__lt__")
        return len(self.items) < len(other.items)


class Tracker(Flow):
    """Also defines the length and text protocols."""

    def __len__(self):
        record("__len__")
        return len(self.items)

    def __str__(self):
        record("__str__")
        return f"Tracker{tuple(self.items)}"

    def __repr__(self):
        record("__repr__")
        return self.__str__()


class NonEmpty(Tracker):
    """__bool__ defined: truthiness does not fall to __len__."""

    def __bool__(self):
        record("__bool__")
        return len(self.items) > 0


class Silent:
    """Defines no special method at all."""


def measure(action):
    LOG.clear()
    try:
        action()
        status = "produced value"
    except Exception as e:
        status = f"!{type(e).__name__}"
    return list(LOG), status


SINGLE = (("len(n)", len, Tracker), ("str(n)", str, Tracker), ("repr(n)", repr, Tracker),
         ("bool(n)", bool, Tracker), ("iter(n)", iter, Tracker))
COLLECTING = (("list(n)", list, Flow), ("sum(n)", sum, Flow),
            ("sorted(n)", sorted, Flow))

print(f"{'builtin':<12s} {'calls':>6s}  protocol sequence called")
total = 0
for name, f, cls in SINGLE + COLLECTING:
    c, _ = measure(lambda f=f, s=cls: f(s()))
    total += len(c)
    print(f"  {name:<10s} {len(c):6d}  {' '.join(c)}")
print(f"\neight built-ins, total {total} calls")

print()
print(f"{'builtin':<12s} {'object defining protocol':<27s} object not defining it")
for name, f, cls in SINGLE + COLLECTING:
    _, d1 = measure(lambda f=f, s=cls: f(s()))
    _, d2 = measure(lambda f=f: f(Silent()))
    print(f"  {name:<10s} {d1:<27s} {d2}")

print()
c, _ = measure(lambda: bool(Tracker()))
print(f"bool(__bool__ undefined) -> {' '.join(c)}")
c, _ = measure(lambda: bool(NonEmpty()))
print(f"bool(__bool__ defined)   -> {' '.join(c)}")

print()
c, _ = measure(lambda: sorted(Flow()))
print(f"sorted(single object, items are numbers) -> {' '.join(sorted(set(c)))}")
TRIO = [Flow((1, 2, 3)), Flow((1,)), Flow((1, 2))]
c, _ = measure(lambda: sorted(TRIO))
print(f"sorted(three-object list)                -> {' '.join(sorted(set(c)))}")

print()
TRIO_TRACKER = [Tracker((1, 2, 3)), Tracker((1,)), Tracker((1, 2))]
print(f"{'three-object list':<28s} {'protocol called':<20s} is comparison asked of object")
for name, f in (("sorted(list)", lambda: sorted(TRIO_TRACKER)),
              ("sorted(list, key=len)", lambda: sorted(TRIO_TRACKER, key=len))):
    c, _ = measure(f)
    print(f"  {name:<26s} {' '.join(sorted(set(c))):<20s}"
          f" {'yes' if '__lt__' in c else 'no'}")
c, _ = measure(lambda: sorted(TRIO_TRACKER, key=len))
print(f"  key=len calls per item: {c.count('__len__')} / {len(TRIO_TRACKER)} items")

builtin       calls  protocol sequence called
  len(n)          1  __len__
  str(n)          1  __str__
  repr(n)         2  __repr__ __str__
  bool(n)         1  __len__
  iter(n)         1  __iter__
  list(n)         5  __iter__ __next__ __next__ __next__ __next__
  sum(n)          5  __iter__ __next__ __next__ __next__ __next__
  sorted(n)       5  __iter__ __next__ __next__ __next__ __next__

eight built-ins, total 21 calls

builtin      object defining protocol    object not defining it
  len(n)     produced value              !TypeError
  str(n)     produced value              produced value
  repr(n)    produced value              produced value
  bool(n)    produced value              produced value
  iter(n)    produced value              !TypeError
  list(n)    produced value              !TypeError
  sum(n)     produced value              !TypeError
  sorted(n)  produced value              !TypeError

bool(__bool__ undefined) -> __len__
bool(__bool__ defined)   -> __bool__

sorted(single object, items are numbers) -> __iter__ __next__
sorted(three-object list)                -> __lt__

three-object list            protocol called      is comparison asked of object
  sorted(list)               __lt__               yes
  sorted(list, key=len)      __len__              no
  key=len calls per item: 3 / 3 items

A One-to-One Match

The top table’s first five rows confirm the contract. len makes one call and what it calls is __len__; str one call, __str__; iter one call, __iter__. There is no code in the built-in’s body that counts a length, builds text, or produces an iterator — if there were, the call count would be zero, because there would be no need to ask the object at all.

The 2 in the repr row looks like an exception, but is not. repr still calls a single method: __repr__. The second call does not come from the built-in — the body of __repr__ itself calls __str__. The built-in’s contract is not broken; the measured sequence also shows the work the called method does itself.

The bool row reproduces the first lesson’s result: because __bool__ is not defined on the object, the trial falls to __len__. The two lines further down put the pair side by side — while __bool__ is defined, only it is called, __len__ never enters the trial. This is the one real exception to the rule that a built-in maps to a single method: bool looks for not one but two, in order.

The Collectors’ Shared Sequence

The last three rows give the same sequence: one __iter__, four __next__, total 5. Three separate built-ins — one builds a list, one takes a sum, one sorts — and all three collect items in exactly the same way.

The sequence is character-for-character the one the second lesson measured, and for the same reason it is one more: four __next__ for three items, the last one announcing collection is done by throwing StopIteration. for, list, sum, and sorted — all four call the same protocol the same number of times. The difference between them is what they do with the items collected, not how they collect them.

The practical consequence: an object defining the iteration protocol works with all of these built-ins automatically. list support, sum support, sorted support are not added one by one — the moment __iter__ and __next__ are written, all three arrive.

Sorting’s Two Protocols

The bottom two lines set sorted apart and show it does not map to a single method.

In the first line, the sorted object’s items are numbers. The protocols called are __iter__ and __next__ — collecting only. Comparison happens too, of course, but between numbers; the rig’s object never enters that comparison and nothing lands in the log.

In the second line, what is sorted is a list of three objects. This time the log holds only __lt__. Because the list is already a list, the rig’s protocol is not called during collection; during sorting, every comparison is asked of the object.

Read together, the two lines show sorted calls two separate protocols: the iteration protocol to collect items, the comparison protocol to order them. Which one lands in the log depends on which role the rig’s object is playing — the outer container, or an item inside.

The reading that follows: sorted does not know how to sort, it asks the object to compare. What decides the sort order is not the built-in’s body, it is the items’ __lt__ body. The built-in only asks which item comes before which and follows the answer it gets. The same list is arranged in a different order just by changing the items’ __lt__ definition, without a single character changing in the sorted call.

A Key Changes the Protocol

The last table measures how the same call switches to a different protocol with a keyword argument. The two rows sort the same list, the same objects; the only thing that changes is whether the key argument is given.

In the call with no key, the protocol landing in the log is __lt__ — comparison is asked of the objects. In the call with a key, the protocol in the log is __len__ and __lt__ is never called at all. The right-hand column says this directly: comparison is no longer asked of the object.

What happens: given a key, sorted first applies the key function to every item, then does the ordering on the returned key values. The objects are never compared with each other; what gets compared is their keys. This is why even objects with no __lt__ can be sorted with a suitable key — sortability becomes a property of the key, not the object.

The last line gives how many times the key is called: 3 calls for three items, exactly once per item. The key is not recomputed on every comparison; it is computed once and stored. The result is that an expensive key function is paid for once over the whole sort — no matter the comparison count.

Having a Default and Not

The middle table measures the case where the protocol cannot be found, and the eight built-ins split in two.

Five define no default at all: len, iter, list, sum, and sorted produce not a value but TypeError when the method cannot be found. For these built-ins the shared reference’s fourth reading holds literally — no protocol, no built-in either.

Three produce a value. str and repr give a text form even for an object with no method defined; the language defines a default text representation for every object. bool applies the first lesson’s third rung and counts the object true.

The reason for the split is whether the question has an answerable default. “What is this object’s length?” has no reasonable answer if the object does not say — any number made up would be wrong. “What is this object’s text form?” does have an answer producible for every object. Whether a built-in has a default depends on whether the question has an answer independent of the object.

This split has a cost that does not show in the table: built-ins with a default run silently. A class that forgot to write __len__ shows it immediately on a len call; a class that forgot __bool__ returns true from every if trial with no signal at all. The error surfacing early is a guarantee the TypeError-producing built-ins provide.

Summary

• Built-ins are not functions doing the work themselves, they are thin shells calling a special method on the object; len does not count the length, str does not build the text, iter does not produce an iterator — all three ask.
• Eight built-ins produce a total of 21 calls on a three-item object; the first five map to a single method with one call each, while bool looks for two methods in order and falls to __len__ if __bool__ is absent.
• list, sum, and sorted build the same five-call sequence — one __iter__, four __next__ — character-for-character the same as the for loop’s sequence; a class writing the iteration protocol gains all three at once.
• sorted calls not one but two protocols: iteration to collect, comparison to order. The sort order is decided not by the built-in’s body but by the items’ __lt__ body.
• With a keyword argument given, sorted never asks the object for comparison at all: the key function is called instead of __lt__ — exactly once per item — and sortability becomes a property of the key, not the object.
• On an object defining no protocol, five of eight built-ins give TypeError, three produce a value; the ones with a default are the ones whose question has an answer independent of the object, and they pay the cost of passing over an omission silently.

Next Step

Every protocol measured up to here followed the expected path. Syntax looked for a method, found it, and called it; when not found it fell to a second method; when that too was absent it either applied a default or threw a well-formed TypeError. Even the StopIteration ending a loop was not an accident, it was the protocol’s planned last step.

So what if a protocol fails? The method called is found, it runs, but cannot finish its job — a file is missing, a number cannot convert, an index falls outside the bound. In this lesson TypeError showed up once; in earlier lessons NameError, UnboundLocalError, IndexError, and StopIteration showed up. All five were classes, and all can be caught with the same notation. The next lesson builds how these classes relate to each other: which one catching also catches which others, and what a broad catch swallows.