Operators

The previous lesson measured that the notation x += extra does two different jobs depending on the type: in three types it rebound the name to a new object, in one it mutated the object in place. The job it does was never named. This lesson names it.

The Programming Fundamentals course built operators under the headings of precedence, associativity, and short-circuit evaluation, and also showed that the same operator can carry different meaning across different types. None of that is repeated. What was built there was the concept — an operator’s meaning depends on its operand’s type. What is measured here is the machinery of that link: which name each operator looks for, where it falls back to if it cannot find what it is looking for, and what happens if it finds nothing anywhere.

An Operator Is a Method Call

The first lesson’s setup is extended for this lesson. Tracker keeps its existing methods; subtraction, multiplication, right-hand multiplication, less-than, negation, and intersection are added on top. Every method still logs its own name when called.

• LF24 — All eleven forms measured run on two Tracker objects or one Tracker and an integer; the measurement looks not at the operator’s result but at the name of the method it calls.
• LF25 — The last form (n / m) is deliberately left unmatched: Tracker defines no method for division. The exclamation-prefixed entry on that line marks an exception, not a call.

"""Special methods called by operators; Tracker logs each participation."""

LOG = []


def record(name):
    LOG.append(name)


class Tracker:
    """The operators portion of the shared setup; six methods are added in this lesson."""

    def __init__(self, items=(1, 2, 3)):
        self.items = list(items)

    def __iter__(self):
        record("__iter__")
        self._i = 0
        return self

    def __next__(self):
        record("__next__")
        if self._i >= len(self.items):
            raise StopIteration
        value = self.items[self._i]
        self._i += 1
        return value

    def __contains__(self, x):
        record("__contains__")
        return x in self.items

    def __add__(self, o):
        record("__add__")
        return Tracker(self.items + list(o.items))

    def __iadd__(self, o):
        record("__iadd__")
        self.items.extend(o.items)
        return self

    def __eq__(self, o):
        record("__eq__")
        return isinstance(o, Tracker) and self.items == o.items

    def __sub__(self, o):                      # added
        record("__sub__")
        return Tracker([x for x in self.items if x not in o.items])

    def __mul__(self, k):                      # added
        record("__mul__")
        return Tracker(self.items * k)

    def __rmul__(self, k):                     # added
        record("__rmul__")
        return Tracker(self.items * k)

    def __lt__(self, o):                       # added
        record("__lt__")
        return len(self.items) < len(o.items)

    def __neg__(self):                         # added
        record("__neg__")
        return Tracker([-x for x in self.items])

    def __and__(self, o):                      # added
        record("__and__")
        return Tracker([x for x in self.items if x in o.items])


def measure(function):
    """Runs an operator form and returns the protocols it called."""
    LOG.clear()
    try:
        function()
    except Exception as e:
        LOG.append(f"!{type(e).__name__}")
    return list(LOG)


def in_place_addition():
    a = Tracker()
    a += Tracker()


FORMS = (
    ("n + m", lambda: Tracker() + Tracker()),
    ("n += m", in_place_addition),
    ("n - m", lambda: Tracker() - Tracker()),
    ("n * 2", lambda: Tracker() * 2),
    ("2 * n", lambda: 2 * Tracker()),
    ("-n", lambda: -Tracker()),
    ("n < m", lambda: Tracker() < Tracker()),
    ("n == m", lambda: Tracker() == Tracker()),
    ("n & m", lambda: Tracker() & Tracker()),
    ("x in n", lambda: 9 in Tracker()),
    ("n / m", lambda: Tracker() / Tracker()),
)

print(f"{'operator form':<14s} {'calls':>5s}  protocol")
for name, function in FORMS:
    c = measure(function)
    print(f"{name:<14s} {len(c):5d}  {' '.join(c)}")

logs = [measure(f) for _, f in FORMS]
unique = sorted({p for c in logs for p in c if not p.startswith("!")})
print(f"\n{len(FORMS)} forms, distinct protocols {len(unique)}, "
      f"total calls {sum(len(c) for c in logs)}, "
      f"unmatched forms {sum(1 for c in logs if any(p.startswith('!') for p in c))}")

operator form  calls  protocol
n + m              1  __add__
n += m             1  __iadd__
n - m              1  __sub__
n * 2              1  __mul__
2 * n              1  __rmul__
-n                 1  __neg__
n < m              1  __lt__
n == m             1  __eq__
n & m              1  __and__
x in n             1  __contains__
n / m              1  !TypeError

11 forms, distinct protocols 10, total calls 11, unmatched forms 1

Eleven forms call 10 distinct protocols, and 1 form goes unmatched. The table reads one way: the symbol written on the left is shorthand for the name on the right. The - operator means __sub__, * means __mul__, < means __lt__. The & row is the clearest example of this reading — what the operator does is left to the type: on integers it does a bitwise AND, here Tracker has defined it as intersection. An operator does not carry a meaning, it carries a name.

The last line is the other side of this. Because Tracker defines no method for division, the n / m form falls with TypeError. The source is flawless and passes the compile phase; the defect is that the name to call cannot be found. There is no such thing as an operator being “unsupported” — there is only an undefined method. This is also why there is exactly one way to give a type division: write a method named __truediv__.

Addition and In-Place Addition

The table’s first two rows pay off the course’s second claim. n + m calls __add__, n += m calls __iadd__. They are not shorthand for each other; separate names, separate contracts.

__add__ does not touch its operands, it produces and returns a new object. __iadd__ mutates the existing object in place and returns itself. This is exactly the distinction the previous lesson measured: if a type has defined __iadd__, += mutates the object and a second name bound to it sees the change too; if it has not, the += notation falls back to __add__, a new object is produced, and only the name on the left is bound to it. This is exactly why integer, string, and tuple came out as “name rebound” in the last lesson — these types are immutable and so cannot define __iadd__.

A practical consequence follows: if a += b is written where a is bound to an object shared with others, it also changes the value they see. The notation a = a + b never does. The two lines do not do the same job, and the choice between them is not a matter of style.

Falling Back to the Right-Hand Operand

The lines n * 2 and 2 * n call different methods: __mul__ and __rmul__. The reason is this: when 2 * n is written, the left-hand operand’s type is asked first. An integer does not know how to multiply by a Tracker, and it answers “I do not know.” At that point the right-hand operand is asked the reverse direction of the operation — the name sought is not __mul__ but the one prefixed with r, __rmul__.

This fallback is the basis of the language’s extensibility. A built-in type cannot have new behavior added to it; but a new type can join expressions where a built-in type stands on the left, through methods like __radd__, __rmul__. If the sought method is absent from both operands, the result is what the n / m line showed.

The Call Order Precedence Decides

If an expression has more than one operator, precedence decides which is called first. The Programming Fundamentals course built the precedence and associativity table; what is measured here is that table’s code-level counterpart. Precedence is not a reading convenience: it shapes the tree the second lesson showed, and that tree, in turn, gives the order methods are called in.

• LF26 — Three expressions are built on the same three objects and differ only by their operators and parentheses; what is measured is the order of the log sequence, not its length.

def multiplication_first():
    Tracker((1,)) + Tracker((2,)) * 2


def addition_first():
    (Tracker((1,)) + Tracker((2,))) * 2


def comparison_after():
    Tracker((1,)) + Tracker((2,)) == Tracker((1, 2))


print(f"{'expression':<16s} call order")
for name, function in (("n + m * 2", multiplication_first),
                  ("(n + m) * 2", addition_first),
                  ("n + m == k", comparison_after)):
    print(f"{name:<16s} {' '.join(measure(function))}")

expression       call order
n + m * 2        __mul__ __add__
(n + m) * 2      __add__ __mul__
n + m == k       __add__ __eq__

The first two lines call the same two methods, in reverse order. Because multiplication comes before addition, __mul__ runs first, and the object it produces enters __add__ as an operand; once parentheses are added, the order flips. The third line shows that arithmetic comes before comparison: equality cannot be asked before the addition finishes, because the object to compare does not exist yet. Precedence rules decide which method gets to see which object as its operand.

When a Membership Test Cannot Find Its Method

The second claim’s most visible example is the membership test. The form x in n first looks for __contains__; if it cannot find it, it falls back to the iteration protocol and scans the object from the start, comparing items one by one. The syntax is the same; the cost is not.

• LF27 — The two compared objects carry the same three items; the only difference is that Tracker defines __contains__ and Iterable does not.
• LF28 — Every test is run once with a value that is found and once with a value that is not; what is measured is the result together with the call count.

class Iterable:
    """Does not define __contains__; carries only the iteration protocol."""

    def __init__(self, items=(1, 2, 3)):
        self.items = list(items)

    def __iter__(self):
        record("__iter__")
        self._i = 0
        return self

    def __next__(self):
        record("__next__")
        if self._i >= len(self.items):
            raise StopIteration
        value = self.items[self._i]
        self._i += 1
        return value


MEMBERSHIP = (
    ("2 in Tracker()    ", lambda: 2 in Tracker()),
    ("9 in Tracker()    ", lambda: 9 in Tracker()),
    ("2 in Iterable()   ", lambda: 2 in Iterable()),
    ("9 in Iterable()   ", lambda: 9 in Iterable()),
)

print(f"{'membership test':<19s} {'result':>6s} {'calls':>5s}  protocol")
for name, function in MEMBERSHIP:
    LOG.clear()
    result = function()
    c = list(LOG)
    print(f"{name:<19s} {str(result):>6s} {len(c):5d}  {' '.join(c)}")

membership test     result calls  protocol
2 in Tracker()        True     1  __contains__
9 in Tracker()       False     1  __contains__
2 in Iterable()       True     3  __iter__ __next__ __next__
9 in Iterable()      False     5  __iter__ __next__ __next__ __next__ __next__

Tracker pays 1 call in both cases: the question asked goes straight to the object, and the object knows the answer itself. Iterable gives the same results, but at a different cost — 3 calls for a value that is found, 5 for one that is not.

The distribution of the numbers also shows how the fallback works. For the value that is found, the scan stops at the second item: one __iter__ and two __next__. For the value that is not found, all three items are read, and the fourth __next__ ends the scan by raising StopIteration — the same “one more than the item count” pattern from the first lesson. A value that is not found always costs a full scan.

That Tracker also defines __iter__ closes one more point: if both methods are defined, the in form picks __contains__, because the fallback only happens when the sought method is absent. The result can come out the same either way; the call count paid does not.

Chained Comparison and Logical Operators

Two notations remain, and neither fits the table, because the number of methods they call is not fixed.

• LF29 — The middle operand is produced by a function call, and that function also joins the log; this way, how many times the operand gets produced can be counted.
• LF30 — Tracker defines no method for a truthiness test; an empty log on the logical-notation lines is meaningful for exactly this reason. Which method a truthiness test calls is measured in the flow topic’s first lesson.

def middle():
    record("middle()")                 # how many times is the middle operand produced?
    return Tracker((1, 2))


def chain_both_true():
    Tracker((1,)) < middle() < Tracker((1, 2, 3))


def explicit_form():
    Tracker((1,)) < middle() and middle() < Tracker((1, 2, 3))


def chain_first_false():
    Tracker((1, 2, 3)) < middle() < Tracker((1, 2, 3, 4))


print(f"{'notation':<32s} {'calls':>5s}  protocol")
for name, function in (("a < middle() < c  (both true)", chain_both_true),
                  ("a < middle() and middle() < c", explicit_form),
                  ("a < middle() < c  (first false)", chain_first_false)):
    c = measure(function)
    print(f"{name:<32s} {len(c):5d}  {' '.join(c)}")

print()
print(f"{'logical notation':<12s} {'calls':>5s}  protocol")
for name, function in (("n and m", lambda: Tracker() and Tracker()),
                  ("n or m", lambda: Tracker() or Tracker()),
                  ("not n", lambda: not Tracker()),
                  ("n & m", lambda: Tracker() & Tracker())):
    c = measure(function)
    print(f"{name:<12s} {len(c):5d}  {' '.join(c) if c else '(none)'}")

print()
print("what and/or return:", repr("ab" and "cd"), repr("" or "cd"),
      repr(0 or []), "| what not returns:", repr(not "ab"))

notation                         calls  protocol
a < middle() < c  (both true)        3  middle() __lt__ __lt__
a < middle() and middle() < c        4  middle() __lt__ middle() __lt__
a < middle() < c  (first false)      2  middle() __lt__

logical notation calls  protocol
n and m          0  (none)
n or m           0  (none)
not n            0  (none)
n & m            1  __and__

what and/or return: 'cd' 'cd' [] | what not returns: False

The chained comparison calls two __lt__, but produces the middle operand only once. The manually spelled-out notation performs the same two comparisons and produces the middle operand twice: four calls against three. The difference is not just in count — it can be in correctness too: if the middle expression carries a side effect, the two notations do not give the same program. The third line shows the chain’s second half: when the first comparison comes out false, the second is never performed at all, and the call count stays at 2.

The lower table draws a boundary. and, or, and not call no special method at all; their logs are empty. These cannot be overloaded, because what they do is not an operation but a choice: they run a truthiness test on an operand and return one of the operands as-is. The last line confirms this — "ab" and "cd" does not give a truth value, it gives the string 'cd'; 0 or [] gives the empty list. Only not produces a truth value. The pair most prone to confusion is and and &: the first is a choice, the second is an operator that calls __and__.

Summary

• Every operator’s name is shorthand for a specific special method; eleven forms call 10 distinct protocols, and a form with no counterpart falls with TypeError. Precedence decides the order these methods are called in.
• n + m calls __add__, n += m calls __iadd__; the first produces a new object, the second mutates the existing one, and this difference is observable on a shared object.
• If the left-hand operand cannot support the operation, the right-hand operand’s reverse-direction method (__rmul__) is sought; if neither has it, the operator goes unmatched.
• x in n first looks for __contains__ (1 call); if it cannot find it, it falls back to the iteration protocol and pays 5 calls for a value that is not found.
• A chained comparison produces the middle operand once, and skips the second comparison entirely if the first one is false.
• and, or, and not call no special method; they return one of the operands as-is, and are distinct from & and |, which have bitwise counterparts.

Next Step

This lesson saw that the + operator produces a new object on Tracker. The same operator works on strings too, and there is not even a choice there: a string is an immutable type, so it cannot define __iadd__. The next lesson counts the cost of this — why turning a string to lowercase, stripping its whitespace, or replacing a part of it has to produce a new object every time, how formatting notations reduce this production to a single step, and how many objects come out of a chain of successive changes?