Type Conversion

The previous lesson measured that the notation str(n) calls __str__: turning an object into text is itself a protocol call. The same question can be asked in the reverse direction. What does int(n) call, what does float(n) ask of the object, and what happens when a conversion with no counterpart is requested?

The Programming Fundamentals course already established explicit and implicit conversion, lossy narrowing, and validating outside input before converting it; it also showed that the move from a real number to an integer truncates rather than rounds. None of that is repeated. What is measured here is the conversion’s machinery: which built-in looks for which name, which exception it raises when it cannot find what it is looking for, and whether what the language does when two different numeric types are added is really a “conversion.”

Conversion Is a Protocol Call Too

int, float, str, and list are built-in functions, and all of them work the same way: they ask the given object for a method carrying a specific name. If the name is absent, there is no conversion either.

• LF39 — The measured object takes part in four conversion protocols at once and logs each participation; the exclamation-prefixed entry in the log marks an exception, not a call.
• LF40 — Two counter-examples are used: an object defining no conversion method at all, and one defining only __int__ while not carrying the index protocol.
• LF41 — A fixed five-letter string is used in the indexing measurement; the letter returned depends on the number the object gives as an index.

"""Which special method do conversion built-ins call?"""

LOG = []


def record(name):
    LOG.append(name)


class Measurement:
    """Takes part in the conversion protocols and logs each participation."""

    def __init__(self, items=(1, 2, 3)):
        self.items = list(items)

    def __int__(self):
        record("__int__")
        return sum(self.items)

    def __float__(self):
        record("__float__")
        return float(sum(self.items))

    def __str__(self):
        record("__str__")
        return f"measurement {self.items}"

    def __index__(self):
        record("__index__")
        return len(self.items)

    def __iter__(self):
        record("__iter__")
        self._i = 0
        return self

    def __next__(self):
        record("__next__")
        if self._i >= len(self.items):
            raise StopIteration
        value = self.items[self._i]
        self._i += 1
        return value


class Plain:
    """Defines no conversion method."""


class IntOnly:
    """Defines only __int__; does not carry the index protocol."""

    def __int__(self):
        record("__int__")
        return 2


def measure(function):
    LOG.clear()
    try:
        result = repr(function())
    except Exception as e:
        LOG.append(f"!{type(e).__name__}")
        result = "—"
    return list(LOG), result


FORMS = (
    ("int(n)", lambda: int(Measurement())),
    ("float(n)", lambda: float(Measurement())),
    ("str(n)", lambda: str(Measurement())),
    ("list(n)", lambda: list(Measurement())),
    ("'north'[n]", lambda: "north"[Measurement((1, 2))]),
    ("int(Plain())", lambda: int(Plain())),
    ("'north'[m]", lambda: "north"[IntOnly()]),
    ("int(m)", lambda: int(IntOnly())),
)

print(f"{'notation':<14s} {'calls':>5s} {'protocol':<32s} result")
for name, function in FORMS:
    c, result = measure(function)
    print(f"{name:<14s} {len(c):5d} {' '.join(c):<32s} {result}")

notation       calls protocol                         result
int(n)             1 __int__                          6
float(n)           1 __float__                        6.0
str(n)             1 __str__                          'measurement [1, 2, 3]'
list(n)            5 __iter__ __next__ __next__ __next__ __next__ [1, 2, 3]
'north'[n]         1 __index__                        'r'
int(Plain())       1 !TypeError                       —
'north'[m]         1 !TypeError                       —
int(m)             1 __int__                          2

The first three lines establish the rule: the int built-in calls __int__, float calls __float__, str calls __str__. The correspondence between the built-in’s name and the method’s name is not a coincidence, it is the contract. The built-in is not what performs the conversion — it only knows who will do it; the object itself does the work and returns the result.

The fourth line sets apart one more conversion. list(n) does not look for a __list__ method; it uses the iteration protocol and builds a new list by collecting items one at a time. 5 calls are paid for three items: one __iter__ and four __next__ — the exact “one more than the item count” pattern from the first lesson. A general rule follows from this: conversion to a collection asks for no separate protocol, it is built on the one that already exists. For the same reason, tuple(n) and set(n) seek no new method either; all three consume the same iteration protocol and differ only in the type of container they build.

The fifth and seventh lines should be read together. Using an object as an index does not ask for __int__, it asks for __index__. The two are separate contracts: __int__ means “this object can be converted to an integer,” __index__ means “this object already is an integer position.” The distinction is not arbitrary — a real number carries __int__ but cannot serve as an index, because truncating during indexing would be a silent defect. The object that defines only __int__ shows exactly this in the last two lines: int(m) works, 'north'[m] falls with TypeError.

The sixth line is the same result as in the first lesson. On an object defining no conversion method, the int call raises TypeError. 6 of eight notations find a method, 2 cannot and fall. Conversion is not an entitlement, it is a matter of having the method.

A Failed Conversion Is an Exception

The table above measured conversion from an object to a number. Data coming from the outside world, though, is a string, and the situation there is different: the str type does not define __int__. When int("12") is written, what happens is not a protocol call but parsing the text — and parsing can fail.

• LF42 — Ten inputs are tried: eight are strings, two are of another type. Every input is tried with both int and float; what is measured is the type of the returned value or the exception.
• LF43 — The count collects only exceptions; successful conversions do not enter the count.

"""Failed conversion: which input produces which exception?"""

INPUTS = ("12", " 12 ", "+12", "1_2", "12.5", "", "north", "0x1f", None, [1])


def try_convert(function, value):
    try:
        return repr(function(value)), "—"
    except Exception as e:
        return "—", type(e).__name__


print(f"{'input':<10s} {'int(...)':>10s} {'exception':<12s}"
      f" {'float(...)':>12s} {'exception':<12s}")
counts = {}
for value in INPUTS:
    i_value, i_error = try_convert(int, value)
    f_value, f_error = try_convert(float, value)
    for h in (i_error, f_error):
        if h != "—":
            counts[h] = counts.get(h, 0) + 1
    print(f"{repr(value):<10s} {i_value:>10s} {i_error:<12s}"
          f" {f_value:>12s} {f_error:<12s}")

print(f"\ntotal attempts {2 * len(INPUTS)}, "
      + ", ".join(f"{h} {n}" for h, n in sorted(counts.items())))
print("with a base given:", "int('0x1f', 16) ->", int("0x1f", 16))

input        int(...) exception      float(...) exception   
'12'               12 —                    12.0 —           
' 12 '             12 —                    12.0 —           
'+12'              12 —                    12.0 —           
'1_2'              12 —                    12.0 —           
'12.5'              — ValueError           12.5 —           
''                  — ValueError              — ValueError  
'north'             — ValueError              — ValueError  
'0x1f'              — ValueError              — ValueError  
None                — TypeError               — TypeError   
[1]                 — TypeError               — TypeError   

total attempts 20, TypeError 4, ValueError 7
with a base given: int('0x1f', 16) -> 31

11 of twenty attempts end in an exception, and the exceptions split into two kinds: 7 ValueError, 4 TypeError. The split is not arbitrary, and this is exactly what needs to be learned.

TypeError is born when the input’s type cannot be converted. In the None and list rows, the problem is not what the value is, it is that the type carries no method for converting to a number — the same defect class as the Plain row in the first table. ValueError, by contrast, is born when the type is right but the value is not suitable. 'north' is a string, and a string can be converted to a number; what cannot be converted is the text that string carries.

The accepted inputs are instructive too. Leading and trailing whitespace is dropped, a plus sign is accepted, an underscore is read as a digit separator. But the string '12.5' is invalid for int and valid for float: int expects an integer representation and does not silently truncate a fractional text. The truncating behavior measured in the previous course happens on the move from a real number object to an integer, not from a string. The '0x1f' row is of the same class too; hexadecimal notation is only read when a base is explicitly given, and the last line shows this.

The rule that follows from this is the same as the find/index distinction from the previous lesson: a failed conversion produces not a return value but an exception. No built-in returns a special number that says “I could not convert this”; had it done so, that value could not be told apart from a real result. A noisy exception was chosen over a silent defect.

The table’s accepted column carries one more warning. The Programming Fundamentals course gave validating the format before conversion as one of two patterns; the measurement shows the limit of that pattern. A pre-check that only tests for being made of digits rejects the inputs ' 12 ' and '+12' — yet both convert without a problem. The set a pre-check accepts and the set a conversion accepts are not the same, and keeping the two sets consistent by hand means writing the conversion rules a second time. Trying and catching the fallen conversion is therefore the narrower contract: it leaves the acceptance rule in a single place, the conversion itself.

Is Conversion Reversible

A failed conversion makes noise and gets found. The real danger is in a conversion that looks successful: a value is converted, no exception is born, and converting it back does not give the original. The Programming Fundamentals course showed this loss; what is measured here is the loss’s boundary — which path is guaranteed, and which is not.

• LF44 — Five integers go and come back by two separate paths: through a string and through a real number. What is measured is whether the returned value is equal to the starting one.
• LF45 — In the real-number measurement, the return path is built through the text that identifies the object; the four chosen values are picked from opposite ends in magnitude and fraction.

"""Is conversion reversible: does the round trip give back the same value?"""

INTEGERS = (12, -3, 2 ** 53, 2 ** 53 + 1, 10 ** 23)
REALS = (0.1, 3.5, 1 / 3, 1e300)

print(f"{'integer':<26s} {'via string':>16s} {'via float':>17s}")
for n in INTEGERS:
    print(f"{n!s:<26s} {str(int(str(n)) == n):>16s} "
          f"{str(int(float(n)) == n):>17s}")

print(f"\n{'real number':<26s} {'via string':>16s}")
for x in REALS:
    print(f"{x!r:<26s} {str(float(repr(x)) == x):>16s}")

integer                          via string         via float
12                                     True              True
-3                                     True              True
9007199254740992                       True              True
9007199254740993                       True             False
100000000000000000000000               True             False

real number                      via string
0.1                                    True
3.5                                    True
0.3333333333333333                     True
1e+300                                 True

The string column reads True in all five rows. Going and coming back through text is always reversible for an integer, because — as measured in the third lesson — an integer has no fixed width: however many digits it has, the text stretches to match. The same guarantee holds for a real number too — all four rows of the lower table read True. The text that identifies a real number is produced carrying enough digits to give that number back.

The upper table’s via float column, though, splits in two: the first three values come back with no problem, the last two do not. The boundary shows up right in the table: two to the fifty-third works fine, one more than that does not. The reason is that a real number’s representation can only carry a limited number of significant digits; above that limit, two adjacent integers collapse onto the same real number, and on the way back the two cannot be told apart. This is the Python-side counterpart of the representation limit established in the How Computers Work course. The equality test reading False in the table also shows that the conversion raised no exception: both conversions completed successfully, only the result was not the starting value.

The practical consequence is singular: if an integer’s value needs to be preserved, it is not routed through a real number. Neither an exception is born nor a warning appears; only an equality test comes out wrong. The exceptions measured throughout this lesson are the exact opposite of this class — an exception makes the defect audible.

Numeric Promotion Is Not Magic

Implicit conversion remains. When 1 + 2.5 is written, the integer is “promoted” to a real number and the result comes out real. This gives the impression that the language carries a separate rule for numbers. The measurement shows this is not true.

• LF46 — The methods behind the operator are called directly; what is measured is the returned value. When the method cannot produce a result, what it returns is not an exception but a special marker.
• LF47 — Four additions are tried; in two, two distinct numeric types meet, in two, a number meets a string.

"""Numeric promotion is not magic, it is the fallback in operators."""

print("(1).__add__(2.5) ->", (1).__add__(2.5))
print("(2.5).__radd__(1) ->", (2.5).__radd__(1))
print("1 + 2.5 ->", 1 + 2.5, "| type", type(1 + 2.5).__name__)

print()
print(f"{'expression':<14s} {'result':>8s}  exception")
for left, right in (("3", 4), (3, "4"), (3, 2.5), (True, 1)):
    try:
        result, error = repr(left + right), "—"
    except TypeError as e:
        result, error = "—", type(e).__name__
    print(f"{repr(left) + ' + ' + repr(right):<14s} {result:>8s}  {error}")

(1).__add__(2.5) -> NotImplemented
(2.5).__radd__(1) -> 3.5
1 + 2.5 -> 3.5 | type float

expression       result  exception
'3' + 4               —  TypeError
3 + '4'               —  TypeError
3 + 2.5             5.5  —
True + 1              2  —

The first line is decisive. The integer’s __add__ method, given a real number, returns not a result but NotImplemented — a special value meaning “I cannot do this job.” At that point, the fallback measured in the fourth lesson kicks in: the right-hand operand’s reverse-direction method is tried, and as seen in the second line, the real number takes over the job. The third line gives the reason the result comes out real — the side that does the arithmetic is the real-number type.

So there is no separate mechanism called “promotion.” Numeric types work together because they support each other’s reverse-direction methods; this is the exact same thing as the __rmul__ fallback from the fourth lesson. The lower table draws the boundary too: a string and a number support neither direction of each other’s method, so both notations give TypeError. Which one stands on the left and which on the right does not change the result. Code that wants to add a string and a number has to write the conversion itself; the language not filling this gap on its own is not a shortcoming, it is a decision made so as not to produce a silent result.

The last line closes a Python detail. The expression True + 1 gives 2, because, as measured in the third lesson, booleans are a subtype of integer. No conversion happens here; the object already is an integer.

Summary

• Conversion built-ins call specially named methods: int calls __int__, float calls __float__, str calls __str__; if the method is absent, the conversion falls with TypeError.
• list(n) asks for no separate method, it uses the iteration protocol and pays 5 calls for three items.
• Being used as an index asks for __index__, not __int__; the first states convertibility, the second states that the object already is a position.
• 11 of twenty string attempts end in an exception: 4 TypeError when the type cannot be converted, 7 ValueError when the type is right but the value is not suitable.
• A failed conversion produces not a special return value but an exception; had it not, that value could not be told apart from a real result. The silent defect lives in a large integer that is routed through a real number and comes back different.
• Numeric promotion is not a separate rule: the integer’s __add__ method returns NotImplemented, and the real number’s reverse-direction method takes over the job.

Next Step

This lesson closed a topic while leaving one question open. When conversion fails, an exception appears, and an exception tears the flow away from where it was and carries it somewhere else — meaning it is not a defect report, it is a flow tool. The next topic takes up flow itself and opens its first lesson with this question: when a condition is written, what exactly is being tested? A number, a string, or an empty list can all be placed in an if line, and all are accepted. So who decides whether an object counts as true or false — and what happens when there is no method to make that decision?