Exceptions

Every protocol measured up to this point followed the expected path. for asked the iterator for something and got it, len returned a number, sorted gave back a sorted list. Syntax called, a special method answered, a value came back. So what happens when a protocol fails?

Python’s answer already showed up in the previous lesson’s loop measurement: on a three-item object, the fourth __next__ call returns no value, it raises StopIteration. What ends the loop is not a return value, it is an exception. This lesson looks at that machinery itself. The concept of an exception and error handling were built in the Programming Fundamentals course; the concept was established there, what is measured here is the machinery Python realizes it with: an exception is an object, the object has a class, and catching looks at that class’s lineage.

An Exception Is an Instance of a Class

What is raised is not a message, it is an object. The raise statement takes an object, the except clause takes a class, and the two are compared with isinstance. This one sentence determines the rest of the lesson: because except writes a class name, all of that class’s subclasses get caught too.

The lineage starts from two ends. Every exception class derives from BaseException. Right below it stands Exception, and everything expected to be handled inside the program descends from there. Below Exception sit intermediate classes — ArithmeticError, LookupError, OSError — and their leaves are concrete classes like ZeroDivisionError, KeyError, FileNotFoundError.

These intermediate classes are not empty decoration. Catching a leaf catches only that leaf; catching an intermediate class catches every leaf beneath it. How many it catches is a measurable number, and that number is exactly what this lesson measures.

The Measurement’s Setup

The measurement builds eleven events: small functions each of which really raises an exception. Against them sit seven catch classes — two leaves, three intermediate classes, and two roots. Every event is tried against every catch class; the result is a 77-pair table.

The events split into three tags, and the tag is the oracle: because we built the setup ourselves, we know which event is really a domain error, which is a program defect, which is an interrupt. A domain error is something expected to be handled — a missing key, an unparseable string, a file that cannot be found. A program defect is a mistake in the code itself, and it needs not to be handled but to escape. An interrupt is a request to end the program, and it must not be swallowed. StopIteration is a fourth tag: not an error, part of the protocol.

The measurement’s assumptions:

• EF1 — All eleven events are really run, and the object they raise is caught and kept; no class name is written by hand — every class name in the table is read from the raised object.
• EF2 — The tag is an oracle and comes from the setup itself; it is not derived by looking at the message or the class name.
• EF3 — A pair “matching” means the catch class covers the event’s object through isinstance; this is exactly the test the except clause performs.
• EF4 — The lineage chain is read from the class’s resolution order; object is not counted, because every class descends from it and it is not distinctive.
• EF5 — The measurement is not a performance measurement; what is counted is the number of events caught, not time.

Measurement — the Catch Matrix

"""Exception hierarchy: catching which class also catches which events."""
import math

LOG = []


class Tracker:
    """The core of the shared setup; only the iteration part is used here."""

    def __init__(self, items=(1, 2, 3)):
        self.items = list(items)

    def __iter__(self):
        LOG.append("__iter__")
        self._i = 0
        return self

    def __next__(self):
        LOG.append("__next__")
        if self._i >= len(self.items):
            raise StopIteration
        value = self.items[self._i]
        self._i += 1
        return value


def zero_division():
    return 1 // 0


def overflowing_power():
    return math.exp(10000)


def missing_key():
    return {"north": 1}["slope"]


def overflowing_index():
    return [1, 2, 3][9]


def unconvertible_string():
    return int("north")


def missing_file():
    return open("missing/measurement.txt")


def exhausted_iterator():
    return next(iter(Tracker(())))


def wrong_type():
    return len(5)


def undefined_name():
    return unknown_name


def missing_attribute():
    return (1).north


def exit_request():
    raise SystemExit


EVENTS = (
    ("zero division",        zero_division,        "domain"),
    ("overflowing power",    overflowing_power,    "domain"),
    ("missing key",          missing_key,          "domain"),
    ("overflowing index",    overflowing_index,    "domain"),
    ("unconvertible string", unconvertible_string, "domain"),
    ("missing file",         missing_file,         "domain"),
    ("exhausted iterator",   exhausted_iterator,   "protocol"),
    ("wrong type",           wrong_type,           "defect"),
    ("undefined name",       undefined_name,       "defect"),
    ("missing attribute",    missing_attribute,    "defect"),
    ("exit request",         exit_request,         "interrupt"),
)

CATCHES = (
    ("ZeroDivisionError", ZeroDivisionError),
    ("ArithmeticError",   ArithmeticError),
    ("LookupError",       LookupError),
    ("ValueError",        ValueError),
    ("OSError",           OSError),
    ("Exception",         Exception),
    ("BaseException",     BaseException),
)


def throw(function):
    try:
        function()
    except BaseException as e:
        return e
    return None


print("event                   class              place in the hierarchy")
for name, function, _ in EVENTS:
    s = type(throw(function))
    chain = " < ".join(k.__name__ for k in s.__mro__ if k is not object)
    print(f"  {name:21s} {s.__name__:18s} {chain}")

print()
print("catch class         catches  domain  protocol  defect  interrupt  escapes")
for cname, cls in CATCHES:
    counts = {"domain": 0, "protocol": 0, "defect": 0, "interrupt": 0}
    for name, function, kind in EVENTS:
        if isinstance(throw(function), cls):
            counts[kind] += 1
    caught = sum(counts.values())
    print(f"  {cname:18s} {caught:7d} {counts['domain']:5d} {counts['protocol']:9d}"
          f" {counts['defect']:6d} {counts['interrupt']:8d} {len(EVENTS) - caught:6d}")

print()
matches = sum(isinstance(throw(f), cls)
            for _, f, _ in EVENTS for _, cls in CATCHES)
print(f"events {len(EVENTS)} | catch classes {len(CATCHES)} | "
      f"pairs {len(EVENTS) * len(CATCHES)} | matching pairs {matches}")
for kind in ("defect", "interrupt"):
    items_list = [name for name, _, k in EVENTS if k == kind]
    print(f"{kind} {len(items_list)}: {', '.join(items_list)}")

LOG.clear()
throw(exhausted_iterator)
print(f"empty Tracker: {LOG.count('__iter__')} times __iter__, "
      f"{LOG.count('__next__')} times __next__ — that one call returned no value, "
      f"it raised StopIteration")

event                   class              place in the hierarchy
  zero division         ZeroDivisionError  ZeroDivisionError < ArithmeticError < Exception < BaseException
  overflowing power     OverflowError      OverflowError < ArithmeticError < Exception < BaseException
  missing key           KeyError           KeyError < LookupError < Exception < BaseException
  overflowing index     IndexError         IndexError < LookupError < Exception < BaseException
  unconvertible string  ValueError         ValueError < Exception < BaseException
  missing file          FileNotFoundError  FileNotFoundError < OSError < Exception < BaseException
  exhausted iterator    StopIteration      StopIteration < Exception < BaseException
  wrong type            TypeError          TypeError < Exception < BaseException
  undefined name        NameError          NameError < Exception < BaseException
  missing attribute     AttributeError     AttributeError < Exception < BaseException
  exit request          SystemExit         SystemExit < BaseException

catch class         catches  domain  protocol  defect  interrupt  escapes
  ZeroDivisionError        1     1         0      0        0     10
  ArithmeticError          2     2         0      0        0      9
  LookupError              2     2         0      0        0      9
  ValueError               1     1         0      0        0     10
  OSError                  1     1         0      0        0     10
  Exception               10     6         1      3        0      1
  BaseException           11     6         1      3        1      0

events 11 | catch classes 7 | pairs 77 | matching pairs 28
defect 3: wrong type, undefined name, missing attribute
interrupt 1: exit request
empty Tracker: 1 times __iter__, 1 times __next__ — that one call returned no value, it raised StopIteration

Reading the Numbers

The upper table gives the lineage and shows the depth is not fixed. ValueError reaches the root in two steps, FileNotFoundError in three. The difference between them is not a detail: every intermediate step corresponds to an except clause that would catch at that step.

The lower table counts the cost of this. Catching ZeroDivisionError matches 1 event and misses 10. One step up, ArithmeticError matches 2 — zero division and the overflowing power. LookupError also matches 2, the missing key and the overflowing index. The way to write a single except line that handles two separate events is to write their common ancestor.

The OSError row shows 1, because only one event in the setup descends from it. The number is the setup’s count, not the class’s scope: many leaves sit beneath OSError, and all of them would land on this row. The lesson writes the count it itself measured.

What a Broad Catch Swallows

The two root rows are the table’s central finding. Catching Exception matches 10 events and misses only 1. 6 of these ten are domain errors — exactly what is expected to be handled. But the same row also swallows 3 program defects: wrong type, undefined name, missing attribute. None of these three come from user input or the file system; they come from a mistake in the code itself. A block written as except Exception catches these too, silently, and the program keeps running in its broken form.

It also swallows 1 protocol event: StopIteration. The last line states where this comes from: on an empty Tracker, __iter__ was called 1 time, __next__ 1 time, and that single call raised StopIteration instead of returning a value — one more than the item count. A broad catch can mistake a loop’s normal end for an error and fall into its own branch.

BaseException goes one step further: it matches all 11 events, missing 0. The one extra thing it catches is the exit request, and this is the lesson’s sharpest number. Exception misses it — missing it is by design. A request to end the program is not a domain error and is not something to be handled; writing except BaseException means swallowing that request too. This is the only difference between the two root rows, and in the table it stands as 11 − 10 = 1.

The pattern is this: as the catch class climbs, the events it matches grow, but only part of what it matches is something that should be handled. 28 of 77 pairs match, and 21 of these 28 matches come from the two root rows. Narrowing means shrinking what is matched and zeroing out what is swallowed.

Re-Raising and the Cause Chain

Catching an exception does not mean it has to be handled. Catching it, logging it, and then re-raising it is a common need, and Python offers a few distinct forms for this. What differs between the forms is the identity of the object that escapes and whether the inner object survives.

Every exception object carries two fields. __context__ points to the other exception if this one was born while that other one was being handled, and the interpreter sets this up on its own. __cause__, by contrast, is only filled in when from is written, and it is the declaration “this exception’s cause is that one.” from None suppresses the context.

• EF6 — Five re-raising forms are produced from the same source; because the source produces a new object on every call, the forms do not share each other’s object.
• EF7 — Whether the objects are the same is tested with is; no identity number is printed.
• EF8 — The chain length is counted by following unsuppressed context and cause links; because no exception escapes in the swallowing form, its length is zero.
• EF9 — The branch table runs the three states separately, and whether each branch ran is read from a log placed inside the branch; the order is not guessed, it is measured.

"""Re-raising and the chain: which object escapes, does the inner one survive."""

STEPS = []


def source():
    """Produces a domain error; a new object forms on every call."""
    return {"north": 1}["slope"]


def bare():
    try:
        source()
    except KeyError:
        raise


def wrap():
    try:
        source()
    except KeyError:
        raise RuntimeError("could not read the measurement")


def with_cause():
    try:
        source()
    except KeyError as e:
        raise RuntimeError("could not read the measurement") from e


def suppress():
    try:
        source()
    except KeyError:
        raise RuntimeError("could not read the measurement") from None


def swallow():
    try:
        source()
    except KeyError:
        return None


FORMS = (("bare raise", bare), ("wrapping", wrap),
            ("with from", with_cause), ("with from None", suppress),
            ("swallowing", swallow))

print(f"{'form':<17s}{'escapes as':<15s}{'cause':<14s}"
      f"{'context':<14s}chain")
for name, function in FORMS:
    try:
        function()
    except BaseException as exc:
        escapes = type(exc).__name__
        cause = type(exc.__cause__).__name__ if exc.__cause__ else "-"
        context = type(exc.__context__).__name__ if exc.__context__ else "-"
        length, cur = 0, exc
        while cur is not None:
            length += 1
            cur = cur.__cause__ or (None if cur.__suppress_context__
                                else cur.__context__)
    else:
        escapes = cause = context = "-"
        length = 0
    print(f"  {name:<15s}{escapes:<15s}{cause:<14s}{context:<14s}{length}")

print()
try:
    try:
        source()
    except KeyError as e:
        inner = e
        raise
except KeyError as exc:
    print("does a bare raise send the same object out:", exc is inner)

try:
    try:
        source()
    except KeyError as e:
        inner = e
        raise RuntimeError("could not read the measurement") from e
except RuntimeError as exc:
    print("with from, is the outgoing object the same:", exc is inner)
    print("with from, is the cause the same object:", exc.__cause__ is inner)

print()
BRANCHES = ("try", "except", "else", "finally")
print(f"{'state':<16s}" + "".join(f"{b:<9s}" for b in BRANCHES))
for state, function in (("no error", lambda: 1),
                     ("caught", source),
                     ("uncaught", lambda: 1 // 0)):
    STEPS.clear()
    try:
        try:
            STEPS.append("try")
            function()
        except KeyError:
            STEPS.append("except")
        else:
            STEPS.append("else")
        finally:
            STEPS.append("finally")
    except ZeroDivisionError:
        pass
    print(f"  {state:<14s}"
          + "".join(f"{('ran' if b in STEPS else 'skipped'):<9s}"
                    for b in BRANCHES))

form             escapes as     cause         context       chain
  bare raise     KeyError       -             -             1
  wrapping       RuntimeError   -             KeyError      2
  with from      RuntimeError   KeyError      KeyError      2
  with from None RuntimeError   -             KeyError      1
  swallowing     -              -             -             0

does a bare raise send the same object out: True
with from, is the outgoing object the same: False
with from, is the cause the same object: True

state           try      except   else     finally  
  no error      ran      skipped  ran      ran      
  caught        ran      ran      skipped  ran      
  uncaught      ran      skipped  skipped  ran

What the Chain Costs

Bare raise sends KeyError out, and the line below states it is the same object. Because no new object is produced, the chain is 1. Catching and re-raising loses no information.

The wrapping form sends RuntimeError out; __cause__ is empty but __context__ holds the KeyError. The chain is 2. Nobody wrote this link — it formed on its own because the exception was born while another exception was being handled.

In the form written with from, __cause__ is filled and the chain is again 2. The difference is in the output’s lower lines: the escaping object is not the same as the inner one, but __cause__ points to the same object. Wrapping and writing from give the same chain length; where they part ways is whether the link was declared or formed on its own.

from None brings the chain down to 1. __context__ is still filled, but because it is suppressed it does not enter the count — someone looking from outside cannot see the KeyError. For a boundary that wants to hide a domain error, the right tool is one used knowing the inner detail is lost.

The swallowing form gives 0: nothing escapes at all. Combined with the broad catch from the previous section, this zero is dangerous — this is exactly where the three program defects got swallowed.

The last table gives which of the four branches ran in which state. try ran in all three states; finally ran in all three too — even while an uncaught exception was escaping. else only ran when no exception occurred, except only when one was caught. Keeping the source call inside try and moving the success path into else keeps the except clause from accidentally catching too much.

Summary

• An exception is an object, the object has a class, and except takes a class; catching covers all of that class’s subclasses.
• Eleven events and seven catch classes produce 77 pairs, and 28 of them match; leaf classes match 1–2 events, while Exception matches 10 and BaseException matches 11.
• Catching Exception swallows 6 domain errors alongside 3 program defects and 1 protocol event; BaseException additionally swallows the exit request.
• A bare raise sends the same object out (chain 1); wrapping and writing from make the chain 2, from None brings it down to 1, swallowing to 0.
• finally runs in all three states, else only when no exception occurs; keeping the success path inside else narrows the except clause’s scope.

Next Step

Every class in this lesson came ready-made: KeyError, ValueError, OSError. The measurement could group them by common ancestor because the lineage was already built. But what if what is raised is the domain’s own error — a measurement device reporting an out-of-range value, a field missing from a record? Then it is us who write the class, and where we hang it in the lineage decides which of the later catches will match. The next lesson measures a custom exception class’s place in the hierarchy and what that place costs in catching behavior.