Properties and Descriptors

The previous lesson wrote the side that joins syntax: len(n) ran a body, n[0] ran a body. Reading an attribute, though, ran no body at all — n.items was a dict lookup and nothing stepped in between. This is exactly the gap the third lesson left open: a write that bypasses the body protecting the invariant stays unsupervised, and naming did nothing to stop it.

This lesson measures catching access itself. The measurement’s axis does not change: what is asked is still who answers the call, but this time the call does not look like a method call — it looks like a field read. A read and a write to an attribute can also be routed through a body; routing it does not change the spelling, it changes the cost. Three questions are asked: how many bodies does the same access sequence run across two designs, where does the intervening code hold its data, and of the two hooks catching access, which one runs how many times?

How Many Bodies Did Access Run Through

Three classes offer the same single field, and all three are used from outside with the spelling n.value. Direct keeps the field bare. WithProperty defines the field as a property: it writes one body for reading and one for writing. WithDescriptor hands the same check off to a descriptor — an object carrying the access logic in its own class, built once in the class body.

All three classes are given the same job: increment the value three times, then try writing an invalid value.

• CD52 — All three classes offer their field under the name value; the calling side writes the same lines on all three.
• CD53 — The measured work is seven accesses: three increments each produce one read and one write, and the last line adds one more write. Records made during setup do not count.
• CD54 — The invalid value is deliberately negative; the check sits inside the written body and raises ValueError. Without the check, the value is written as given.
• CD55 — The “final value” column is the value read from the field after the measurement ends; this read is not counted.

"""How many times does access run through intervening code: a plain field, a property, a descriptor."""

LOG = []


class Direct:
    """The field is bare; access runs through no body at all."""

    def __init__(self):
        self.value = 0


class WithProperty:
    """Access runs through two intervening bodies."""

    def __init__(self):
        self._value = 0

    @property
    def value(self):
        LOG.append("read")
        return self._value

    @value.setter
    def value(self, new_value):
        LOG.append("write")
        if new_value < 0:
            raise ValueError("value cannot be negative")
        self._value = new_value


class Field:
    """Descriptor: carries the access logic in its own class."""

    def __set_name__(self, owner, name):
        self.name = "_" + name

    def __get__(self, obj, owner=None):
        LOG.append("read")
        return getattr(obj, self.name)

    def __set__(self, obj, new_value):
        LOG.append("write")
        if new_value < 0:
            raise ValueError("value cannot be negative")
        setattr(obj, self.name, new_value)


class WithDescriptor:
    """Hands the same check off to a descriptor."""

    value = Field()

    def __init__(self):
        self.value = 0


def measure(cls):
    LOG.clear()
    n = cls()
    LOG.clear()
    for _ in range(3):
        n.value = n.value + 1
    try:
        n.value = -1
    except ValueError as e:
        error = type(e).__name__
    else:
        error = "none"
    return list(LOG), n.value, error


print(f"{'class':<14s} {'reads':>6s} {'writes':>6s} {'total bodies':>13s} "
      f"{'final value':>11s} {'negative write':>14s}")
for cls in (Direct, WithProperty, WithDescriptor):
    log, final, error = measure(cls)
    print(f"{cls.__name__:<14s} {log.count('read'):>6d} "
          f"{log.count('write'):>6d} {len(log):>13d} {final:>11d} {error:>14s}")

class           reads writes  total bodies final value negative write
Direct              0      0             0          -1           none
WithProperty        3      4             7           3     ValueError
WithDescriptor      3      4             7           3     ValueError

Same lines, three different costs. Direct runs 0 bodies across seven accesses; access goes straight to the instance’s dict and back. WithProperty and WithDescriptor run 7 bodies on the same seven accesses — three reads, four writes. The number matches the access count exactly, because the intervening code runs on every access, not the first or some of them.

The cost’s payoff is in the last two columns. On the unchecked field, the negative write passed and the final value became -1; the class was left in an invalid state and no one knows it. On the two classes with an intervening body, the same write fell with ValueError and the final value stayed at 3, its last valid value. This is exactly where the third lesson’s gap closes: the only place protecting an invariant is a body, and the way to route n.value = -1 through that body is to put the field behind a definition.

The most important property of this design is that nothing changes on the calling side. All three classes are used with the spelling n.value; whether the field is bare or answered by two bodies is invisible from the calling side. Putting a field under a check later can be done without changing a single line that uses it. This is why hiding fields behind get–set methods up front is unnecessary: a bare field is a field a body can be put behind whenever needed.

If two definitions give the same result, what decides between them? A property keeps its bodies inside the class itself; it is the direct spelling for a check specific to one class. A descriptor moves the logic to a separate class, and that class can be reused in many places; if the same check is needed on ten fields, a property asks for ten pairs of bodies, a descriptor asks for ten lines.

The same arrangement has two more common uses, and both pay the same measured number. First is a read-only field: only a read body is written, no write body is written at all, and assignment to the field falls with AttributeError. Fields whose value is set at construction and should never change afterward are declared this way. Second is a computed field: instead of returning a stored value, the read body produces it fresh every time. Both look like an ordinary field from outside, and this is exactly where the appearance is misleading — n.value reads like a free dict lookup, but a running body sits behind it. Reading the same field a thousand times inside a loop means running a body a thousand times. This is why the intervening body is expected to stay cheap; an expensive operation is not presented as a field, it is presented as an explicit method call.

Where a Descriptor Holds Its Data

A descriptor has a trap, and it explains why the first measurement’s definition wrote to the instance’s own dict. A descriptor object is built in the class body; that means there is one per class, and every instance shares it. A descriptor holding its data on itself ends up sharing that data too.

• CD56 — Two descriptors do the same job; the only difference is where they write the data.
• CD57 — Three instances are built and each writes its own index; then all three are read.
• CD58 — The last column is the total entry count across the three instances’ own dicts; their contents are not printed.

class HeldOnSelf:
    """Holds the data on the descriptor object itself."""

    def __get__(self, obj, owner=None):
        return self.value

    def __set__(self, obj, new_value):
        self.value = new_value


class HeldOnInstance:
    """Holds the data in the accessed instance's dict."""

    def __set_name__(self, owner, name):
        self.name = "_" + name

    def __get__(self, obj, owner=None):
        return getattr(obj, self.name)

    def __set__(self, obj, new_value):
        setattr(obj, self.name, new_value)


class BadBox:
    value = HeldOnSelf()


class GoodBox:
    value = HeldOnInstance()


print(f"{'class':<10s} {'what three see':>16s} {'single descriptor':>18s} "
      f"{'in instance dict':>17s}")
for cls in (BadBox, GoodBox):
    trio = [cls() for _ in range(3)]
    for i, n in enumerate(trio):
        n.value = i
    definition = cls.__dict__["value"]
    print(f"{cls.__name__:<10s} {str([n.value for n in trio]):>16s} "
          f"{str(all(type(n).__dict__['value'] is definition for n in trio)):>18s} "
          f"{sum(len(n.__dict__) for n in trio):>17d}")

class        what three see  single descriptor  in instance dict
BadBox            [2, 2, 2]               True                 0
GoodBox           [0, 1, 2]               True                 3

Three instances wrote three separate values. On BadBox, all three read back the last value written; the first two writes were lost, and the instances’ own dicts carry 0 entries. On GoodBox, each instance reads back what it wrote itself, and 3 entries were opened. The middle column gives the reason: on both classes, all three instances share a single descriptor object. When a descriptor writes its data onto itself, all three instances’ writes go to the same field on that same single object.

This is the same arrangement as the first lesson’s container-in-the-class-body measurement, turning up in another guise: anything built in the class body is shared across instances. A descriptor, too, is built in the class body. This is why it takes the accessed instance as its first argument — the place it will store its data is that instance, not itself.

Where the name used to write to the instance comes from is a separate question too. A descriptor is bound in the class body with value = ..., but the object itself does not know that name; there is a hook called while the class is being built that lets it learn the name given to it. Both the first measurement’s definition and HeldOnInstance here write that hook, and produce the name they will use on the instance from it. Without writing the hook, the name would have to be given by hand, and if the same descriptor were used on two fields, both would write to the same place — another form of BadBox’s result.

The answer to “who” in this measurement has two layers. The class answering the call is the descriptor’s class — BadBox’s own body has no body at all for value. But which data the answer looks at is decided not by the answering side, it is decided by where the data is written. When the two come apart, the measurement comes apart with them.

Two Hooks, Two Frequencies

The last form of catching access is not for individual fields, it is for all attribute access. There are two hooks, and they are easy to confuse: one runs on every access, the other runs only when normal lookup fails.

• CD59 — A single class writes both hooks; both only keep a log and do not disturb normal lookup.
• CD60 — Four names are read: two exist in the instance dict, two exist nowhere. Records made during setup are cleared.
• CD61 — What is measured is how many times each hook runs; returned values are not printed.

TRACE = []


class Hooked:
    """Two hooks: one on every access, the other only when lookup fails."""

    def __init__(self):
        self.first = 1
        self.second = 2

    def __getattribute__(self, name):
        TRACE.append(("every access", name))
        return object.__getattribute__(self, name)

    def __getattr__(self, name):
        TRACE.append(("not found", name))
        return None


n = Hooked()
TRACE.clear()
READS = ("first", "second", "third", "fourth")
for name in READS:
    getattr(n, name)

print(f"{'name read':<12s} {'in instance':>12s} {'every-access hook':>18s} "
      f"{'not-found hook':>15s}")
for name in READS:
    print(f"{name:<12s} {str(name in ('first', 'second')):>12s} "
          f"{sum(1 for k, a in TRACE if k == 'every access' and a == name):>18d} "
          f"{sum(1 for k, a in TRACE if k == 'not found' and a == name):>15d}")

print()
print(f"reads {len(READS)}, every-access hook ran "
      f"{sum(1 for k, _ in TRACE if k == 'every access')} times, not-found hook "
      f"{sum(1 for k, _ in TRACE if k == 'not found')} times")

name read     in instance  every-access hook  not-found hook
first                True                  1               0
second               True                  1               0
third               False                  1               1
fourth              False                  1               1

reads 4, every-access hook ran 4 times, not-found hook 2 times

Across four reads, the first hook ran 4 times, the second 2. The numbers say where each hook sits: the first is in front of lookup and runs no matter what name is read; the second is behind lookup and only kicks in when the search comes back empty.

The practical consequence of this split is cost and risk. Once a hook running on every access is written, all of the class’s attribute reads — including method access — pass through that body; in this measurement four reads became four calls, but on a real class that number is as large as everything the class does. Worse, using self. inside that body’s own code retriggers the hook and sets up an infinite chain; this is why the measured body calls the common base’s lookup directly.

The hook that runs only on names not found, by contrast, is cheap in this measurement: it never appears on the path of names that exist. Answering names that do not exist, offering access delegated to another object, or computing a missing field on demand is exactly what this hook is for. The rule is: a per-field definition to check what exists, a not-found hook to answer what does not.

Reading the three measurements together shows that attribute access is not a single dict lookup. When a name is read, the class’s resolution order is checked first; if a descriptor that also has a write body is found there, its read body runs and the instance’s dict is never even asked. If no such definition exists, the instance’s dict is checked. If it is not there either, lookup falls through to ordinary class values and read-only definitions, and finally the not-found hook kicks in. This order also explains why the first measurement’s check could not be skipped: even if an entry with the same name were opened on the instance, a definition with a write body takes priority on read. The first lesson’s “instance first, then class” rule is the middle of this chain; what this lesson adds is the bodies writable at the chain’s front and back. Controlling access means placing a link in this chain — and every link is paid on every access.

One asymmetry in the chain is worth noting too: the write side is not as rich as the read side. A descriptor can catch writes too, but code writing directly to the instance’s dict runs through none of the class’s bodies. The 0 block the third lesson measured still holds after this lesson; the only thing that changes is that the customary spelling now runs through a body.

Summary

• An intervening definition runs on every access: seven accesses run 0 bodies on a plain field, 7 on a property or descriptor — three reads, four writes.
• The cost buys a check: an invalid write passes on a bare field and the final value becomes -1; on a class with the two bodies it falls with ValueError and the final value stays at 3.
• The calling side writes the same lines on all three classes; a bare field can therefore be put behind a body later without changing any of its callers.
• A descriptor is built once in the class body and shared by every instance: with a definition holding its data on itself, three instances read [2, 2, 2] and 0 entries open in their own dicts; with a definition holding it on the instance, [0, 1, 2] is read and 3 entries open.
• Two hooks run at two separate frequencies: across four reads, the every-access hook fires 4 times, the hook that runs only on names not found fires 2.

Next Step

Every body written up to here was written by hand. The previous lesson asked for twelve bodies for eleven syntax forms; this lesson added two more bodies for a single field’s check. But most of the classes we write repeat the same pattern: they carry a few fields, get compared by their fields, and get printed by their fields. The next lesson measures what replacing this repetition with a short definition produces: which special methods does the short definition actually write into the class’s dict, which ones are missing next to a hand-written equivalent, and how many new names does each option added to the definition put into the dict?