Data Classes

The previous two lessons counted written bodies: twelve bodies for eleven syntax forms, two more for a single field’s check. But most of the classes we write repeat the same pattern. They carry a few fields, fill those fields at construction, get compared by their fields, and get printed by their fields. Writing this pattern by hand in every class both grows longer and carries the risk of forgetting a field on every write.

The standard library offers a short definition form for this pattern. This lesson’s question is not convenience, it is measurement: which special methods does the short definition actually write into the class’s dict, which ones are missing next to a hand-written equivalent, and how many new names does each option added to the definition put into the dict?

What a Short Definition Writes

Five classes carry the same two fields: a string and an integer. HandWritten writes its construction, representation, and equality bodies by hand. The remaining four are data class definitions: fields are declared by name and annotation, no bodies are declared. The four definitions differ from each other only in the options given — one is the default, one asks for ordering, one is frozen, one is both.

The measurement looks for ten special method names in each class’s own dict. Names coming from ancestors do not count; what is asked is what the short definition writes into that class.

• CD62 — All five classes’ fields are the same; their names, order, and annotations are the same too.
• CD63 — The “source lines” row counts how many filled source lines each definition holds; blank lines do not count, the decorator line does.
• CD64 — The ten sought names are looked for only in the class’s own dict; answering_class is not used, because what is measured is whether the name is written, not where it comes from.
• CD65 — The “hash value” row looks not at whether the name exists but at whether its value is empty; the emptying the previous lesson measured shows up here again.

"""Which special methods does a short definition actually write into the class dict."""

import inspect
from dataclasses import dataclass


class HandWritten:
    def __init__(self, name, number):
        self.name = name
        self.number = number

    def __repr__(self):
        return f"HandWritten(name={self.name!r}, number={self.number!r})"

    def __eq__(self, o):
        if not isinstance(o, HandWritten):
            return NotImplemented
        return (self.name, self.number) == (o.name, o.number)


@dataclass
class Default:
    name: str
    number: int


@dataclass(order=True)
class Ordered:
    name: str
    number: int


@dataclass(frozen=True)
class Frozen:
    name: str
    number: int


@dataclass(frozen=True, order=True)
class FrozenOrdered:
    name: str
    number: int


CLASSES = (HandWritten, Default, Ordered, Frozen, FrozenOrdered)
SOUGHT = ("__init__", "__repr__", "__eq__", "__hash__", "__lt__", "__le__",
          "__gt__", "__ge__", "__setattr__", "__delattr__")


def source_line(cls):
    """How many filled source lines the definition holds."""
    return sum(1 for s in inspect.getsource(cls).splitlines() if s.strip())


print(f"{'special method':<16s}" + "".join(f"{s.__name__:>14s}" for s in CLASSES))
for name in SOUGHT:
    print(f"{name:<16s}"
          + "".join(f"{('written' if name in s.__dict__ else '.'):>14s}"
                    for s in CLASSES))

print(f"{'total written':<16s}"
      + "".join(f"{sum(1 for a in SOUGHT if a in s.__dict__):>14d}"
                for s in CLASSES))
print(f"{'source lines':<16s}" + "".join(f"{source_line(s):>14d}" for s in CLASSES))
print(f"{'hash value':<16s}"
      + "".join(f"{('none' if s.__hash__ is None else 'set'):>14s}"
                for s in CLASSES))

special method     HandWritten       Default       Ordered        Frozen FrozenOrdered
__init__               written       written       written       written       written
__repr__               written       written       written       written       written
__eq__                 written       written       written       written       written
__hash__               written       written       written       written       written
__lt__                       .             .       written             .       written
__le__                       .             .       written             .       written
__gt__                       .             .       written             .       written
__ge__                       .             .       written             .       written
__setattr__                  .             .             .       written       written
__delattr__                  .             .             .       written       written
total written                4             4             8             6            10
source lines                10             4             4             4             4
hash value                none          none          none           set           set

The first two rows are the comparison itself. The hand-written definition writes 4 names across 10 source lines; the short definition writes the same 4 names in 4 lines. The written names are identical — the short definition leaves nothing out and adds nothing extra. What the measurement really says is that the short definition’s gain is not just six lines. In the hand-written class, every field name is written in all three of the three bodies — construction, representation, and equality. Adding a new field means all three bodies have to be updated; in the short definition, a field is declared once. An equality body that silently compares too little because of a forgotten update is this pattern’s most expensive mistake, and it never raises an exception.

The fourth row shows the same mechanism the previous lesson measured, turning up again. __hash__ is written in all five classes’ dicts, but the row below says its value is empty in the first three. Every class writing equality loses hashing capability, and the short definition is no exception to this rule. The hand-written class is in the same situation, for the same reason. The result is data classes’ most common surprise: an object comparable by value cannot be put into a set or used as a dict key unless it is frozen.

The options show clearly what they add to the dict. The ordering option adds 4 comparison methods and raises the total to 8. The freezing option adds 2 methods catching the write and delete paths, raises the total to 6, and brings back the hash value. Given together, the total comes to 10 — a four-line definition produces a ten-body class.

This table also defines what the short definition is: not a definition, a write operation. Field declarations in the class body are read, and the corresponding bodies are placed into the class’s dict. Every name placed in the dict is identical to what could be hand-written — this is why hand-writing any method on a class written with the short definition also overrides it, since it is not written on top of a name already present in the body.

What is not written also has to be counted. The measurement searched for ten names, and most of the fourth lesson’s eleven syntax forms are not on this list: length, iteration, membership, index access, and context syntax do not come from the short definition. The short definition writes a record, not a container. If a data class is meant to behave like a container, the bodies needed for that are still written by hand.

How fields are found also leaves a point open. The short definition reads which names are fields from the annotations in the class body — the type name in the spelling name: str declares the field’s existence. What that type name does at runtime is a separate question, measured in this course’s second topic; what can be said here is that the short definition uses it only as a marker.

A Mutable Default

What the short definition rejects is measurable too, not just what it writes. The Python Fundamentals course measured that a default value is evaluated once, when the function is defined, and a mutable default is shared across calls; that measurement is not repeated here. What is asked here is what the same spelling produces in two definition forms.

• CD66 — Three definitions try setting up the same field with the same default; the first is the hand-written class, the others are the short definition.
• CD67 — A definition that fails to build records the exception’s name with an exclamation prefix, and that row’s measurement columns are left blank.
• CD68 — Three instances are built from every definition that builds, and each appends its own index to the container; sharing is shown with is.

from dataclasses import field


class HandBox:
    """The default value is evaluated once; three instances share it."""

    def __init__(self, items=[]):
        self.items = items


def short_definition(default):
    """Try building the same default with the short definition."""
    try:
        @dataclass
        class Temp:
            items: list = default
    except ValueError as e:
        return f"!{type(e).__name__}", None
    return "built", Temp


DEFINITIONS = (
    ("hand-written, default []", "built", HandBox),
    ("short definition, default []", *short_definition([])),
    ("short definition, factory function", *short_definition(field(default_factory=list))),
)

print(f"{'definition':<36s} {'at definition time':>19s} {'what three see':>17s} "
      f"{'container shared':>17s}")
for name, status, cls in DEFINITIONS:
    if cls is None:
        print(f"{name:<36s} {status:>19s} {'—':>17s} {'—':>17s}")
        continue
    trio = [cls() for _ in range(3)]
    for i, n in enumerate(trio):
        n.items.append(i)
    print(f"{name:<36s} {status:>19s} {str([len(n.items) for n in trio]):>17s} "
          f"{str(trio[0].items is trio[2].items):>17s}")

definition                            at definition time    what three see  container shared
hand-written, default []                           built         [3, 3, 3]              True
short definition, default []                 !ValueError                 —                 —
short definition, factory function                 built         [1, 1, 1]             False

The first row gives the known result: the hand-written definition builds, three instances share one container, and each sees 3 items. The error does not surface at runtime either; the program silently runs the wrong way.

The second row is the short definition’s difference. The same spelling falls with ValueError at definition time; the class never even gets built, and the place the error surfaces is the very line it was written on. The third row shows the accepted spelling: when what is given is not a value but a factory function to be called for every instance, the definition builds and three instances see [1, 1, 1] — the container is not shared.

This is the same arrangement as the first lesson’s class-body container measurement, turning up for the third time. What the short definition adds here is not a new rule, it is moving an existing rule earlier: instead of leaving a known-wrong spelling to runtime, it rejects it at definition time.

How narrow this rejection is also has to be written down. The short definition only recognizes a handful of mutable types it knows about; when a mutable class we wrote ourselves is given as a default, the definition builds, and the sharing gives the same result as the first row. What is caught is not a general rule, it is a commonly made spelling. The rule itself has not changed: a default value is built once at definition time, and any container that should belong to the instance is declared with a factory function.

Freezing’s Measure

The last option closes an object’s writability. The justification for immutability and the concept of a value object were established in the Software Design and Architecture Principles curriculum; that discussion is not repeated here. What is measured here is how many operations a single option closes, and what it opens in exchange.

• CD69 — Two classes carry the same two fields; the only difference is whether they are frozen.
• CD70 — Five operations are tried, and each runs on a fresh instance; the operations do not affect each other.
• CD71 — The last line compares two separate objects carrying the same values; identity is shown with is and no identity number is printed.

from dataclasses import replace

OPERATIONS = (
    ("write to field", lambda n: setattr(n, "number", 9)),
    ("delete field", lambda n: delattr(n, "number")),
    ("open new field", lambda n: setattr(n, "extra", 1)),
    ("new object in its place", lambda n: replace(n, number=9)),
    ("put into a set", lambda n: len({n})),
)


def attempt(cls, action):
    try:
        action(cls("k", 1))
    except Exception as e:
        return f"!{type(e).__name__}"
    return "passed"


print(f"{'operation':<25s} {'Default':>21s} {'Frozen':>21s}")
for name, action in OPERATIONS:
    print(f"{name:<25s} {attempt(Default, action):>21s} {attempt(Frozen, action):>21s}")

print()
for cls in (Default, Frozen):
    fell = sum(1 for _, f in OPERATIONS if attempt(cls, f).startswith("!"))
    print(f"{cls.__name__}: fell out of five {fell}, "
          f"passed {len(OPERATIONS) - fell}")

a, b = Frozen("k", 1), Frozen("k", 1)
print(f"two separate objects: a is b -> {a is b}, a == b -> {a == b}, "
      f"items in the set -> {len({a, b})}")

operation                               Default                Frozen
write to field                           passed  !FrozenInstanceError
delete field                             passed  !FrozenInstanceError
open new field                           passed  !FrozenInstanceError
new object in its place                  passed                passed
put into a set                       !TypeError                passed

Default: fell out of five 1, passed 4
Frozen: fell out of five 3, passed 2
two separate objects: a is b -> False, a == b -> True, items in the set -> 1

Freezing closes 3 operations and opens 1. All three that close are writes: writing to a field, deleting a field, and opening a new field. The third is especially notable — it closes the same door as the third lesson’s attribute-set declaration, but for a different reason. The operation that opens is putting into a set: since a frozen object has a hash value, it can be a key in a container; an ordinary data class cannot.

The fourth row passes on both classes and shows how freezing is meant to be used. An immutable object is not mutated, a new one is put in its place; the function the standard library provides produces a new object with the requested fields changed. Writing is blocked, rebuilding is not.

The last row defines what a frozen data class is: is False, == True, 1 item in the set. Two separate objects stand in for each other because they carry the same values. Being defined by value rather than identity is the sum of these three results — and that is exactly what this lesson’s definition produces.

Freezing’s limit also has to be written down, because the table does not show it. What is closed is the path of writing to a field, not the object the field points to. If a frozen object’s field holds a list, appending an item to that list meets no resistance at all — the third lesson’s measurement’s last row applies here exactly the same way. Immutability is one layer deep; an object being truly unable to change depends on its fields also being unchangeable objects. The hash value carries the same limit: if a frozen class carries a list in a field, the put-into-a-set row falls with TypeError, because a hash is produced from its fields.

Summary

• The short definition writes, across 4 source lines, the same 4 special methods a hand-written 10-line class writes into the class’s dict; the written names are identical.
• As with any class writing equality, the short definition writes the __hash__ name and leaves its value empty; hashing capability only returns with freezing.
• Options add names to the dict: ordering adds 4 comparison methods and raises the total to 8, freezing adds 2 methods and raises it to 6, both together make it 10.
• A mutable default builds in the hand-written definition, and three instances share one container and see [3, 3, 3]; in the short definition it falls with ValueError at definition time, and written with a factory function, [1, 1, 1] is seen.
• Freezing closes 3 of five operations and opens 1: three write paths fall, putting into a set passes, and putting a new object in its place passes on both classes.
• For two separate frozen objects, is is False, == is True, and 1 item remains in the set.

Next Step

This lesson counted what a class writes by looking at its dict; the previous lessons also always searched for the answer in the class tree. Both methods rest on the same assumption: knowing what an object can do requires knowing which class it comes from. But the fourth lesson had already shown the opposite — joining came not from the type’s name, it came from the methods written. Wrapper derives from nothing, yet answers all four of the four capabilities. The next lesson turns this duality into a test: does the same object pass a check that searches for ancestry, or one that searches for methods, and which of the two — and what does an abstract base’s declared contract enforce, and at what point, and what does the method-searching test actually look at?