Runtime Validation

The previous lesson moved a domain rule into the type and showed that the rule fell into a constructor’s body, meaning into run time. Writing a separate class for every domain does not scale: as the number of rules grows, constructors repeat each other, and where the rules live gets scattered.

The name for gathering rules in one place is a schema: which field is which type, and which condition it has to satisfy, sits in a definition separate from the data. Runtime validation applies that definition to the value at call time. This lesson builds that layer and asks the course’s final number: how many layers does it take to see all four violations?

The Schema Is Modeled in This Lesson

One warning, the same as the previous lesson’s: the schema and the validator here are written within the lesson. No ready-made validation layer is used, and no tool’s name appears. What is measured is not a program’s behavior, it is the approach’s scope.

Schema validation itself is not new to this course. The Data Modeling and Relational Theory course built the schema on the data side with integrity constraints; the Web API Design course built domain-level error reporting in request and response bodies too. Those procedures are not repeated here. The question here is narrow and measurable: which layer catches which violation?

The measurement’s assumptions:

• TL28 — The oracle is the setup itself: two of five calls carry a type violation, two carry a domain violation, one is clean. The array comes from the course’s shared setup and is not changed.
• TL29 — The checker and the domain rule are the same models as the previous lessons’; the checker only sees the type, the domain rule only the value’s meaning.
• TL30 — The schema holds three things in one place: field name, expected type, and field condition. The test looks at the type first, and only moves to the condition if the type passes; this order is a design decision and it shows up in the messages.
• TL31 — Validation is applied as a layer wrapping the function: if the test does not pass, the call never starts at all. The catching measure is whether this wrapper raises an exception.
• TL32 — The runtime column is whether the unwrapped call raises an exception; the body is a string formatting and accepts every type.
• TL33 — The “together” count is the union of the calls the two layers catch, not their sum.
• TL34 — In the scope measurement, a run is assumed where three of the five calls are executed; which three is written into the setup, and the measurement counts this not as a probability but as a single run.
• TL35 — No environment-dependent data is written; every number comes from the setup and the run.

Measurement

"""Runtime validation: how many layers does it take to see four violations."""


def describe_person(name: str, age: int) -> str:
    return f"{name}:{age}"


CALLS = (
    ("name", 30),          # valid
    ("name", "thirty"),    # type violation: age is a string
    (7, 30),               # type violation: name is a number
    ("name", -5),          # type valid, domain violation: age negative
    ("", 30),              # type valid, domain violation: name empty
)


def checker(function_, calls):
    """Modeled checker reading annotations: sees only the type."""
    signature = [(a, t) for a, t in function_.__annotations__.items() if a != "return"]
    found = []
    for i, arg in enumerate(calls):
        for (name, type_), value in zip(signature, arg):
            if not isinstance(value, type_):
                found.append((i, name))
    return found


def domain_rule(name, age):
    """Meaning conditions the type cannot see."""
    errors = []
    if isinstance(name, str) and not name:
        errors.append("name empty")
    if isinstance(age, int) and age < 0:
        errors.append("age negative")
    return errors


# The schema is modeled within the lesson: field name, type, and domain rule in one place.
SCHEMA = (("name", str, lambda d: bool(d), "empty"),
        ("age", int, lambda d: d >= 0, "negative"))


def validate(schema, values):
    """Schema-based validation: tests both the type and the domain at call time."""
    errors = []
    for (field, type_, rule, message), value in zip(schema, values):
        if not isinstance(value, type_):
            errors.append(f"{field}: type")
        elif not rule(value):
            errors.append(f"{field}: {message}")
    return errors


def guarded(schema, function_):
    """Layer wrapping the function with the schema: if validation fails, the call never starts."""
    def wrapper(*args):
        errors = validate(schema, args)
        if errors:
            raise ValueError("; ".join(errors))
        return function_(*args)
    return wrapper


found = {b[0] for b in checker(describe_person, CALLS)}
type_violations = [i for i, a in enumerate(CALLS)
              if not isinstance(a[0], str) or not isinstance(a[1], int)]
domain_violations = [i for i, a in enumerate(CALLS) if domain_rule(*a)]
guarded_fn = guarded(SCHEMA, describe_person)

raw_errors = 0
schema_caught = set()
for i, arg in enumerate(CALLS):
    try:
        describe_person(*arg)
    except Exception:
        raw_errors += 1
    try:
        guarded_fn(*arg)
    except ValueError:
        schema_caught.add(i)

print(f"{'call':>6s} {'type violation':>15s} {'domain violation':>17s} "
      f"{'runtime':>10s} {'checker':>10s} {'domain rule':>12s} "
      f"{'schema':>7s}")
for i, arg in enumerate(CALLS):
    print(f"{i:6d} {str(i in type_violations):>15s} {str(i in domain_violations):>17s} "
          f"{'no error':>10s} {str(i in found):>10s} "
          f"{str(i in domain_violations):>12s} {str(i in schema_caught):>7s}")
print()
print(f"calls {len(CALLS)}, violating "
      f"{len(set(type_violations) | set(domain_violations))} | caught by: runtime "
      f"{raw_errors}, checker {len(found)}, domain rule {len(domain_violations)}, "
      f"schema {len(schema_caught)}")
print(f"checker and domain rule together: "
      f"{len(found | set(domain_violations))}")
print(f"schema messages: {[validate(SCHEMA, a) for a in CALLS]}")

EXECUTED = [0, 2, 4]
print()
print(f"written call sites {len(CALLS)}, executed {len(EXECUTED)} | "
      f"checker saw {len(CALLS)}, caught {len(found)} | "
      f"schema reached {len(EXECUTED)}, caught "
      f"{len(schema_caught & set(EXECUTED))}")

  call  type violation  domain violation    runtime    checker  domain rule  schema
     0           False             False   no error      False        False   False
     1            True             False   no error       True        False    True
     2            True             False   no error       True        False    True
     3           False              True   no error      False         True    True
     4           False              True   no error      False         True    True

calls 5, violating 4 | caught by: runtime 0, checker 2, domain rule 2, schema 4
checker and domain rule together: 4
schema messages: [[], ['age: type'], ['name: type'], ['age: negative'], ['name: empty']]

written call sites 5, executed 3 | checker saw 5, caught 2 | schema reached 3, caught 2

Three Layers, Three Separate Catches

The table spreads four violations across three columns, and no column is a copy of another.

Runtime catches 0. All five lines say “no error.” The annotation does not stop the call; the number measured in the course’s first lesson comes out the same here too.

The checker catches 2, and what it catches are the type violations. On the domain violations, the column says false, because -5 is an integer, an empty string is a string.

The domain rule catches 2, and what it catches is exactly what the checker cannot see. The two columns do not intersect: what one finds, the other cannot find.

This has a direct consequence in the row below: the checker and the domain rule together catch 4 — all of the violations. Not a sum, a union; because the two layers’ scopes do not overlap, the union comes out equal to the sum. All four violations only become visible when the two layers are used together.

The schema column, by contrast, catches on 4 of the five calls. The reason is that the schema carries two pieces of information in one place: type and condition. The message list gives this in detail — type on two calls, negative on one, empty on one. So the schema does not bring a new ability to see; it merges what the checker knows and what the domain rule knows into the same definition and applies both at call time.

The message list also exposes one of the schema’s design decisions. On the call whose age is a string, the message is age: type; on the one whose age is negative, it is age: negative. Two messages never come out for the same field, because the test looks at the type first and does not move to the condition until the type passes. This order is mandatory: the comparison d >= 0 cannot be performed with a string. A domain condition assumes the type is correct; when writing a rule, knowing where that assumption comes from matters as much as writing the rule itself.

The wrapping layer has one more difference. When validation does not pass, the call never starts at all; the body does not run, no half-finished work is left behind. The situation measured in the earlier lessons — “run time only falls while doing the operation” — disappears here: the fall sits in front of the call.

This has a cost, and it shows up in the measurement: the wrapper applies the schema from start to finish on every call, violation or not. The clean call gets tested too. The checker, by contrast, does nothing at run time; it does its job once, before the run. The two layers’ costs are of a different kind, just like their scopes: one, once per run; the other, every time, per call.

Scope: Written Code and Reached Call

The last line shows the schema’s cost, and this cost is the course’s measure’s final word.

Written call sites: 5, executed in a single run: 3. The checker sees all five of the five and declares 2 violations — without ever running the code. The schema only tests the 3 calls it reaches, and catches 2 violations in those three; the violations on the two calls it does not reach are invisible in that run.

The pattern is this: the checker’s scope is the written code, validation’s scope is the reached call. Writing the schema more carefully does not close this gap, because the gap is not in carefulness, it is in when the layer runs. In the same way, writing the checker more carefully does not make it find domain violations either — that too is a knowledge limit.

The two layers do not substitute for each other, and they do not confirm each other either: one knows a little, everywhere; the other knows a lot, in some places.

The scope difference also decides where validation gets placed. Putting the schema in front of every function grows the test count with the call count, and the same value gets tested over and over in the same run. Placing a single validation, by contrast, at the point where the data enters the program produces a guarantee for every call after that point — the test is done once, the result is carried inside. This is why the schema in the measurement also carries the field name: when a violation is declared, which field is at fault has to be readable far from where the test was performed too.

This is the scaled-up form of what the Age class in the previous lesson did. There, the rule was written into a single constructor’s body, and a separate class was needed for every field; here, the rules are gathered into a single definition, and the validator reads that definition. What changes is not when the rule runs — both are run time — it is where the rule is written.

Summary

• A schema gathers a field’s name, expected type, and domain condition into a single definition; runtime validation applies that definition at call time, and if validation does not pass, the call never starts.
• Three layers catch three separate things: runtime 0, the checker 2, the domain rule 2. The two layers’ scopes do not intersect, so their union is 4 — all four violations only become visible together.
• Schema-based validation alone catches 4, because it merges what the checker and the domain rule know into the same definition; it adds no new ability to see.
• The scopes are separate: the checker sees all 5 of the five written call sites, the schema tests the 3 calls reached in a single run and catches 2 violations in those three.
• The layers’ limits are of different kinds; the checker’s is a knowledge limit, validation’s is a scope limit, and neither closes the other by being more careful.

Course Wrap-Up

The course opened with a single question and measured that same question across eleven lessons in two halves: who answered the call, who checked the claim? In the first half, the answer was a class; in the second, a layer. In both, what was measured was not what the design did, it was where the responsibility stood.

Lesson	Measured	The answerer or catcher
Class and Instance	whether an attribute sits on the class or the instance: 1 against 3 objects	the namespace that finds the name first — instance dictionary first, then the class
Inheritance and Method Resolution Order	where the body of four capabilities is written: 0/4 against 4/4	the first class in resolution order; Wrapper in composition
Encapsulation Conventions	how many of four access forms give the value: 3, blocked 0	the method body — not the name
Special Methods	syntax forms a class writing no special method takes part in: 5, answers: 0	the silent default; in a class that writes it, the method itself
Properties and Descriptors	the body the same seven accesses run: 0 for a direct field, 7 for an intervening definition	the intervening body; a descriptor is created once on the class
Data Classes	special methods a short definition writes: 4; with options 8, 6, and 10	the methods the short definition writes into the class dictionary
Abstract Base Classes and Protocols	of three candidates, 1 passes the lineage test, 3 pass the method test	both tests; but passing does not say the call will be answered
Type Hints	4 of five calls are violations, erroring at runtime 0	no layer
Generics and Type Variables	violations four notations declare: 0, 4, 3, 2	the side reading the annotation — if a bond is built
Static Type Checking	the checker finds 2, cannot see 2	the checker; its scope is the written code
Runtime Validation	runtime 0, checker 2, domain rule 2	schema-based validation — 4 on the calls it reaches

The table’s two halves give the same pattern. In inheritance, reading where a body is written did not say where the answer would come from; an annotation being written into the code did not say the claim would be checked either. In both cases, the answer to the question does not sit in the writing itself, it sits in the mechanism that resolves it: resolution order in one, a checking layer in the other.

All these measurements shared an assumption that was never written down: a single thread. When the class giving the answer was asked, a single resolution chain ran; when the layer catching the violation was asked, calls ran in sequence, one finishing before the next began. This is why the question “who answered” always had a single answer.

When there is more than one thread, this question reopens. When the same object is reached from two places at once, which write left the attribute, whether a value that passed validation is still valid while the body runs, and which of the two threads acted first can no longer be read by looking at the source. The next course, Concurrency and Performance, starts from exactly this point: in a single thread, who answered is certain; in more than one, this question gets asked again, and this time its answer depends on an ordering.