Formatters and Linters

The previous lesson measured coverage and showed what coverage does not count: eight of eight lines ran, three of three tests passed, and yet in one of four input combinations the real behavior deviated from the oracle. Coverage counts where the code ran.

This lesson steps to the side: who unifies how the code is written? How many separate forms can the same function take on a team, and how many does that number drop to once it is handed off to a tool? For the question to be measurable, “form” has to have a precise definition; this lesson establishes that definition and reads the number.

Two Separate Tool Classes

Two tool classes are often named in the same sentence and confused.

A formatter rewrites the source. Its input is a source text, its output is a source text too; the two share the same parse tree, they differ in layout. What a formatter produces is single: whichever text feeds the same tree, the same text comes out.

A linter reports on the source. Its input is a source text, its output is a list of findings; it does not touch the source. Some findings concern layout, some do not — and this distinction sits at the center of what this lesson measures.

The two classes’ output is also of a different kind, and this is the most concrete difference between them. A formatter’s output replaces the source; a linter’s output stands beside it. The first is a transformation and it cannot be reversed, because the layout information that gets deleted cannot be brought back. The second is an observation and leaves the source as it is; it is still the writer who reads the finding and decides what to do.

Handing a formatting debate off to a tool was measured in the Code Review and Team Process course, as one item of the review axis; the question asked there was how many of a review’s comments were about formatting. Here the question is different, and it is not concerned with the debate itself: how many separate forms of the same source are there, and how far does that number drop?

Setting Up the Model

No lesson in this course writes a tool’s name; tool behavior is modeled inside the lesson and it is said plainly that it is modeled. The model here has three parts.

Forms. The shared setup’s three-decision discount function is written with a single canonical text. Five more forms are produced by applying three layout transformations to this text: changing the indentation step, putting an if body on the same line, stripping the spaces around an operator. No transformation touches the function’s behavior.

Formatter. The model is the standard library function that parses the source and rewrites it from the tree. A real formatter preserves comments, applies a line-width rule, and breaks long expressions; this model does none of that. The only thing the model unifies is layout, and the measurement only asks about layout.

Linter. There are two rule families. Rules that look at text scan the source’s lines: is the indentation step four spaces, are there two statements on one line, is there a space on both sides of an operator. Rules that look at the tree walk the parse tree: is there an unnamed numeric constant, does the function’s decision count exceed a threshold. The second family never sees how the source is written, because the tree does not carry layout.

The measurement’s assumptions:

• QP1 — Six forms are produced from a single canonical text; the only difference between them is the indentation step, where the body sits on the line, and the space around an operator.
• QP2 — The forms’ identity is tested two separate ways: the parse tree’s dump, and the return values across four input combinations. Both stand as separate columns in the table.
• QP3 — The formatter model does not keep comments or blank lines; the measurement is therefore done on a comment-free source, and this limit of the model is stated here.
• QP4 — Three rules that look at text and two rules that look at the tree are the linter’s entirety. These are not a rule catalog, they are the smallest set that suffices to separate the two families.
• QP5 — The finding set is compared as a sorted tuple; if two forms’ finding sets are identical, one distinct outcome is counted, even if their texts are separate.
• QP6 — The discount function’s body is as in the shared setup and is not changed; the flawed ceiling stays in place. The oracle is written separately and applies a ceiling of 25.
• QP7 — No duration is measured, no file is written. What is counted is distinct text, distinct tree, distinct behavior, and distinct finding set.

Measurement

"""Reducing form to one outcome: six forms, how many distinct outcomes."""

import ast
import re

CANONICAL = (
    "def discount(amount, member, coupon):\n"
    '    """A function with three decisions."""\n'
    "    rate = 0\n"
    "    if member:\n"
    "        rate += 10\n"
    "    if coupon:\n"
    "        rate += 15\n"
    "    if rate > 20:\n"
    "        rate = 20\n"
    "    return amount - amount * rate // 100\n"
)
OPERATORS = ("+=", "//", ">", "-", "*")


def reindent(k, step):
    return "\n".join(" " * (step * ((len(s) - len(s.lstrip())) // 4))
                     + s.lstrip() for s in k.split("\n"))


def to_one_line(k):
    return re.sub(r"(?m)^(\s*)(if .+:)\n\s+(.+)$", r"\1\2 \3", k)


def unspace_operators(k):
    for op in OPERATORS:
        k = k.replace(f" {op} ", op)
    return k


FORMS = {
    "four spaces": CANONICAL,
    "two spaces": reindent(CANONICAL, 2),
    "eight spaces": reindent(CANONICAL, 8),
    "one-line body": to_one_line(CANONICAL),
    "unspaced operator": unspace_operators(CANONICAL),
    "both together": unspace_operators(to_one_line(reindent(CANONICAL, 2))),
}


def layout_findings(source):
    """Rules that look at text; the formatter silences these."""
    findings = []
    indents = [len(s) - len(s.lstrip()) for s in source.split("\n") if s.strip()]
    if min((g for g in indents if g > 0), default=4) != 4:
        findings.append("indentation step is not four spaces")
    for no, s in enumerate(source.split("\n"), 1):
        if re.match(r"\s*(if|for|while) .*:\s*\S", s):
            findings.append(f"line {no}: two statements on the same line")
        for op in OPERATORS:
            if op in s and f" {op} " not in s:
                findings.append(f"line {no}: no space around {op}")
    return findings


def structure_findings(source):
    """Rules that look at the tree; the formatter does not silence these."""
    tree = ast.parse(source)
    findings = [f"unnamed constant {d.value}" for d in ast.walk(tree)
                if isinstance(d, ast.Constant) and isinstance(d.value, int)
                and d.value not in (0, 1)]
    decisions = sum(1 for d in ast.walk(tree) if isinstance(d, ast.If))
    return sorted(findings + ([f"decision count {decisions}, threshold 2"]
                               if decisions > 2 else []))


def format_source(source):
    """Formatter model: source is rewritten from the tree by a single rule."""
    return ast.unparse(ast.parse(source)) + "\n"


def behavior(source):
    ns = {}
    exec(compile(source, "<form>", "exec"), ns)
    return tuple(ns["discount"](100, m, c)
                 for m in (False, True) for c in (False, True))


def oracle(amount, member, coupon):
    """Oracle: correct behavior applies a 25 ceiling."""
    rate = min((10 if member else 0) + (15 if coupon else 0), 25)
    return amount - amount * rate // 100


print(f"{'form':<20s} {'lines':>5s} {'layout':>6s} {'structure':>9s} "
      f"{'behavior':>22s}")
for name, k in FORMS.items():
    print(f"{name:<20s} {len(k.strip().split(chr(10))):5d} "
          f"{len(layout_findings(k)):6d} {len(structure_findings(k)):9d} "
          f"{str(behavior(k)):>22s}")

K = list(FORMS.values())
B = [format_source(k) for k in K]
before = {tuple(sorted(layout_findings(k) + structure_findings(k))) for k in K}
after = {tuple(sorted(layout_findings(b) + structure_findings(b))) for b in B}
print()
print(f"distinct source text {len(set(K))} | distinct parse tree "
      f"{len({ast.dump(ast.parse(k)) for k in K})} | distinct behavior "
      f"{len({behavior(k) for k in K})} | distinct text after formatting "
      f"{len(set(B))}")
print(f"linter: {len(before)} distinct finding sets before formatting, "
      f"{len(after)} after")
print(f"remaining findings {len(sorted(after)[0])}: {', '.join(sorted(after)[0])}")
print()
deviating = [(m, c) for m in (False, True) for c in (False, True)
             if behavior(CANONICAL)[2 * m + c] != oracle(100, m, c)]
print(f"in {len(deviating)} of four input combinations, behavior deviates "
      f"from the oracle: {deviating}")

form                 lines layout structure               behavior
four spaces             10      0         6      (100, 85, 90, 80)
two spaces              10      1         6      (100, 85, 90, 80)
eight spaces            10      1         6      (100, 85, 90, 80)
one-line body            7      3         6      (100, 85, 90, 80)
unspaced operator       10      6         6      (100, 85, 90, 80)
both together            7     10         6      (100, 85, 90, 80)

distinct source text 6 | distinct parse tree 1 | distinct behavior 1 | distinct text after formatting 1
linter: 5 distinct finding sets before formatting, 1 after
remaining findings 6: decision count 3, threshold 2, unnamed constant 10, unnamed constant 100, unnamed constant 15, unnamed constant 20, unnamed constant 20

in 1 of four input combinations, behavior deviates from the oracle: [(True, True)]

Six Forms, One Outcome

The bottom line gives the lesson’s central number: 6 distinct source texts, 1 distinct parse tree, 1 distinct behavior, 1 distinct text after formatting.

The three numbers have to be read together. The six texts really are separate — compared as files, all six come out different, and they appear as six separate lines in a change log. Yet the tree is single: what the interpreter sees is the same across all six. The behavior column confirms this independently; all six forms give (100, 85, 90, 80) across the four input combinations.

What a formatter does is exactly close the gap between these three numbers. The tree was already single; the formatter unifies the text too. One rule is applied to the six and 1 text is left. This is the only guarantee the tool gives: the same text from the same tree.

The top table’s lines column shows where the cost sits. The one-line-body forms compress the function into 7 lines, the others hold at 10. The formatted result preserves none of these choices; the writer’s preference over line layout disappears. What the tool takes over is not the debate, it is the decision.

This number’s practical counterpart is that the source becomes comparable. If the difference between two forms does not reach the tree, that difference takes up space in a change log but says nothing. Once six forms are run through the formatter, all of the difference between them is zeroed out; every difference left over is a difference that reaches the tree. The measurement says this directly: the distinct-text count drops from 6 to 1 while the distinct-tree count was already 1 and did not change.

The Rule Written in One Place

The guarantee a formatter gives is a conditional one: two runs operating under the same rule produce the same text. If the rule changes, the output changes too. So the tool alone is not enough; where the rule is written is part of the measurement.

The rule can live in three separate places. If it lives with the writer, the result is as many separate texts as there are people — the measurement’s starting state is exactly this, six forms, six separate texts. If it lives in the tool’s default, the result depends on the tool’s configuration, and when two development environments carry separate defaults, the number is more than one again. If it lives in the project’s own file, the rule travels with the source and every run reads it.

The third is the same shape as the arrangement this course built on the dependency side. A lock file did not remove resolution; it made the decision once and put it in writing. A formatting rule does not remove the debate either; it makes the debate happen once, writes the result to a file, and binds every subsequent run to that file. In both, what becomes single is not the work itself, it is the source of the decision.

The difference between the two is how reversible the decision is. When a lock file is changed, the versions that get installed change, and the program runs with different code. When a formatting rule is changed, the produced text changes but the tree does not; the measurement’s behavior column shows this across all six forms. A formatting decision is therefore a cheap decision: if it is made wrong, its cost is readability, not correctness.

Where the Linter Falls Silent

The layout column gives 0, 1, 1, 3, 6, 10 findings across the six forms. This number changes from form to form because the rule looks at text. The structure column stays at 6 across all six forms: rules that look at the tree never see layout at all, so they are unaffected by the form.

Where the numbers come from can be read one by one. In the one-line-body form, three if lines produce the “two statements on the same line” finding: 3. In the unspaced-operator form, two increment lines, one comparison line, and the three operators in the return line give findings: 6. When both are applied together these add up and the indentation-step finding is added: 10. Finding count grows not with the source’s quality but with how far the source sits from the canonical form.

The second row carries an important detail. Six forms give 5 distinct finding sets, not 6 — because the two-space and eight-space forms produce the exact same single finding: “indentation step is not four spaces.” Being a distinct text does not require being a distinct finding set. A finding set is a coarse summary of the source and can name two separate flaws with the same label.

After formatting, the finding-set count drops to 1 and 6 findings remain in that set: five unnamed constants (10, 15, 20, 20, 100) and one decision-count finding (3, threshold 2). The formatter silenced none of these, because none of them concerns layout. A formatter is not a quality tool, it is a unification tool.

Why the remaining six findings cannot be silenced gathers into one sentence too: none of them has a single correct fix. Giving the 20 constant a name requires choosing the name, and what knows the right name is not the source itself, it is the domain’s rule. Dropping the decision count from three to two requires splitting the body, and where to split cannot be read from the tree. For every layout finding, by contrast, the correct outcome is single and follows from the rule — this is why those can be handed off to a tool and the others cannot.

The last line ties off the measurement. In 1 of four input combinations, real behavior deviates from the oracle: when both membership and coupon are given, the function returns 80, while the oracle, applying its 25 ceiling, expects 75. The number of findings the linter reports is 6, the number of findings that report this deviation is 0.

This is the same shape as the previous lesson’s result, and it sits on a different axis. Coverage counted where the code ran and did not see the flaw; the linter checks how the code is written and does not see the same flaw either. Neither knows the oracle. The only thing that knows the oracle is the test’s expectation, and in this setup that expectation has written down the flawed value.

Summary

• A formatter rewrites the source and reduces separate forms to a single text; a linter does not touch the source, it reports findings.
• The same function’s 6 separate forms give 1 parse tree and 1 behavior; distinct-text count after formatting drops to 1.
• Rules that look at text produce between 0 and 10 findings across forms; rules that look at the tree stay fixed at 6 findings across all six forms.
• Finding-set count is 5 before formatting, 1 after; being a distinct text does not require being a distinct finding set.
• None of the 6 findings left after formatting reports the single call that deviates from the oracle; a formatting tool is not a quality tool.

Next Step

This lesson ran in a single environment: one interpreter, one dependency set, one run. What was measured was the source itself, and the source is independent of the environment. The same cannot be said for tests. The same team, as the shared setup showed, could give a separate decision just by order; order was a configuration detail. Interpreter version and the resolved dependency set are configuration details too. The next lesson opens this axis: running the same team across how many environments gives how many separate decisions, and which decisions bring that number back down?