Feedback Loop

The previous lesson showed that a documentation audit finds silent staleness: if all fifty-six claims are read, thirty-nine of them turn out to be wrong too. The audit’s problem is not that it cannot find them, it is that it never starts. It takes no place on anyone’s calendar by itself, and as long as it is not done, those thirty-nine claims stand in place.

But the documentation keeps being read during this whole time. Every day someone opens it, reads a sentence, acts on it, and gets it wrong. This lesson’s question is: how much of the audit’s job does the reader do instead? Silent staleness’s only natural enemy is the person who hits it and says so. The measurement counts how many of those people it takes.

The Claim the Reader Hits

A reader arrives at the documentation with a question and hits a claim. There are three possible cases. If the claim is true, the reader does their job and produces no feedback. If the claim is stale and embedded in an example, the reader sees an error: the example fails to run, there is a visible malfunction on the screen, and the reader reports it as a malfunction. If the claim is stale and silent, the reader sees no malfunction — they do what the document says, do not get the result they expected, and do not know what happened.

The critical part of the third case is that the reader cannot distinguish their own mistake from the document’s mistake. A document is read as an authority; a reader who gets a result that contradicts it first assumes their own error. Some try again, some give up, some find their own way, and only a portion say “what is written here is wrong.” The measurement takes this portion as a third.

From Support Request to Documentation Gap

The report arrives not as a documentation gap but as a malfunction. Turning it into a documentation record takes three questions: which sentence did the reader act on, which surface is that sentence bound to, and when did the surface change.

# example support request and documentation record, not executed

request: "I'm submitting a measurement, getting an error instead of a record id"
  claim relied on : reference / submit_metric — the source field is required
  bound surface    : signature
  changed at version: 2
  record            : stale claim, silent; 21 more claims are bound to the same surface

The last line is this job’s real gain. A report is not used to fix a single sentence, it is used to search for the other claims bound to the same surface. The reader has reported one claim; what they actually reported is that a surface changed, and twenty-two claims are bound to that surface. A single report triggers a documentation audit with a known target.

The Measurement’s Assumptions

• DO31 — Fifty-six claims, four types, and four surfaces are taken from the shared setup; the measurement is run at version twelve, where every claim is stale. Making noise 17, silent 39.
• DO32 — Every reader visit lands on one claim, and claims are chosen with equal probability. In a real setup, interest is not evenly distributed; equal distribution is the optimistic estimate for coverage.
• DO33 — A reader who hits an embedded stale claim always reports it; the example has failed to run and the malfunction is not in question.
• DO34 — A third of readers who hit a silent stale claim report it; the rest count it as their own mistake, give up, or find their own way.
• DO35 — The same claim being reported more than once is still counted once; what is measured is not visits, it is the count of claims brought into the open.
• DO36 — Reporting only brings a claim into the open; the measurement does not count the fix. The document does not change across visits and no new staleness is added.
• DO37 — Randomness comes from the shared setup’s generator and is called with a single module; the set’s resolution at thirty-nine silent claims is 1/39.

Measurement

"""Feedback loop: silent staleness becoming visible through the reader.

Part 1 - reported silent claims as reader visits climb.
Part 2 - remaining silent staleness per type after two hundred forty visits.
"""
SEED = 20260816
TYPES = {
    "tutorial":    {"flow": 9, "signature": 3, "name": 2, "concept": 1},
    "how-to":      {"flow": 6, "signature": 5, "name": 2, "concept": 1},
    "explanation": {"flow": 1, "signature": 1, "name": 1, "concept": 9},
    "reference":   {"flow": 0, "signature": 13, "name": 2, "concept": 0},
}
FREQUENCY = {"signature": 2, "flow": 3, "name": 5, "concept": 12}
EMBEDDED = {"tutorial": 0.60, "how-to": 0.70, "explanation": 0.10,
            "reference": 0.00}
VERSION = 12
VISITS = (30, 60, 120, 240, 480, 960)
REPORT_SHARE = 3      # a reader hitting a silent stale claim reports it: one in three


def rng(seed):
    d = seed % 2147483646 + 1

    def draw(n):
        nonlocal d
        d = (d * 48271) % 2147483647
        return d % n
    return draw


def claims():
    entries, no = [], 0
    for kind, mix in TYPES.items():
        embedded_target = 0
        for surface, count in mix.items():
            for _ in range(count):
                no += 1
                embedded_target += EMBEDDED[kind]
                embedded = embedded_target >= 1 and surface != "concept"
                if embedded:
                    embedded_target -= 1
                entries.append({"no": no, "type": kind, "surface": surface,
                                 "embedded": embedded})
    return entries


def stale(claim, version):
    return version // FREQUENCY[claim["surface"]] >= 1


def loop(entries, visit_count):
    """Each visit lands on a claim. An embedded stale claim becomes visible when
    the example is run; a silent stale claim becomes visible only if the reader
    reports it."""
    draw = rng(SEED)
    hit, reported = set(), set()
    for _ in range(visit_count):
        claim = entries[draw(len(entries))]
        if not stale(claim, VERSION):
            continue
        hit.add(claim["no"])
        if claim["embedded"] or draw(REPORT_SHARE) == 0:
            reported.add(claim["no"])
    return hit, reported


L = claims()
STALE = [i for i in L if stale(i, VERSION)]
SILENT = [i for i in STALE if not i["embedded"]]
print(f"at version {VERSION}, claims {len(L)}, stale {len(STALE)}, making noise "
      f"{len(STALE) - len(SILENT)}, silent {len(SILENT)} "
      f"({len(SILENT) / len(STALE):.4f})")
print()
print(f"{'visits':>8s} {'hit':>9s} {'silent reported':>18s} "
      f"{'coverage':>8s} {'visits per report':>24s}")
for v in VISITS:
    hit, reported = loop(L, v)
    b = [i for i in SILENT if i["no"] in reported]
    print(f"{v:8d} {len(hit):9d} {len(b):18d} "
          f"{len(b) / len(SILENT):8.4f} {v / len(b):24.1f}")
print()
_, reported = loop(L, 240)
print(f"{'type':>15s} {'silent':>7s} {'reported at 240 visits':>25s} "
      f"{'remaining':>6s}")
for kind in TYPES:
    group = [i for i in SILENT if i["type"] == kind]
    b = [i for i in group if i["no"] in reported]
    print(f"{kind:>15s} {len(group):7d} {len(b):25d} {len(group) - len(b):6d}")

at version 12, claims 56, stale 56, making noise 17, silent 39 (0.6964)

  visits       hit    silent reported coverage        visits per report
      30        20                  6   0.1538                      5.0
      60        34                 10   0.2564                      6.0
     120        47                 19   0.4872                      6.3
     240        54                 31   0.7949                      7.7
     480        56                 36   0.9231                     13.3
     960        56                 39   1.0000                     24.6

           type  silent    reported at 240 visits remaining
       tutorial       7                         6      1
         how-to       5                         3      2
    explanation      12                         9      3
      reference      15                        13      2

The Loop Works, It Works Slowly

The last column closes: coverage 1.0000. Once enough readers have come, all the silent stale claims come into the open. The feedback loop really is the third claim’s antidote; it works independent of documentation audit, without putting anything on anyone’s calendar.

Its cost is in the column on the right. The first thirty visits bring 6 silent claims into the open: 5.0 visits per report. At two hundred forty visits, coverage reaches 0.7949 and the cost climbs to 7.7. Getting the last three claims requires going from four hundred eighty to nine hundred sixty, and visits per report becomes 24.6. Once coverage saturates, cost rises fivefold, because the claims remaining are the ones read the least or the least likely to be reported.

The hit column separates this difference. By the four-hundred-eightieth visit, all fifty-six claims have been hit — meaning the document has been read start to finish. Even so, the number of silent claims reported is 36. The three claims in between were hit but not reported: the reader read them, got it wrong, and did not say so. Being read is not the same as being fixed.

The lower table gives the distribution of two hundred forty visits by type. 13 of reference’s fifteen silent claims and 9 of explanation’s twelve are reported. The remaining eight claims spread across the four types, and no type is fully cleared. This compares directly with the previous lesson’s audit table: the same thirty-nine claims are found by a fully scoped documentation audit reading 56 claims in a single pass; the feedback loop finds them with 960 visits, and every one of those visits is a reader who read a wrong sentence.

The comparison defines the two paths’ roles. Audit is cheap and has to be started; the loop works on its own and the reader pays its cost. The two do not substitute for each other: without the loop, the audit does not know where to look; without the audit, the loop cannot lower the cost it is paying.

Summary

• Silent staleness does not show itself; the only natural way it becomes visible is for the reader who hits it to report it.
• A reader’s report is turned into a documentation gap with three questions: the sentence relied on, the surface it is bound to, the version the surface changed at. A single report also targets the other claims bound to the same surface.
• 31 of thirty-nine silent claims come into the open at 240 visits, 39 at 960; coverage reaches 1.0000, but cost per report climbs from 5.0 visits to 24.6.
• Despite all fifty-six claims having been hit by four hundred eighty visits, the number of silent claims reported is 36; being read is not the same as being fixed.
• A documentation audit finds the same thirty-nine by reading 56 claims; the loop’s cost is paid by the reader who got it wrong. The two do not substitute for each other.

Course Wrap-Up

The course built a single measure and carried it across fifteen lessons: a document’s number is not its length, it is the number of claims still true — and how many of the ones that go stale are silent. Every lesson ran its own measurement; every row of the table below is read from that lesson’s own output block.

lesson	measured	true remaining or silent stale
The Scope of Technical Writing	claim, type, surface, and embeddedness distribution	56/56 true at version 0; embedded 17, silent 39; a signature change drops 22 claims, silent share 0.7727
Tutorial, How-To, Explanation, and Reference	the four types’ lifespan at version two	explanation 11/12 (0.9167), reference 2/15 (0.1333); lifespan ratio 6.875 times
Repository Documents	three repository documents’ surface distribution and half-life version	half-life 3 / 3 / 12; at version 12, decision record 0/10 and 10 silent
API Reference	comparison of hand-written and generated reference	at version 6, hand-written 0/15, generated 13/15; 2 unrecoverable claims on the name surface
Release Notes	number of claims the note format makes datable	raw list 17/56 and 0/39 of the silent ones; meaning list 56/56 and 39/39
Troubleshooting Content	reader decisions symptom-first text meets	symptom–cause–remedy 51/51 (1.0000); covered claims 17/56, unreachable 39
Audience Definition	claims the prerequisite assumption makes readable	true at version 3 is 18; readable climbs from 3 to 18, stale 38 and silent 24 unchanged
Structure and Headings	units read before reaching the reader’s question	average steps from 26.38 to 7.00; at version 6, the two-tier layout carries 24 silently stale headings
Style Guide	ambiguity produced by dual naming	found by searched name in the free regime 25/56; on renaming, missed 31, of which silent 23
Examples and Code Snippets	embeddedness ratio’s effect on silence	at version 12, 0/56 true, silent 39 (0.6964); between 0.8571 and 0.4464 by regime
Use of Visuals	the surface a diagram is bound to and its noise	true/stale unchanged (0/56); silent climbs from 39 to 56 in the mixed scheme
Docs Alongside Code	the effect of bringing documentation into review on staleness	versioned together 300/672 and 39 silent; reviewed 43/672 and 15 silent
Documentation Production Approaches	the surface production saves and cannot save	at version 12, generation from source 22/56 true; 22 of the remaining 34 stale are silent
Staleness and Audit	silent staleness audit scope finds	at version 12, 56/56 stale and 39 silent (0.6964); without audit, only 17 found
Feedback Loop	silent claims reader reporting brings out	31/39 (0.7949) at 240 visits, 39/39 at 960; cost from 5.0 to 24.6

The course following its own rule was not a formality. Every measurement was run, every output block was taken from the run itself; this lesson’s own embedded claims too will make noise when they go stale.

This course is the last one in the Software Development Practice curriculum. Five courses measured five separate things, and all of them asked the same question of a different object: what does the work done leave behind?

course	measure axis
Introduction to Version Control	which region a record stands in and its recoverability from there
Branching and Collaboration	the question history can answer
Advanced Git	the number of objects an operation touches
Code Review and Team Process	the axis looked at and the chunk read
Technical Writing and Documentation	how many claims are still true, how many are silently stale

The first row is read from that course’s own lessons: the distinction between working directory, staging area, and object database; the criterion for recoverability being that the content was written to the object database; reset commits staying in the database; and unreachable objects not being deleted — all of it builds a single axis.

The five courses together built a single team discipline: recording a change, integrating it among more than one person, managing history, having it read by someone else, and narrating it. What all of them share is being language-independent; what was counted was never the written code itself, it was the work around it. The curricula that follow turn from this periphery to the center and measure the language itself: the structures a language offers, the decisions those structures force, and the trace those decisions leave in the program.