Examples and Code Snippets

The previous lesson surfaced a distinction. When a term was renamed, the scan missed 31 claims; 8 of those made noise, 23 did not. The noisy ones shared exactly one trait: they stood inside a runnable example.

This lesson’s question is naming that trait and measuring its cost: how many of a document’s claims are actually embedded, how far does silent staleness retreat if the embeddedness ratio changes, and where does it stop retreating?

An Embedded Claim and the Sentence Beside It

Around an example, two separate kinds of sentence exist, and they behave completely differently in the face of staleness.

The first is the example itself. An example claims a call’s shape, a field’s name, a step’s order; these claims are not written as sentences, they are written as code. When the product changes, the example stops working, and this is an event: a test fails, a build step breaks, the reader gets an error message. An embedded claim is a claim whose correctness can be tested by running it.

The second is the sentence written beside the example: “this call takes the threshold field as a percentage.” It carries the same information, is bound to the same thing, but is tested nowhere. When the product converts the threshold field to seconds, the example still runs — the number is still valid, only its meaning changed — and the sentence goes wrong silently.

The distinction, then, is not measured by “does an example exist.” It is measured by whether the same claim stands inside the example or beside it. Most sentences standing beside can be moved into the example: a field’s unit can be claimed with a validation line, a returned field’s existence with a print, the importance of order with the steps actually running in order.

Runnable, Complete, Verifiable

Embeddedness is not an intention, it is a property, and it requires all three conditions at once. The three are defined separately because they fail separately.

Runnable: the example runs when copied and executed. A missing import line, an undefined variable, or an option left inside a comment makes the example fail to run.

Complete: the example expects no definition from outside itself. A snippet that uses a variable set up in a previous example fails when copied alone; and the reader most often copies alone.

Verifiable: the example’s expected output is written beside it and can be compared. An example with no written output runs, but running wrong goes unnoticed; making noise requires it to fail, while most staleness arrives not as a failure but as a different result.

# example document excerpt, not executed

example written as a snippet
  result = measurement.start(threshold = 5)
  print(result.summary)
  Sentence in the text: "the start call takes the threshold field as
  a percentage."

lines needed to turn it into a complete example
  - where `measurement` comes from
  - the context that must be set up before `start` can be called
  - printing that the `summary` field actually comes back
  - the expected output standing beside the example

The snippet above meets none of the three conditions. It does not run, because the first name is undefined. It is not complete, because its context sits elsewhere. It is not verifiable, because what it should print is not written. A snippet like this looks embedded but is not embedded: it makes no noise when it goes stale, because it was not running in the first place.

This Lesson’s Own Example

If this lesson says “runnable, complete, and verifiable example,” its own example has to be exactly that — otherwise the lesson falls into the same state as the snippet above.

The measurement block below meets all three conditions. It expects no outside definition: the types, surfaces, frequencies, and embeddedness ratios are all inside the block. It runs when copied and executed. The unlabeled block that follows is this block’s actually captured output; it was not written by hand and can be compared. Every number claim in this lesson comes from that output; none comes from the prose.

The measurement’s assumptions:

• WD31 — The 56 claims come from the shared fixture: four types, four surfaces. The oracle is the fixture itself; we know which claim is bound to which surface and at which version it goes stale, because we wrote it.
• WD32 — A claim is stale if the surface it is bound to has changed at least once by that version. Surface frequencies do not change: signature once every 2 versions, flow every 3, name every 5, concept every 12.
• WD33 — An embedded claim makes noise when it goes stale; a non-embedded one goes stale with nothing happening, and is counted silent.
• WD34 — A claim bound to the concept surface can never be embedded in an example. An example cannot claim, by running, why something is the way it is; it can only claim how it behaves.
• WD35 — The only thing this lesson changes is the embeddedness ratio. The types’ surface mix and the surfaces’ frequencies stay as in the shared fixture.
• WD36 — Three embeddedness regimes are tried: snippet (ratios halved; an incompletely written example does not count as embedded), shared fixture (the fixture’s own ratios), full example (every type writes its examples completely, and reference carries examples too).
• WD37 — The silence ratio is computed over the stale claims; at versions where no claim is stale, the ratio is undefined.

Measurement

"""Examples and code snippets: an embedded claim makes noise, a non-embedded one does not.

Part 1 - claims embedded, by type.
Part 2 - version by version, true/stale/silent and the silent-to-stale ratio.
Part 3 - the one thing this lesson changes: the embeddedness ratio.
"""
SEED = 20260816
TYPES = {
    "tutorial":      {"flow": 9, "signature": 3, "name": 2, "concept": 1},
    "how-to":        {"flow": 6, "signature": 5, "name": 2, "concept": 1},
    "explanation":   {"flow": 1, "signature": 1, "name": 1, "concept": 9},
    "reference":     {"flow": 0, "signature": 13, "name": 2, "concept": 0},
}
FREQUENCY = {"signature": 2, "flow": 3, "name": 5, "concept": 12}
EMBEDDED = {"tutorial": 0.60, "how-to": 0.70, "explanation": 0.10, "reference": 0.00}
VERSIONS = (0, 1, 2, 3, 6, 12)
# This lesson's one change: the embeddedness ratio. Surfaces and types stay fixed.
REGIME = {
    "snippet": {t: o / 2 for t, o in EMBEDDED.items()},
    "shared fixture": EMBEDDED,
    "full example": {"tutorial": 0.80, "how-to": 0.85,
                      "explanation": 0.40, "reference": 0.60},
}


def claims(ratios=EMBEDDED):
    """Every claim: its type, the surface it is bound to, whether it is embedded."""
    entries, no = [], 0
    for kind, mix in TYPES.items():
        embedded_target = 0
        for surface, count in mix.items():
            for _ in range(count):
                no += 1
                embedded_target += ratios[kind]
                embedded = embedded_target >= 1 and surface != "concept"
                if embedded:
                    embedded_target -= 1
                entries.append({"no": no, "type": kind, "surface": surface,
                                 "embedded": embedded})
    return entries


def stale(i, version):
    return version // FREQUENCY[i["surface"]] >= 1


def measure(entries, version):
    """True claims, stale claims, and the silent share of the stale ones."""
    stale_ = [i for i in entries if stale(i, version)]
    silent = [i for i in stale_ if not i["embedded"]]
    return len(entries) - len(stale_), len(stale_), len(silent)


A = claims()
print(f"claims {len(A)}, types {len(TYPES)}, "
      f"embedded in example {sum(1 for i in A if i['embedded'])}")
print("embedded distribution: " + ", ".join(
    f"{t} {sum(i['embedded'] for i in A if i['type'] == t)}" for t in TYPES))
print()
print(f"{'version':>7s} {'true':>6s} {'stale':>6s} {'silent':>7s} {'silent/stale':>13s}")
for s in VERSIONS:
    true_, stale_, silent = measure(A, s)
    ratio = f"{silent / stale_:.4f}" if stale_ else "-"
    print(f"{s:7d} {true_:6d} {stale_:6d} {silent:7d} {ratio:>13s}")
print()
print(f"{'embeddedness regime':>19s} {'embedded':>9s} {'v12 stale':>10s}"
      f" {'silent':>7s} {'silent/stale':>13s}")
for name, ratios in REGIME.items():
    entries = claims(ratios)
    true_, stale_, silent = measure(entries, 12)
    print(f"{name:>19s} {sum(1 for i in entries if i['embedded']):9d} {stale_:10d}"
          f" {silent:7d} {silent / stale_:13.4f}")
print()
print(f"{'type':>15s} {'embedded':>9s} {'v3 silent':>10s} {'v12 silent':>11s}")
for t in TYPES:
    group = [i for i in A if i["type"] == t]
    print(f"{t:>15s} {sum(i['embedded'] for i in group):9d}"
          f" {measure(group, 3)[2]:10d} {measure(group, 12)[2]:11d}")

claims 56, types 4, embedded in example 17
embedded distribution: tutorial 8, how-to 9, explanation 0, reference 0

version   true  stale  silent  silent/stale
      0     56      0       0             -
      1     56      0       0             -
      2     34     22      17        0.7727
      3     18     38      24        0.6316
      6     11     45      28        0.6222
     12      0     56      39        0.6964

embeddedness regime  embedded  v12 stale  silent  silent/stale
            snippet         8         56      48        0.8571
     shared fixture        17         56      39        0.6964
       full example        31         56      25        0.4464

           type  embedded  v3 silent  v12 silent
       tutorial         8          5           7
         how-to         9          4           5
    explanation         0          2          12
      reference         0         13          15

Most Staleness Is Silent

The first table pays off the course’s third claim. At the twelfth version, 56 of 56 claims are stale, and 39 of those are silent: 0.6964. Two-thirds of the document is wrong, and it is telling no one that it is wrong.

The number is not only large at the last row. Of the 22 claims stale at the second version, 17 are silent — the ratio is at its highest point, 0.7727. This is because the first surface to go stale is signature, and most signature claims sit in reference; reference has no embedded claims. The part of the document that goes stale earliest is also its quietest part.

The ratio drops to 0.6316 and 0.6222 at the third and sixth versions. This drop is not an improvement: the flow surface enters, most flow claims sit in tutorial and how-to, and those two types carry embedded examples. So the claims that go stale in this range are the ones that make noise. At the twelfth version, once the concept surface also falls, the ratio rises again, because concept claims can never be embedded.

The practical meaning of this ratio is an illusion. A team tracking its document’s health by “what broke” gets 17 reports at the twelfth version and fixes all of them. Once done, they believe the document is error-free. The real number is 56 stale claims; the unseen 39 stand right where they were. Fixing noisy staleness does not make silent staleness visible; it produces evidence that makes the document look healthy instead.

The smallest measurable difference in this set is 1/56 = 0.0179. The differences between regimes are many times above this and comparable; the gap between 0.6316 and 0.6222 is smaller than a single claim and should not be read as a trend.

Raising Embeddedness

The second table shows the one thing this lesson changes. Same 56 claims, same surfaces, same frequencies; the only thing that changes is how many claims are embedded in a runnable example.

In the snippet regime, embedded claims drop to 8 and silence rises to 0.8571. This regime does not describe a document without examples; it describes one with examples that are not complete. A half-written example sits on the page, shows the reader something, and says nothing when it goes stale.

In the full example regime, embedded claims rise to 31, silence falls to 0.4464. Silent claims drop from 39 to 25: 14 claims stop being silent and cross into the noise-making class. Reference carrying runnable examples too produces most of this gap.

The table also shows a floor: silence does not zero out. Even at the highest ratios, 25 claims stay silent, because claims bound to the concept surface can never be embedded in an example. An example can claim, by running, how something behaves; it cannot claim why it is that way. Embeddedness is a maintenance tool, and it has a region it cannot reach; for that region, there is no path other than a document audit.

A Copy Is Not Embedded

The table does not ask one thing: what exactly makes a claim embedded? A code block on a page is, on its own, just text; no text reports its own staleness. What makes noise is that same text actually being run somewhere.

There are two ways to do this. The first is the document’s own example being run — like this lesson’s measurement block. The second is the example being pulled from a source that is run; the document does not host the code, it displays it.

There is a third case, and it is the most deceptive one: the copy. The team keeps examples somewhere that is tested, and pastes a copy of them into the document too. The source is tested, stays true, and everyone assumes the examples are tested. But what is tested is not the copy; when the product changes, the source gets fixed, the document does not, and the two texts drift apart silently. A copied example does not count as embedded in the measurement; it moves the document straight into the snippet column above and raises silence to 0.8571. A copy’s cost exceeds the cost of having no example at all: a document with no example misleads no one; a document with a copy misleads both the reader and the writer.

This standard’s second consequence is the example’s size. Completeness stretches an example longer; the scannability the previous lesson measured punishes length — the reader pays for every line as a step. The two conditions genuinely conflict, and the answer is not somewhere in the middle. The right setting is the smallest complete example: not the whole of real usage, but the shortest runnable whole that carries the claim. Every line cut from an example takes one claim out of being embedded; so the cutting decision is not aesthetic, it is measurable — which claim’s noise is being given up?

Versioning an example alongside the source and putting it through review is not this topic’s job; the course’s last topic builds documentation operations separately. What is measured here is only this: a claim makes noise only if the text carrying it is actually run.

The Type Distribution of Silence

The last table compares the four types by silence and reveals a reversal. Explanation is the longest-living type — at the third version it has only 2 silent claims. At the twelfth version, 12 of its 12 claims are silently stale: the whole type goes wrong without producing a single warning.

Reference does the opposite: it already carries 13 silent claims at the third version, rising to 15 at the twelfth. It goes stale fast and is silent start to finish. Tutorial and how-to are in the best shape, with 7 and 5 silent claims respectively; together the two carry all 17 embedded claims.

The result sets writing priority. The two types carrying the most examples are already the ones making the most noise; adding one more example there pays off little. The gain is in the types with no examples at all: reference’s 15 claims and explanation’s 12 are quieter than the rest of the product combined.

Summary

• An embedded claim stands inside the example and is tested by running it; a sentence written beside the example carries the same information but is untested and goes wrong silently.
• An example needs all three conditions at once: runnable, complete, verifiable. If any one fails, the example looks embedded but is not. This lesson’s own measurement block meets all three, and its output was actually captured.
• At the twelfth version, 56 of 56 claims are stale, 39 silent: 0.6964. The highest silence is at the second version (0.7727), because the first surface to go stale is signature, and most signature claims sit in example-free reference.
• As the embeddedness ratio changes, silence moves between 0.8571 and 0.4464; a half-written example drops embeddedness from 17 to 8, a complete example raises it to 31.
• Silence cannot be zeroed out: claims bound to the concept surface can never be embedded. The two quietest types are explanation (12/12) and reference (15); that is where adding an example pays off.

Next Step

The example is the one element in a document that can test itself. There is a second element standing beside the text that can never be run under any condition: the diagram. A diagram carries claims too — a component’s name, a flow’s order, a concept’s structure — and the claims it carries are bound to a surface just like sentences are. The next lesson measures which surface a diagram is bound to, how that choice sets its rate of staleness, and why visual staleness is silent without exception.