Documentation

The previous three lessons measured three separate places that sit in the source but that the compiler never looks at: a closed flag’s body, a property tested by running, two implementations’ operation count. In all three, what was measured was a part of the source that was code — a function, a block, a loop. This is the fourth and final lesson of this topic, Project Lifecycle, and it asks the same question of the part of the source most often overlooked: the explanation written next to a function. A documentation sentence is also part of the source, but it is not syntax the compiler checks, it is prose a human reads; this lesson asks who (or whether no one) checks this prose’s correctness — and shows that two different answers can stand side by side in the same source.

This question’s answer cannot be given in a single sentence, because a documentation comment has two different sections: the prose read above, and a small code example, usually placed below it, showing how to call the function. These two sections sit together inside the same comment, but their fates are not the same — that is exactly what this lesson measures. This topic’s other three lessons each built their own measurement; this lesson returns to the course’s twelve-observation guarantee core and uses its documentation-accuracy observation directly.

The Documentation Itself, and the Contradiction in the Source

In Rust, writing lines starting with three slashes (///) above a function forms that function’s documentation comment. This comment differs from an ordinary explanation: subheadings like # Examples and code fragments wrapped in backticks can be placed inside it. All three functions below carry this same layout — an explanatory sentence, and an example below it. The first function is the guarantee core’s (guarantee.rs) documentation-accuracy section: the documentation line says the function returns the largest value, while the function actually returns the smallest. The other two functions — one returning the smaller value, one summing a slice — have correctly written documentation, and were added here for comparison.

// doc_body.rs — guarantee.rs's documentation-accuracy section + two more tested examples
/// This documentation line says the function returns the **largest** value.
/// The function returns the smallest. No tool checks this contradiction.
///
/// # Examples
///
/// ```
/// assert_eq!(doc_body::wrong_doc(3, 9), 3);
/// ```
pub fn wrong_doc(a: i32, b: i32) -> i32 {
    if a < b {
        a
    } else {
        b
    }
}

/// Returns the smaller of two values.
///
/// # Examples
///
/// ```
/// assert_eq!(doc_body::smaller_of(3, 9), 3);
/// ```
pub fn smaller_of(a: i32, b: i32) -> i32 {
    if a < b {
        a
    } else {
        b
    }
}

/// Returns the sum of a slice.
///
/// # Examples
///
/// ```
/// assert_eq!(doc_body::sum_slice(&[1, 2, 3]), 6);
/// ```
pub fn sum_slice(slice: &[i32]) -> i32 {
    slice.iter().sum()
}

PL13. This file carries no fn main at all; it stands as a source section. All three functions compile correctly — even though wrong_doc’s prose is wrong, its body is type-consistent, and the compiler never reads the prose at all. The body below calls these three functions.

// doc_body_main.rs — the documentation contradiction stays in the source, the program still compiles
mod doc_body;
use doc_body::{wrong_doc, smaller_of, sum_slice};

fn main() {
    println!("doc says 'largest', real result: {}", wrong_doc(3, 9));
    println!("smaller_of(3, 9) = {}", smaller_of(3, 9));
    println!("sum_slice(&[1, 2, 3]) = {}", sum_slice(&[1, 2, 3]));
}

doc says 'largest', real result: 3
smaller_of(3, 9) = 3
sum_slice(&[1, 2, 3]) = 6

Compilation printed no warning at all. The wrong_doc(3, 9) call returned 3 — the smaller value, not the larger one the documentation sentence claims. This is expected behavior in an ordinary compilation process: rustc does not parse a documentation comment starting with three slashes (///) as Markdown, does not read the sentences inside it, does not compile the code example embedded within it. The entire documentation comment — both the wrong prose and the correctly working example — is invisible text in an ordinary compilation; the compiler neither confirms nor rejects it, because it never looks at it at all.

This is exactly the class the guarantee core measures: nobody checks it. The compiler checked the function’s body (checked its type, resolved its names) but never saw the function’s documentation at all. Even though a documentation sentence is part of the source, it sits entirely outside the compiler’s sphere of interest — just like the closed flag’s body in the Dependencies and Feature Flags lesson, but for a different reason: there, the body never entered compilation at all; here, it is not that it fails to enter, but that the compiler sees it as belonging to another category entirely — not code, prose.

It matters to keep these two situations apart. A closed flag’s body is a piece of code that could enter compilation once the flag is turned on — it is potentially checkable, merely unchecked today. A documentation sentence’s prose part, on the other hand, never enters compilation under any condition; it was never code, and never will be. The compiler separates a /// documentation comment from the rest of the source and stores it as information attached to the function — so it is not entirely ignored, it sits somewhere a documentation tool can read it. But an ordinary compilation’s type-checking and name-resolution stages never look at this information’s content at all: they do not verify the sentence inside it, they do not compile the code example inside it. What is checked and what is merely stored are different things.

This comes from the same root as the rot measured in the Dependencies and Feature Flags lesson. There, a closed flag’s body could lose sync as the rest of the source changed, because nobody checked it. Here, the same thing happens between a function’s body and its documentation: the body can change over time — a bug gets fixed, a behavior gets reversed — but the documentation sentence above it does not update itself. wrong_doc was probably written correctly at some point (perhaps the function really did return the largest value), and the documentation sentence was forgotten when the body changed; there is no way to prove this, but the fact that both sides can live independently explains why such drift is possible at all. What the two share is that no checking mechanism keeps the two sides tied to each other.

The Example Inside the Documentation: Testing from Within a Program

Each function’s documentation above has a # Examples section, containing a code example wrapped in three backticks. Even though these examples are invisible in ordinary compilation, the language’s own documentation tool handles them specially: it scans the source and compiles and runs each example as a separate test. This is the second form of measurement built from within a program in this curriculum — in the Procedural Macros lesson, the compiler itself was invoked from within a program; here, what is invoked is not the compiler, but one of the compiler’s side tools. The program below invokes this tool from within itself, using std::process::Command, and prints only how many examples passed — no path, duration, or machine name is printed.

// doc_test.rs — invokes the documentation tool from within the program and prints the passing test count
use std::process::Command;

fn main() {
    let build = Command::new("rustc")
        .args([
            "--edition", "2021", "--crate-type", "lib",
            "-o", "libdoc_body.rlib", "doc_body.rs",
        ])
        .output()
        .expect("could not start the compiler");
    if !build.status.success() {
        println!("passing tests: 0");
        return;
    }

    let test_run = Command::new("rustdoc")
        .args(["--edition", "2021", "--test", "-L", ".", "doc_body.rs"])
        .output()
        .expect("could not start the documentation tool");

    let output = String::from_utf8_lossy(&test_run.stdout);
    let count = output
        .lines()
        .find(|s| s.starts_with("test result:"))
        .and_then(|line| line.split("passed").next())
        .and_then(|part| part.trim().rsplit(' ').next())
        .and_then(|s| s.parse::<u32>().ok())
        .unwrap_or(0);

    println!("passing tests: {}", count);
}

passing tests: 3

PL14. All three examples passed — wrong_doc’s, smaller_of‘s, and sum_slice‘s. This program runs in two steps. In the first step, it compiles the source as a library; this step is necessary because every example calls the real functions in the source by their full names, like doc_body::wrong_doc, and the documentation tool needs to find the pre-compiled library to resolve these names. In the second step, the documentation tool is invoked in test mode; the -L . option tells the tool to look for the pre-compiled library in the working directory. The tool’s own output contains, line by line, a path, a duration, and a detailed breakdown; this program extracts only the single number from the “passed” line out of that output and never prints the rest — no field dependent on the environment leaks into this program’s own output.

This run reverses the previous section’s observation. In ordinary compilation, the documentation was completely invisible; here, the documentation tool reads the same source with a different eye — it treats the code fragment inside each # Examples block as a separate source file and genuinely compiles and runs it. wrong_doc‘s example tests real behavior (the result 3) with the equality-checking call inside it, and passes — because the example code itself is written correctly, only the prose sentence above it is wrong. The class measured in this lesson changes here: still holds. The documentation tool is part of the language’s own distribution, and, just like rustc checks a body, it checks the example by compiling and running it.

This provides a defense against the rot described in the previous section — but only for the example, not for the prose. Had smaller_of’s signature changed (say, if it took three arguments instead of two), the call inside its example would no longer compile, and the documentation tool would mark it “failed”; the example is forced to stay in sync with the body. Nothing, however, keeps wrong_doc’s prose in sync with the body — whether the body changes or not, the sentence stays exactly as it is. The guarantee the documentation tool gives is therefore real but narrow: only the example is forced to stay in sync with the body, not the explanation surrounding it.

Distinguishing this from a related idea: the Production Go: Testing, Tooling and Performance course’s measurement axis was who states a piece of knowledge — the source, a tool, or a run. A documentation sentence is exactly a piece of knowledge the source states; this lesson asks whether that knowledge is verified. The answer splits in two: the prose part is read by no tool at all (nobody checks it), while the example part is run by the language’s own documentation tool (still holds) — the same source sentence’s two different parts meet two different fates. Compared to that course’s three-way distinction, a documentation sentence’s prose falls exactly into the “stated by the source” category — no tool or run verifies it; the documentation sentence’s example, however, steps outside that three-way split, because it stops being something the source merely states and becomes something a run verifies.

A Bounding Measurement

Reading the two runs above together produces a striking result: wrong_doc’s example is one of the three passing tests, and its passing does not change the fact that the prose sentence above it is still wrong. The example tests 3, and 3 is the correct result; the prose says “largest,” and that is wrong. Because the documentation tool runs the example and gets the correct result, it marks the test as passed — without ever looking at what the prose says.

PL15. This is this measurement’s most important limit: what the documentation tool checks is the example, not the explanation. An example tests the claims inside it (the expected values written with the equality-checking call); the prose sentence above the example neither influences nor checks what it says, in any way. A person could read wrong_doc‘s documentation, understand “returns the largest,” see the example below the same documentation, and even know that it passes — and still walk away from the source with a wrong impression, because a passing example says nothing at all about the prose’s correctness.

This limit’s scope is also clear-cut: the documentation tool sees only code blocks marked with ```; every sentence outside these blocks, including the # Examples heading, is ordinary text to it. An example being written correctly does not mean that example is consistent with the explanation — it only means the example is correct on its own terms (that it matches the real behavior in the source code). In this lesson, the smaller_of function has exactly the same body as wrong_doc, but its prose is correct; both of their examples pass the same kind of test, because the example only tests what gets called and what gets returned at the code level, it does not compare the two functions’ prose.

This limit has one more consequence: an example passing does not mean the case that example represents is complete, either. wrong_doc‘s example tests only a single call (3 with 9); what the function returns when called with a different input never appears in this example at all. While the documentation tool marks this example as passed, it says nothing about what the function does for every input — exactly the same limit seen in the Testing and Property-Based Testing lesson applies here too: exactly as much guarantee as the tested input, no more, no less. The guarantee the documentation tool gives is therefore doubly bounded: it tests only the example (not the prose), and it tests only the input the example tests (not the function’s entire behavior).

Summary

• In an ordinary compilation (rustc), both a documentation comment’s prose and the code example inside it are invisible; the compiler looks at neither.
• wrong_doc’s documentation sentence says “largest,” the body returns the “smallest”; this contradiction passes without warning in ordinary compilation — the class is nobody checks it.
• The documentation tool was invoked from within the program, and only the passing test count (3) was printed; duration, path, and machine name were never printed.
• The documentation tool genuinely compiles and runs the code inside # Examples; for this section, the class is still holds — the language’s own tool carries rustc’s checking one step further.
• The bounding measurement: wrong_doc’s example passed, but the wrong prose sentence above it is still wrong — the documentation tool tests the example, not the explanation.
• The same function can carry two guarantee classes at once: its documentation’s prose is nobody-checks-it, its documentation’s example is still-holds.

Next Step

This topic’s four lessons, Project Lifecycle, showed how the compiler’s guarantee stops in four separate places: in a closed flag’s body (nobody checks it), in a test established by running (handed to the writer), in a counted unit of operation (nobody checks it), and now in a documentation sentence — where, for the first time, two classes (nobody-checks-it and still-holds) appear together within a single lesson. All four are different views of the same underlying fact: wherever the compiler stops, the remaining gap is either filled by nobody, filled by the writer through running, or entered by one of the language’s own tools with a different kind of checking.

The course’s final lesson — the course wrap-up — brings these four places together and turns to look at a fifth, this one outside a source: a version string. That string, too, sits in the source and corresponds to nothing; which version of a library preserves which signature is the Publishing lesson’s subject. This lesson leaves its mark there, too: a version string, just like a documentation sentence, is a claim that sits in the source, that a human reads, but that the compiler never checks.