Go Command Family

The previous three lessons called three members of the language’s own command family: the compiler decided whether a source was valid, the formatter canonicalized its appearance, vet caught a pattern the compiler overlooked. All three were concerned with the source itself. This final lesson looks at the family’s fourth member: the doc tool that produces the text describing what a package offers to the outside. This tool neither compiles the source, nor formats it, nor tests it against a pattern — it only reads, and decides which name, with which comment, to show and when. The course’s final question: is it the comment itself that makes this decision, or the first letter of the name?

Across the course’s first three topics, only one member of this command family was called: the compiler. Its job was narrow — deciding whether the source turns into an executable program. But beyond whether a source text compiles or not, two more questions stand: how does this source look, and which conventions does this source follow? The compiler is indifferent to both — a source produces the same program no matter what its indentation, spacing, or line breaks are, and the language’s syntax rules never decide at all whether a convention is “good” or “bad”; they only look at whether the convention is valid. The answer to these two questions therefore sits outside the compiler, in two separate members of the same command family.

Running a Source Through the Formatter

PT12. A source’s form — indentation, spacing, alignment — is not enforced by the language’s syntax rules at all; a source carrying the same meaning can be written in many different forms. PT13. Even so, the formatter turns any valid source given to it into a single canonical form, and this transformation never changes the source’s meaning at all.

The routines below use the exact same compiler-calling pattern built in the previous lessons: the source is written to a temporary module, the command is called, the result is read. The difference is here — this time the command called does not compile the source, it changes the source itself. So the measurement, too, reads something different: in the previous lessons there was a single question (did it compile or not); here two sources — before and after formatting — are held side by side and compared. Every measurement again runs against its own completely empty temporary module and is called with the GOTOOLCHAIN=local environment variable, exactly as in the previous two lessons — so that the command called neither sends a network request nor sees a file left behind by a previous trial.

// tools/tools.go — shared routines that call the go command family from within the program
package tools

import (
	"os"
	"os/exec"
	"path/filepath"
	"strings"
)

func newModule(dir string) {
	os.MkdirAll(dir, 0o755)
	os.WriteFile(filepath.Join(dir, "go.mod"), []byte("module trial\n\ngo 1.24\n"), 0o644)
}

func environment() []string {
	return append(os.Environ(), "GOTOOLCHAIN=local")
}

// Format runs source through the formatter in a temporary module; it returns the
// source before and after formatting, as well as how many lines changed.
func Format(source string) (before, after string, changedLines int) {
	dir, _ := os.MkdirTemp("", "trial")
	defer os.RemoveAll(dir)
	newModule(dir)
	path := filepath.Join(dir, "main.go")
	os.WriteFile(path, []byte(source), 0o644)

	cmd := exec.Command("go", "fmt", "./...")
	cmd.Dir = dir
	cmd.Env = environment()
	cmd.Run()

	data, _ := os.ReadFile(path)
	before = source
	after = string(data)

	beforeLines := strings.Split(strings.TrimRight(before, "\n"), "\n")
	afterLines := strings.Split(strings.TrimRight(after, "\n"), "\n")
	if len(beforeLines) == len(afterLines) {
		for i := range beforeLines {
			if beforeLines[i] != afterLines[i] {
				changedLines++
			}
		}
	}
	return before, after, changedLines
}

// RunAndCapture runs source in a temporary module and returns its standard output.
func RunAndCapture(source string) string {
	dir, _ := os.MkdirTemp("", "trial")
	defer os.RemoveAll(dir)
	newModule(dir)
	os.WriteFile(filepath.Join(dir, "main.go"), []byte(source), 0o644)

	cmd := exec.Command("go", "run", ".")
	cmd.Dir = dir
	cmd.Env = environment()
	output, _ := cmd.Output()
	return strings.TrimRight(string(output), "\n")
}

// VetCatches compiles source in a temporary module and runs it through vet; it is
// used to catch patterns the compiler lets through but vet rejects.
func VetCatches(source string) (compiled, vetCaught bool) {
	dir, _ := os.MkdirTemp("", "trial")
	defer os.RemoveAll(dir)
	newModule(dir)
	os.WriteFile(filepath.Join(dir, "main.go"), []byte(source), 0o644)

	build := exec.Command("go", "build", "-o", os.DevNull, "./...")
	build.Dir = dir
	build.Env = environment()
	compiled = build.Run() == nil

	vet := exec.Command("go", "vet", "./...")
	vet.Dir = dir
	vet.Env = environment()
	vetCaught = vet.Run() != nil
	return compiled, vetCaught
}

Format writes the source to a file and calls the formatter, then reads the same file again — because formatting changes the source in place, before and after are two separate readings. If the line counts are equal (the formatter neither added nor removed a line), it compares the corresponding lines one by one and counts how many changed. VetCatches, on the other hand, calls two separate commands one after another: first the compiler, then vet — only the success/failure state of each is read, no output text is stored or printed at all.

One Source, One Convention

The measurement below gathers two separate questions into a single run: when the same untidy source is run through the formatter, how much of it changes, and when a separate source is run through vet, is a pattern the compiler overlooked caught? The two measurements are independent of each other — one tests the appearance of one source, the other tests the validity of a different source — but both are carried out with the same procedure, calling the command from within the program and only reading the result.

// main.go — the formatter changes the source, vet catches a pattern the compiler does not reject
package main

import (
	"fmt"
	"strings"

	"lesson/tools"
)

func main() {
	source := "package main\n\n" +
		"import \"fmt\"\n\n" +
		"type Point struct {\n" +
		"  X int\n" +
		"  Y int\n" +
		"  Label string\n" +
		"}\n\n" +
		"func Add(a int,b int) int {\n" +
		"      return a+b\n" +
		"}\n\n" +
		"func main() {\n" +
		"    p := Point{X: 1, Y: 2, Label: \"a\"}\n" +
		"    fmt.Println(p,Add(2,3))\n" +
		"}\n"

	before, after, changedLines := tools.Format(source)
	totalLines := strings.Count(strings.TrimRight(before, "\n"), "\n") + 1

	fmt.Println("-- does the formatter change the source --")
	fmt.Printf("identical byte for byte: %v\n", before == after)
	fmt.Printf("total lines: %d, changed lines: %d\n", totalLines, changedLines)

	fmt.Println()
	fmt.Println("-- limiting measurement: the form changes, the meaning does not --")
	outputBefore := tools.RunAndCapture(before)
	outputAfter := tools.RunAndCapture(after)
	fmt.Printf("output of the source before formatting: %s\n", outputBefore)
	fmt.Printf("output of the source after formatting: %s\n", outputAfter)
	fmt.Printf("are the two outputs the same: %v\n", outputBefore == outputAfter)

	fmt.Println()
	fmt.Println("-- vet catches a pattern the compiler does not reject --")
	patternSource := "package main\n\n" +
		"import \"fmt\"\n\n" +
		"func main() {\n" +
		"\tfmt.Printf(\"%d\\n\", \"text\")\n" +
		"}\n"
	compiled, caught := tools.VetCatches(patternSource)
	fmt.Printf("compiled: %v\n", compiled)
	fmt.Printf("did vet catch it: %v\n", caught)
}

-- does the formatter change the source --
identical byte for byte: false
total lines: 18, changed lines: 7

-- limiting measurement: the form changes, the meaning does not --
output of the source before formatting: {1 2 a} 5
output of the source after formatting: {1 2 a} 5
are the two outputs the same: true

-- vet catches a pattern the compiler does not reject --
compiled: true
did vet catch it: true

PT14. 7 of the 18-line source’s lines change after passing through the formatter; the remaining 11 lines were already in canonical form and are never touched at all. PT15. In six of the seven changed lines, the indentation character turns from a space into a tab; in two of these, extra spacing is also added so that the struct fields’ types line up in the same column, in two the space after a comma is completed — one in a function definition’s parameter list, one in a call — and in one, a space enters on both sides of an addition operator — these categories overlap on some lines (an indentation fix and an alignment can occur together on the same line), but none of them adds a new line or deletes a line.

What is unwritten here is a preference: the person writing the source did not write whether to use spaces or tabs for indentation, whether to align a struct’s fields, or whether to leave a space after a comma. The formatter fills these gaps with its own canonical decision — and this is another appearance of the course’s “defined” class from the zero-value discussion: something unwritten is filled not by the compiler’s rejection, but by a separate tool’s own rule.

This decision standing at a single canonical form produces a separate consequence in situations where more than one person works on the same source. When indentation and alignment stay a matter of preference, two sources carrying the same meaning can diverge from each other by appearance alone — and this difference, when compared against the source’s previous state (when looking at what changed line by line), shows up as noise: which line changed in meaning and which changed only in form get mixed together. The formatter’s canonical decision cuts this noise at the source — no matter which indentation the writer chooses, the source always turns into the same form once it passes through the formatter, and this is why the real difference between two sources can only be seen by comparing the two formatted states.

Vet Catches a Pattern the Compiler Lets Through

PT16. The call fmt.Printf("%d\n", "text") compiles — because Printf‘s variadic arguments are accepted as type interface{}, passing a string as the counterpart of a %d verb is syntactically valid in the compiler’s eyes. PT17. The same source is rejected when run through vet: %d expects an integer while the argument is a string, and this mismatch between the format verb and the value’s type is not a rule the compiler checks.

This distinction expands the course’s “rejected” class. In the previous topics the rejecting party was always the compiler; here the rejecting party is vet — a separate command, a separate pass, a separate rule set. The compiler only looks at whether the source turns into a program the virtual machine can run; the agreement between Printf‘s format string and its arguments falls outside this question, because in the compiler’s eyes the format string is an ordinary string value with no special meaning at all. Vet parses this string and matches every verb against its corresponding argument, catching the ones that do not match — it works at a layer the compiler never looks at all.

The value of this catch is in changing where the error surfaces. A call given a string in place of a %d compiles and runs — but when it runs, instead of the expected number it prints the format verb itself along with an error marker, and this is only visible once that call line actually runs. If such a call in a codebase falls onto a rarely run branch, the defect can go unnoticed for a long time; and because compilation raises no warning at all, someone reading the source cannot infer this from the source itself. Vet catches this call without ever running it at all, purely by reading the source — a tool that moves a defect that would otherwise fall into the course’s “left open” class, without ever leaving it to run time, into the “rejected” class instead. The compiler itself does not perform this move, because the compiler’s job is limited to checking syntax and type agreement; the content of a format string falls outside the compiler’s type system, it is a string value interpreted only at run time.

The Limiting Measurement: The Form Changes, the Meaning Does Not

PT18. The same source’s state before and after formatting, when each is run separately, gives exactly the same output: {1 2 a} 5. The byte-for-byte comparison says false, while the run-output comparison says true — these two results do not contradict each other, on the contrary they build the lesson’s thesis: the formatter changes the source’s appearance, not its behavior.

The reason for this lies in what the formatter does: indentation, spacing, and alignment — everything the formatter touches — are whitespace that is meaningless in the language’s syntax. An int variable’s name, the order of statements in a function’s body, a struct’s field names — none of these change during formatting, because the formatter never touches them at all. The boundary here is sharp: the formatter reads a parse tree and rewrites the same tree with canonical whitespace, it never changes the tree itself. This is why no new assumption is needed at all about a formatted source’s behavior — what the measured program does when it runs is the same after formatting as it was before formatting.

This boundary sits in a different place than vet’s boundary. The formatter never rejects a source at all — it accepts and rewrites every valid source given to it, and never decides at all whether the input is “good” or “bad.” Vet does the exact opposite: it never changes the source at all, it only rejects or passes it. Both tools are members of the same command family, both work at a layer the compiler does not look at, but one does its job by transforming the source, the other by judging the source. This lesson tested both with the same procedure — writing the source to a temporary module and calling the command — and both results were read only as a single flag (did it change, did it pass); at no stage was an error text or a formatted diff printed line by line at all.

Summary

• The language’s own command family covers not only compiling but also formatting and vetting; all three, as in this lesson, can be called from within the program as an argument.
• The formatter turns the source given to it into a single canonical form; 7 of an 18-line source’s lines change in this transformation, the rest were already in canonical form.
• Vet catches a pattern the compiler does not reject (a mismatch between a format verb and an argument’s type); the compiler never checks this agreement at all, because in its eyes the format string is an ordinary string value.
• The limiting measurement: the source before and after formatting gives exactly the same output when run — the form changes, the meaning does not.
• Both tools’ names appeared only in the code block, as the argument of a call; in the prose they were referred to only by their job names.

Next Step

This lesson measured who applies a source’s form and a pattern the compiler overlooks — both are separate members of the language’s own command family, both work at a layer the compiler does not look at. But none of these three tools (the compiler, the formatter, vet) ever reads at all the text that tells what a package offers to the outside — the writing that explains what a name is for. The course’s final lesson looks at this last corner: how is a document produced from the source, which names never enter that document at all, and does the doc tool check a comment’s correctness — or only its position?