Test Package Basics

The previous course measured that type information about a value mostly disappears once it crosses a boundary: someone reading the source usually could not see what type the value on the far side of a boundary was. This course moves the same question up one level — now the subject is not a single value but a whole program. You cannot see from the source whether a program runs correctly, which of its lines ran, where it allocated memory, or which platform it was built for. So who states this information — the source itself, an answer a tool derives without running anything, or a result learned only by actually running it?

This first lesson asks the question in its narrowest form: did a test function pass, or did it fail? The measurement uses a small package that keeps recurring throughout the topic — an inventory package that holds a store item and the total value of a list of items. The item itself is simple: a name, a quantity, a unit price in cents. The total value is a simple product-sum. This lesson’s question is not what the package does, but who states that the package does it correctly.

This question carries particular weight in a program running in production. A source review can read a file’s lines and decide the logic looks correct — but there is a gap between “looked correct” and “was tested to be correct,” and the only thing that closes that gap is a test actually run alongside the source. The topic’s remaining five lessons measure different corners of this same gap: whether the case list itself is written in the source, how a network layer is tested without ever actually engaging, what a fake object standing in for a dependency leaves out, which of two writings’ numbers is reproducible, and where the limit of what a coverage ratio states lies. All of them start from the same root: a test’s verdict comes not from the source, but from the run.

The Test Function and Helper Follow the Language’s Own Rule

The test package is part of the language’s own standard library, and it imposes its own rule: for a function to be recognized as a test by the test package, its name must start with Test, the letter after Test must not be lowercase (TestTotal is recognized, Testtotal is not), it must take a single *testing.T parameter, and the file’s name must end in _test. These three rules are not syntax the compiler checks — they are the test package’s own scanning rule, and anyone reading the source who knows this rule can tell from the text which functions will run as tests. The rule itself requires no registration step: there is no need to add a test function to a separate list or declare it with a call — the test package finds it on its own the moment the name and signature are right.

The same file also has a provideSamples function, but it is not a test — its name does not start with Test, so the test package never runs it. It is instead a helper called by two test functions: defined once so the same sample item list is not written repeatedly, and the t.Helper() call in its body tells the test package “report a failure from the caller’s line, not from here.” This distinction — which function is a test and which is a helper — can also be read from the source; the test package does not have to run any function to make this decision, it only looks at the name and the signature.

TM1. The test file below defines two functions, TestTotalValue and TestNoAssertion; both follow the test package’s rule with their names and signatures, and how many there are can be learned by counting the func Test occurrences in the source — with no tool called at all.

// tools/tools.go — shared routines that call the language's own command family in a temporary module from inside a program
package tools

import (
	"os"
	"os/exec"
	"path/filepath"
)

// Setup builds a temporary module with the given files and returns its directory; the path is never printed anywhere.
func Setup(files map[string]string) string {
	dir, _ := os.MkdirTemp("", "trial")
	os.WriteFile(filepath.Join(dir, "go.mod"), []byte("module trial\n\ngo 1.24\n"), 0o644)
	for name, content := range files {
		os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644)
	}
	return dir
}

// Run runs the language's own command in the temporary module; it returns success, not the output itself.
func Run(dir string, extraEnv []string, args ...string) (string, bool) {
	cmd := exec.Command("go", args...)
	cmd.Dir = dir
	cmd.Env = append(append(os.Environ(), "GOTOOLCHAIN=local"), extraEnv...)
	output, err := cmd.CombinedOutput()
	return string(output), err == nil
}

Setup is needed so the measurement starts from a completely empty module every time; otherwise a file left behind by a previous run could affect the result. Run always calls the language’s own command family the same way: as a subprocess, with a fixed environment, returning only the output together with a success value. These two routines are used the same way throughout the topic’s six lessons; no lesson names the command in its prose, it only appears as a call argument inside a code block.

The tools functions live in their own package — the package and visibility separation set up in the Go Fundamentals course works exactly the same way here: Setup and Run start with an uppercase letter, so they can be called from the main package by importing the lesson/tools path. The measurement’s actual body — which source is being tested, which question is being asked — always stays in the main package; the package boundary is drawn only to write the repeated setup and call logic once and use it across all six lessons.

The Verdict Comes Only From Running

How many tests a test file contains can be read from the source, but whether those tests pass cannot be read the same way. When go test is called, the test package first compiles every file in the package, then finds the functions whose names start with Test and runs each one in turn, checking at the end of each whether it was cut short. These two steps — compiling and running — are separate: a test file can be compilable while none of the tests inside it pass, or the reverse. The compile step is also a source of information, but from a different class: whether a package compiles can be learned by passing the source through the compiler without ever running the tests — so this is tool information. Whether a compiled package’s tests pass, though, only becomes clear after compiling finishes, after the tests are actually run; that is run information. The measurement below keeps the two apart without mixing them: it runs both tests against a correct version of the inventory package, then reruns the same test file, unchanged, against a broken version of the package.

// measurement.go — the test's verdict is read not from the source but by running it
package main

import (
	"fmt"
	"strings"

	"lesson/tools"
)

const inventoryText = `package trial

type Item struct {
	Name     string
	Quantity int
	Price    int
}

func TotalValue(items []Item) int {
	total := 0
	for _, item := range items {
		total += item.Quantity * item.Price
	}
	return total
}
`

const brokenInventoryText = `package trial

type Item struct {
	Name     string
	Quantity int
	Price    int
}

func TotalValue(items []Item) int {
	total := 0
	for _, item := range items {
		total += item.Quantity
	}
	return total
}
`

const testText = `package trial

import "testing"

func provideSamples(t *testing.T) []Item {
	t.Helper()
	return []Item{{"pen", 3, 250}, {"notebook", 2, 990}}
}

func TestTotalValue(t *testing.T) {
	items := provideSamples(t)
	if got := TotalValue(items); got != 2730 {
		t.Errorf("TotalValue = %d, expected 2730", got)
	}
}

func TestNoAssertion(t *testing.T) {
	provideSamples(t)
}
`

func testCount(source string) int {
	return strings.Count(source, "\nfunc Test")
}

func main() {
	fmt.Println("-- source: test count in the test file --")
	fmt.Println("test function count in the file:", testCount(testText))

	fmt.Println()
	fmt.Println("-- run: verdict against the correct source --")
	correct := tools.Setup(map[string]string{"inventory.go": inventoryText, "inventory_test.go": testText})
	correctOutput, _ := tools.Run(correct, nil, "test", "-v", "./...")
	fmt.Println("TestTotalValue passed:", strings.Contains(correctOutput, "--- PASS: TestTotalValue"))
	fmt.Println("TestNoAssertion passed:", strings.Contains(correctOutput, "--- PASS: TestNoAssertion"))

	fmt.Println()
	fmt.Println("-- run: same test file, verdict against the broken source --")
	fmt.Println("test function count in the file:", testCount(testText))
	broken := tools.Setup(map[string]string{"inventory.go": brokenInventoryText, "inventory_test.go": testText})
	brokenOutput, _ := tools.Run(broken, nil, "test", "-v", "./...")
	fmt.Println("TestTotalValue passed:", strings.Contains(brokenOutput, "--- PASS: TestTotalValue"))
}

-- source: test count in the test file --
test function count in the file: 2

-- run: verdict against the correct source --
TestTotalValue passed: true
TestNoAssertion passed: true

-- run: same test file, verdict against the broken source --
test function count in the file: 2
TestTotalValue passed: false

TM2. The only thing that changes between the two runs is the content of inventory.go; the test file is byte-for-byte identical, and the test count counted from the source is also the same in both: 2. What changes is the verdict these two tests give when run — TestTotalValue says failed when run against the broken version that forgets to multiply by the unit price. The only way to foresee this difference by reading the source is to manually compare the two TotalValue implementations and decide for yourself which one is correct — the test package itself does not reach this verdict by reading the source, but by actually running the go test call. The measurement above does exactly this: the Run routine runs the go test -v ./... command as a subprocess, and the program reads a truth value only by checking for the presence of a --- PASS: <name> line in the output. Neither the error text nor the test’s raw output is printed anywhere — only the single thing asked for is returned: whether this test passed.

Both runs of the measurement use the same Setup routine, the same test file, and the same command; the only difference is which inventory.go content gets written into the temporary module. This shows that the result comes not from the test framework but from the source being tested — the measurement itself is written once and invoked twice, and the difference between the two calls lies only in the input.

The measurement uses the -v flag, and this choice is not arbitrary: without the flag, go test gives only a single verdict for the whole package — success if every test in the package passed, failure if any one failed. Reading what verdict TestTotalValue and TestNoAssertion give separately requires verbose output, because the test package only writes this information on a separate line per test when -v is requested. The package-level single verdict is also run information, but at a coarser granularity: it does not say which test failed, only whether anything failed at all. Once hundreds of tests accumulate in a single package in a production codebase, the difference between these two granularities — knowing that something broke versus knowing what broke — turns into a large difference in practice.

The Limiting Measurement: What a Passing Test Says

In the run above, TestNoAssertion also passed — it would keep passing against the broken source too (judging by the file), because this function does not test anything at all. It calls the provideSamples helper, discards the returned value without using it, and contains no t.Error or t.Fatal call. In the eyes of the test package, a test function is considered passed if it finishes running without being cut short by a t.Error, a t.Fatal, or a panic — whether the function’s body contains any assertion at all does not affect this verdict in any way.

TM3. This is the topic’s first limiting measurement: a passing test does not say it was correct, only that what it tested held — and a test that tests nothing at all passes in exactly the same way, because what the test package measures is not whether an assertion exists, but whether an existing assertion was cut short. TestNoAssertion is the plainest view of this difference: its name promises a check, but its body verifies nothing, and the test package itself carries no mechanism to notice this gap. A test package’s not asking this question is not a flaw but its own boundary — the test package only answers the question “was something cut short while running,” not “is this assertion sufficient.” This limit stays true in the topic’s later lessons too: the next lesson counts a case list from the source and asks whether the number of subtests that run matches the number of cases in the source — but it never asks “are these cases enough,” because the test package never answers that question at all.

This limit has a concrete counterpart in production code: even if a function’s body is left entirely empty with only a comment noting it will be completed later, a test that calls that function and verifies nothing still passes. A test suite looking green — all tests appearing to pass — is therefore not on its own sufficient proof; how many assertions actually ran is a separate question, and the topic’s closing lesson re-measures exactly this distinction through the coverage ratio.

This limit is not a flaw in the test package but a consequence of where its boundaries were drawn. The test package was designed to answer one question: whether a run was cut short or not. Whether an assertion is meaningful — whether it would give the same result on a wrong input too, whether it covers a wide enough range — is an entirely different question, and answering it is the writer’s job, not the tool’s. TestNoAssertion shows this gap in its plainest form because it contains no assertion at all; but the same gap can appear just as readily in an assertion written with a condition that, by mistake, is always true — and in that case the gap is far less visible to an eye reading the source.

The inventory package itself also stays fixed for the rest of this lesson: Item and TotalValue keep working on the same example that develops across the topic. The next lesson tests another function of the same package with a table-driven test, and asks where the case list itself is written.

Summary

• Whether a function counts as a test to the test package depends on its name starting with Test, taking a single *testing.T parameter, and its file name ending in _test; this rule can be read from the source.
• The test count in the source can be counted from the text, but whether a test passes can only be learned by actually running go test — the same test file gives different verdicts against a correct and a broken version of the package.
• The measurement tool is called from inside the program, and only the requested truth value is read; the test’s raw output is never printed anywhere.
• The limiting measurement: a passing test does not say it was correct, only that what it tested held — a test containing no assertion at all passes in exactly the same way.
• The inventory package continues as the same example throughout the topic; this lesson used only TotalValue.

Next Step

This lesson measured that a test’s verdict can only be learned by running it, and that a passing test says “was not cut short,” not “correct.” The measurement itself also relied on two routines that will not change for the rest of the topic — tools.Setup and tools.Run; the following lessons will call the same two routines again with their own test file, their own source, and their own question — with one exception, the HTTP lesson, which never runs go test as a subprocess because it calls the handler directly. The next lesson takes another function of the same package and tests it with a table-driven test — this time the question is not the test’s verdict but the case list itself: how many cases are written, how many of them actually run a different line, and whether adding a case always means testing more.