--- title: 'File and I/O Interfaces' source: 'https://academia.sh/en/courses/go-applications/file-and-io-interfaces' course: 'The Standard Library and Network Applications' language: en updated: '2026-09-14T10:59:31+00:00' license: 'CC BY-SA 4.0' objectives: - 'Measures that none of the file, buffer and string sources carries type information when passed through the same reader interface' - "Shows that the same writer interface writes to a file and a buffer identically, and that the two destinations' content matches exactly" - 'Limits by running that reading with a small buffer returns the data and the end of file in separate steps' --- # File and I/O Interfaces Four separate sources (a file, a buffer, a string and the network body measured earlier) are passed through the same two methods — read and write — and none of them is shown to carry type information across the boundary. The limiting measurement runs a small buffer to show that reading returns the data and the end of file in separate steps, and that managing this distinction is the caller's job. The last lesson of the Networking and Data Formats topic closed with a question: a framing rule tells you where a message starts and where it ends, but it does not tell you what the bytes inside that boundary are. This topic asks the same question without carrying it outside the program, to the network. A byte read from a file, a byte accumulated in a buffer, and a byte that just arrived from the network — all three are looking for an answer to the same question: does a byte sequence coming from the other side say, on its own, what type it belongs to? The answer is no, and this topic's first lesson shows where that answer comes from: Go's standard library makes the file, the buffer, the network body and the string speak through a single shared contract — read and write. This contract is so narrow that it says nothing about the source itself; it only streams bytes. The Networking and Data Formats topic already measured that the HTTP body follows this same contract; that is not rebuilt here, and this lesson passes three **new** sources — a file, a buffer, a string — through the same contract and shows the same result (that type information does not carry across) for each one separately. **Determinism note:** the file source sits in a temporary directory built with `os.MkdirTemp`; the directory's own path is never printed, and only numbers related to its content are used. ## Read and Write Are Two Methods Go's standard library has a single condition for a source to count as "readable": carrying a method with the signature `Read(p []byte) (n int, err error)`. Interfaces themselves — the implicit implementation model, where a type satisfies an interface only by carrying the right methods without a separate declaration saying "I satisfy this interface" — were established in the Interfaces, Errors and Generics course and are not rebuilt here. This lesson measures that model's most common example in the standard library: the `io.Reader` interface asks for a single `Read` method, and the file, network, compression, encryption and text-processing packages are **all** built on top of this one interface. The write side has a symmetric interface: `io.Writer`, whose only method is `Write(p []byte) (n int, err error)`. The smallness of these two interfaces is a design decision, not a shortcoming. Because the only condition for "readable" is this one return signature, types that share no common ancestry at all — `os.File`, `bytes.Buffer`, `strings.Reader`, a network body — can each satisfy the same interface **independently**. None of them has to imitate the others; each writes its own `Read` method against its own internal data, and outside code can run the same loop without ever knowing which type it holds. The implicit implementation is especially visible here: the word `io.Reader` never appears in the definition of the `os.File` type, but because it carries a `Read` method with the right signature, it satisfies the interface — the connection is built by signature match, not by declaration. ## Four Sources, One Interface **SD1.** — The same four-character text (`"bolt"`) is deliberately placed into the three sources below (a file, a buffer, a string); the goal is to compare the tool's **behavior**, not the sources' content. The fourth source — the network body — was already run with the same result (lost) in the Networking and Data Formats topic and is not rebuilt here. ```go // read.go — a helper that passes three separate sources through the same reader interface package main import ( "bytes" "fmt" "io" "os" "strings" ) func read(r io.Reader) (int, string) { b, _ := io.ReadAll(r) return len(b), fmt.Sprintf("%T", b) } type source struct { name string r io.Reader } func main() { dir, _ := os.MkdirTemp("", "sd1") defer os.RemoveAll(dir) os.WriteFile(dir+"/content.txt", []byte("bolt"), 0o644) file, _ := os.Open(dir + "/content.txt") defer file.Close() var buffer bytes.Buffer buffer.WriteString("bolt") str := strings.NewReader("bolt") sources := []source{ {"file", file}, {"buffer", &buffer}, {"string", str}, } for _, k := range sources { length, kind := read(k.r) fmt.Printf("%-10s length=%d type=%s\n", k.name, length, kind) } } ``` ``` file length=4 type=[]uint8 buffer length=4 type=[]uint8 string length=4 type=[]uint8 ``` The three sources have completely separate implementations: `*os.File` talks to the operating system through system calls, `*bytes.Buffer` manages a slice that grows in memory, and `*strings.Reader` walks over an immutable string. The `read` function never sees this difference — its parameter is `io.Reader`, which expects only the `Read` method. All three produce exactly the same result: four bytes, and a type that is always `[]uint8` (in Go, `byte` is an alias for `uint8`; `%T` shows it under its own name). The source's original type — file, buffer, or string — does not survive anywhere past this point; the only thing the read operation produces is a raw byte slice. This is the first concrete example of this topic's measurement axis: the value crossing the boundary (the data `Read` returns) falls into the **lost** class, because no matter who sent it back on the other side, the result stays in the same raw type. Four bytes read from a file and four bytes read from a string are indistinguishable after the read operation. The power of implicit implementation does not stop at the standard library's own types. Any type that writes the `Read` signature correctly satisfies the `io.Reader` interface — even a handwritten type that never touches the filesystem or memory. ```go // handwritten.go — a type that writes its own Read method also satisfies the io.Reader interface package main import ( "fmt" "io" ) // RepeatReader is a handwritten reader that repeats one byte the requested number of times. type RepeatReader struct { Byte byte Remaining int } func (t *RepeatReader) Read(p []byte) (int, error) { if t.Remaining == 0 { return 0, io.EOF } n := 0 for n < len(p) && t.Remaining > 0 { p[n] = t.Byte t.Remaining-- n++ } return n, nil } func main() { var source io.Reader = &RepeatReader{Byte: 'x', Remaining: 5} data, _ := io.ReadAll(source) fmt.Printf("data from our handwritten reader: %q type=%T\n", data, data) } ``` ``` data from our handwritten reader: "xxxxx" type=[]uint8 ``` `RepeatReader` neither opens a file nor wraps a string; it produces bytes according to its own `Remaining` counter. Yet `io.ReadAll` sees no difference between it and the other three sources, because the only thing it looks for is a `Read` method with the right signature. This shows that the four sources' common ground is actually open to a fifth, a sixth, any number of new sources — the interface stays fixed; the number of implementers is unbounded. ## The Write Side: io.Writer The write direction has the same symmetry. When data is written to a destination that implements `io.Writer`, the writer does not care whether the destination is a file or a buffer; the only thing that matters is how many bytes were written. **SD2.** — The same five-byte data (`"screw"`) is written to both a file and a buffer; both are then read back and checked for equality with each other and with the source. ```go // write.go — the same data is written to two different writer destinations package main import ( "bytes" "fmt" "io" "os" ) func write(w io.Writer, data []byte) (int, string) { n, _ := w.Write(data) return n, fmt.Sprintf("%T", w) } func main() { dir, _ := os.MkdirTemp("", "sd2") defer os.RemoveAll(dir) file, _ := os.Create(dir + "/output.txt") var buffer bytes.Buffer n1, t1 := write(file, []byte("screw")) file.Close() n2, t2 := write(&buffer, []byte("screw")) fmt.Printf("written to file n=%d type=%s\n", n1, t1) fmt.Printf("written to buffer n=%d type=%s\n", n2, t2) readBack, _ := os.ReadFile(dir + "/output.txt") fmt.Println("read back from file :", string(readBack)) fmt.Println("read back from buffer :", buffer.String()) fmt.Println("are the two equal :", string(readBack) == buffer.String()) } ``` ``` written to file n=5 type=*os.File written to buffer n=5 type=*bytes.Buffer read back from file : screw read back from buffer : screw are the two equal : true ``` The `write` function's signature is symmetric to `read`'s: its parameter is not a concrete destination but the `io.Writer` interface. `*os.File` and `*bytes.Buffer` are two types that do not resemble each other at all, yet both can pass through the same function because both carry a `Write` method. The result is symmetric too: the same five bytes are written to both, and the content read back from each is equal to the other. This line adds one more answer to the topic's question — on the write side too, the value crossing the boundary is carried as the same raw byte sequence regardless of the destination's own type; whether the destination is `*os.File` or `*bytes.Buffer` never affects the content written. ## Composition: io.Copy and bufio Are Built on the Same Interface Narrow interfaces let the standard library write generic tools between a source and a destination. `io.Copy(dst io.Writer, src io.Reader)` moves data from a source to a destination without ever knowing the concrete type of its parameters — the source can be a file, a buffer, a string, or any other `io.Reader`; the destination can likewise be any `io.Writer`. The same idea holds in `bufio`: `bufio.NewScanner` takes an `io.Reader` and turns it into a wrapper that reads it line by line (or word by word), without ever asking whether the source it wraps is a file or a string. ```go // combine.go — io.Copy and bufio work on any reader/writer without knowing the concrete type package main import ( "bufio" "bytes" "fmt" "io" "os" "strings" ) func main() { dir, _ := os.MkdirTemp("", "sd4") defer os.RemoveAll(dir) os.WriteFile(dir+"/source.txt", []byte("nut and bolt"), 0o644) file, _ := os.Open(dir + "/source.txt") defer file.Close() var dest bytes.Buffer n, _ := io.Copy(&dest, file) fmt.Println("bytes moved from file to buffer :", n) fmt.Println("buffer contents :", dest.String()) lineScanner := bufio.NewScanner(strings.NewReader("first line\nsecond line")) lineCount := 0 for lineScanner.Scan() { lineCount++ } fmt.Println("line count counted by bufio:", lineCount) } ``` ``` bytes moved from file to buffer : 12 buffer contents : nut and bolt line count counted by bufio: 2 ``` The `io.Copy` call's signature contains no concrete type at all — no file name, no buffer kind, just two interfaces. This function is written once in the standard library and is reusable in every combination — file to network, network to buffer, buffer to another file; no matter how many concrete types the source and destination have, `io.Copy` itself never changes. `bufio.NewScanner` follows the same principle: its input is an `io.Reader`, and although it wraps another `io.Reader`, it offers a richer interface (line-by-line scanning) — small interfaces composing on top of each other is the direct result of producing small, pluggable pieces instead of writing one large "file reader" type. ## Limiting Measurement: Partial Reads and End of File The two measurements so far used `io.ReadAll` — a helper that runs the loop internally until the source is exhausted. But the `Read` method itself does not guarantee this: a single call is not required to return the full number of requested bytes, and the data ending and the error arriving are not required to happen in the same call. This distinction is written clearly in the standard library's documentation, but it is hard to see without running it. **SD3.** — A five-byte string is deliberately read with a two-byte buffer; how many bytes arrive at each step, and at which step the error appears, are printed separately. ```go // partial.go — reading with a small buffer returns the data and end of file in separate steps package main import ( "fmt" "strings" ) func main() { source := strings.NewReader("board") buffer := make([]byte, 2) total := 0 step := 0 for { n, err := source.Read(buffer) step++ fmt.Printf("step=%d read=%d chunk=%q err=%v\n", step, n, buffer[:n], err) total += n if err != nil { break } } fmt.Println("total bytes read:", total) } ``` ``` step=1 read=2 chunk="bo" err= step=2 read=2 chunk="ar" err= step=3 read=1 chunk="d" err= step=4 read=0 chunk="" err=EOF total bytes read: 5 ``` The five-byte string "board" requires four separate `Read` calls when read with a two-byte buffer — the third step leaves a single byte, and the error is still `nil` at that step. The end-of-file signal (`io.EOF`) arrives only on the fourth call, with no data at all. This is a contract of the source's behavior: `Read` can return a byte count greater than `0` and a non-nil error at the same time in a single call, or it can split the data and the error across separate calls. Code that relies only on a helper like `io.ReadAll` never sees this distinction, because the helper completes the loop internally. But code that calls `Read` directly can skip the last byte at the third step if it assumes the data ending and the error arriving happen in the same step. This is the boundary specific to this lesson on the topic's measurement axis: the read interface does not carry a loss — all five bytes arrived in the right order — but it never takes on the job of separating a partial read from the end of file. That distinction stays the responsibility of the code calling `Read`; the standard library's `io.ReadFull` and `io.ReadAll` helpers take on this responsibility, but they are tools the source itself does not require. For an in-memory source like `strings.NewReader`, managing this distinction by hand looks unnecessary — all the data is already ready in memory, and `Read` not giving everything at once is just a matter of choice. But the interface's contract is the same for every implementer; for an `io.Reader` that reads from a network socket or a pipe, a partial read is not a choice, it comes from the nature of the source — if the other side sends the data in pieces, `Read` returning everything in a single call is not possible. The `io.Reader` contract does not distinguish between these two cases (an in-memory source and a source arriving in pieces); both carry the same signature, and code that correctly manages partial reads works for both. ## Summary - `io.Reader` and `io.Writer` are two of the language's narrowest interfaces; they ask for a single `Read` and a single `Write` method respectively, which is why types that resemble each other not at all — a file, a buffer, a string — can each satisfy the same contract independently. - The same content read from a file, a buffer and a string becomes indistinguishable after the read operation: all three give a result of the same length and the same `[]uint8` type — the value crossing the boundary carries no type information. - The same symmetry holds in the write direction too: `*os.File` and `*bytes.Buffer` pass through the same `write` function, and the content written to each turns out equal to the other, byte for byte. - Tools like `io.Copy` and `bufio.NewScanner` work without ever knowing the concrete type; small interfaces can therefore compose freely across a file, a buffer and a string. - A partial read and the end of file are separate events: reading with a small buffer can split the data across several steps, and the end-of-file signal arrives independently of the data, in its own step. - Managing this distinction is the caller's job; helpers like `io.ReadAll` take on this responsibility, but the interface itself does not require it. ## Next Step All three sources measured in this lesson sat somewhere reachable while the program was running: on a filesystem path, or in the program's own memory. But the data a program needs is not always available while it is running — a configuration file, an HTML template, a static asset, can instead travel inside the program's own binary. The next lesson looks at the mechanism that moves this link to compile time: how does a source file sit directly inside a compiled program without ever being looked up while it runs?