HTTP Client

The previous lesson measured the boundary from the receiving side: what a request brings to the server, and how blindly the server trusts the body. This lesson asks the same boundary from the sending side. A client takes on three responsibilities when it sends a request: ensuring the request resolves in a reasonable time, reusing connections that talk to the same server, and closing the body of the response it receives. These three might look independent of each other, but the measurement below shows that two of them (connection reuse and closing the body) are actually two faces of the same decision.

This lesson also uses net/http/httptest like the previous one; but this time it is not httptest.NewRecorder, it is httptest.NewServer — a server running in memory that the client can really open and close a connection against. The server’s address (server.URL) is only used to build requests, never printed in any measurement line; what is measured is not the address itself but how the client behaves against that address. This difference is itself part of the measurement: the previous lesson could run its server without ever touching the network, because what it measured was only the content of the request and the response; this lesson measures the client’s connection management, so it needs a real (but nowhere printed) local connection.

The three responsibilities are handled in order. The first is the guarantee that a request will not wait forever. The second is avoiding the cost of establishing a new connection for every one of a series of successive requests. The third is closing the body of a received response instead of abandoning it — and the measurement below shows that this third responsibility is actually the condition that makes the second one possible.

Timeout: Not a Duration, a Canceled Context

The pattern established in the previous course’s Context Package lesson continues here exactly: a timeout is built not with time.Sleep or a real duration measurement, but with a context that has been already canceled. This keeps the lesson’s output from changing between runs — if a real timeout were tried, how long the request would take to fail would depend on the machine and the load.

// servercore.go — the shared server body for client measurements
package main

import (
	"fmt"
	"net/http"
)

// responderHandler is a handler that returns a fixed text body for every
// request. The body is kept small so reading and closing it does not affect the measurement.
func responderHandler(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "text/plain")
	fmt.Fprint(w, "qty=7 unit=kg")
}

// measureLine prints one of the three outcomes at the boundary (preserved / rebuilt /
// lost) alongside a piece of evidence.
func measureLine(part, class, evidence string) {
	fmt.Printf("%-30s %-10s %s\n", part, class, evidence)
}

// timeout.go — a timeout built not with a duration, but with an already-canceled context
package main

import (
	"context"
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"
)

func main() {
	fmt.Println("-- timeout: not with a duration, but with an already-canceled context --")

	server := httptest.NewServer(http.HandlerFunc(responderHandler))
	defer server.Close()

	ctx, cancel := context.WithCancel(context.Background())
	cancel() // the context was already canceled before the request even started

	request, _ := http.NewRequestWithContext(ctx, http.MethodGet, server.URL, nil)
	_, err := http.DefaultClient.Do(request)

	fmt.Printf("is there a request error=%t\n", err != nil)
	fmt.Printf("does the error match the context cancellation=%t\n", errors.Is(err, context.Canceled))
}

-- timeout: not with a duration, but with an already-canceled context --
is there a request error=true
does the error match the context cancellation=true

ND7. The cancel() call runs before the request is sent; this ordering is deliberate, because the measurement’s goal is to leave no room for any argument about timing. http.DefaultClient.Do is called while the context is already canceled, and the client returns an error immediately, without ever putting the request on the wire. The check errors.Is(err, context.Canceled) comes out true — the returned error is not one the network layer produced itself, it is the wrapped form of the fixed error the context carries. If a real timeout (context.WithTimeout) had been set up, the same mechanism would work, only the error would be context.DeadlineExceeded instead of context.Canceled; both come from the same ctx.Err() path.

ND8. This measurement asks, at this boundary, the same question the previous course’s closing lesson left open: what stopped a goroutine — a checkpoint in its own code path. http.Client follows the same principle: it checks the context’s cancellation in its own internal loop (while establishing the connection, while waiting for the response), and stops and returns the error once it reaches this checkpoint. The difference is that this time the checkpoint is not visible in the source code — the client itself is part of the standard library, but the behavior is the same: a context is not a forcing tool, it is a source of news, and http.Client is one of the parties listening to that news. This consistency comes from the standard library holding its own internal functions to the same principle; once the context package is learned, it behaves the same way regardless of which side of a boundary it is on.

No real duration was used at all in this measurement; calling cancel() immediately after context.WithCancel makes it certain that the request will fail, completely removing the uncertainty of “it might make it in time, it might not.” If a real context.WithTimeout had been used, the measurement’s result would depend on the speed of the machine the verifier runs on — the same source would time out on some runs and not on others, because the server would respond fast enough. An already-canceled context moves this uncertainty from run time to design time.

Connection Reuse: The Cost of Closing the Body

Establishing a TCP connection is not a cheap operation; this is why Go’s http.Transport keeps used connections in a connection pool and tries to reuse them for later requests. The connection pool concept itself was built in the Data Access Layer course; what is measured here is under what condition Go’s standard library http.Transport operates this pool. This reuse is not unconditional: unless a response’s body is fully read and closed, the connection underneath it does not go back into the pool. This is a responsibility independent from the timeout in the previous section — even if the request completes successfully, even if the status code returns 200, ignoring the body can cost the connection.

The measurement below observes this with net/http/httptrace: the GotConn field of httptrace.ClientTrace reports, for every request, whether the connection obtained was reused (the Reused field, a bool). This package is also part of the standard library and exposes no real address or port, it only gives a logical flag.

The previous section used http.DefaultClient while measuring the timeout, because what was measured there was context cancellation, and which client was used did not change the outcome. This section explicitly builds its own clients, because what is measured here is precisely the connection pool a client carries — using http.DefaultClient would make this measurement liable to mix with connections left behind by code running in another lesson of the course. This is the first sign that an HTTP client carries invisible state (open connections): two different *http.Client values, even against the same server, can accumulate a completely different history from each other.

// connectionreuse.go — closing the body determines whether the connection returns to the pool
package main

import (
	"context"
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
	"net/http/httptrace"
)

// requestWithTrace sends a request with the given client; learns whether the
// connection was reused with httptrace and closes the body according to closeBody.
func requestWithTrace(client *http.Client, url string, closeBody bool) (reused bool) {
	ctx := httptrace.WithClientTrace(context.Background(), &httptrace.ClientTrace{
		GotConn: func(info httptrace.GotConnInfo) { reused = info.Reused },
	})
	request, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
	response, err := client.Do(request)
	if err != nil {
		return false
	}
	if closeBody {
		io.ReadAll(response.Body)
		response.Body.Close()
	}
	return reused
}

func main() {
	fmt.Println("-- connection reuse: with the body closed and left unclosed --")

	server := httptest.NewServer(http.HandlerFunc(responderHandler))
	defer server.Close()

	// Each client carries its own Transport, so the connection pools stay
	// independent of each other and an unclosed body on one client does not affect the other.
	clientA := &http.Client{Transport: &http.Transport{}}
	firstA := requestWithTrace(clientA, server.URL, true)
	secondA := requestWithTrace(clientA, server.URL, true)
	fmt.Printf("body closed each time: first=%t second=%t\n", firstA, secondA)

	clientB := &http.Client{Transport: &http.Transport{}}
	requestWithTrace(clientB, server.URL, false)
	secondB := requestWithTrace(clientB, server.URL, true)
	fmt.Printf("first request's body never closed: second=%t\n", secondB)
}

-- connection reuse: with the body closed and left unclosed --
body closed each time: first=false second=true
first request's body never closed: second=false

ND9. In the two requests made with clientA, the body is read and closed every time. Reused=false on the first request is an expected result, since there is not yet an open connection; what is actually measured is the second request. Because the body was fully read and closed on the first request, the connection has already gone back into the pool, and the second request inherits the same connection with Reused=true. This shows why the client side’s third responsibility (closing the body) is not just “cleanup” but a direct performance decision.

Each of clientA and clientB carries its own *http.Transport value; this is a deliberate choice. If the Transport field were left empty, both would use the standard library’s shared default transport, and the two clients’ connection pools would actually be the same pool — clientB’s unclosed body could affect clientA’s next request too, even though they have different names. Giving each its own Transport isolates the two scenarios from each other: the measured effect really comes from clientB’s own unclosed body, not from a side effect of another request.

ND10. In the first request made with clientB, the body is never read, never closed. The second request is sent with the same client, with its body properly closed, but the result is still Reused=false. The reason is that the first request’s connection is still considered busy: Transport never puts a connection whose response body was left unread and unclosed back into the pool, because it does not know when that connection will become free. The second request is therefore forced to open a new connection — the first request’s unclosed body also affects the request that comes after it.

It is clear that this effect grows with scale: a program that sends hundreds of successive requests but forgets to close some of its bodies keeps one connection busy forever for every unclosed body. The program itself does not crash, does not error either — only the client’s open connection count builds up over time, and this buildup stays as silent as the body itself.

ND11. The rule that emerges when these two measurements are read together is clear: connection reuse gives a single answer to the question of whose job it is — the caller’s. The standard library does not automatically read and close the body; if Body.Close() is never called, or even if it is called but the body is not first fully read, the connection leaks, or at least cannot be reused. This stands in the same place as the principle seen in the previous lesson’s middleware chain: the standard library offers a mechanism on this side of the boundary, but using that mechanism correctly stays the responsibility of the side that writes the code.

This rule also adds an example to the type system’s role at the boundary: the type of response.Body (io.ReadCloser) itself says nowhere “do not close me before fully reading me.” The compiler does not reject a response whose body is left unread — the source compiles fine, runs fine, only the connection pool quietly shrinks. A contract of this kind (that a resource must be consumed before it is closed) lives not in the type itself but in its documentation and in the habits of the side that writes the code.

The Limiting Measure: The Body Is a Reader, Not a Value

The previous two measurements showed why closing the body matters. This measurement clarifies what the body is: response.Body is a stream implementing the io.ReadCloser interface, consumed once just like the request body, and nothing is left once it is consumed. This is a fundamentally different behavior from the copyable values seen in earlier courses of the language (a slice, a struct): assigning a slice to two variables and reading both of them separately was possible, because a slice was a value. A reader carries not a value but a position — that position advances as it is read, and does not go back.

// bodyreading.go — the response body is a reader, not a value
package main

import (
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
)

func main() {
	fmt.Println("-- the response body is a reader, it cannot be read twice --")

	server := httptest.NewServer(http.HandlerFunc(responderHandler))
	defer server.Close()

	response, _ := http.Get(server.URL)
	defer response.Body.Close()

	firstRead, _ := io.ReadAll(response.Body)
	secondRead, secondErr := io.ReadAll(response.Body)

	measureLine("response body (read on the client)", "lost",
		fmt.Sprintf("type=%T first-length=%d", firstRead, len(firstRead)))
	fmt.Printf("second read length=%d error=%v\n", len(secondRead), secondErr)
}

-- the response body is a reader, it cannot be read twice --
response body (read on the client) lost       type=[]uint8 first-length=13
second read length=0 error=<nil>

ND12. The first io.ReadAll call reads the body from start to end and returns a thirteen-byte slice — what reaches this side of the boundary is again []byte, the same class seen in the previous lesson: lost. The second io.ReadAll call returns a result of length 0 and a nil error; because nothing is left to read, the read operation finishes instantly, but this does not count as an error. If the body were a slice or a string, it would be possible to read the same value as many times as wanted; because it is a reader, the moment it is consumed, only emptiness remains behind it.

ND13. The practical consequence of this behavior: a response body can only be parsed once. If a function wants to both log the body and decode it to a target, it has to first copy the body into a buffer (like bytes.Buffer) and do the two different reads through that buffer — returning to the source itself a second time is not possible. This limitation is also connected to the connection reuse from the previous section: copying the body into a buffer means fully consuming it at the same time, so this copying operation both makes a second read possible and lets the connection underneath go back into the pool — the same io.ReadAll call solves two separate problems at once.

The full distribution of the course’s thirteen boundary crossings (1 preserved, 5 rebuilt, 7 lost) is built in the next lesson, JSON Serialization; this lesson only measured the client side’s three responsibilities and the body’s nature as a reader.

Summary

• A timeout is built with an already-canceled context instead of a duration; http.Client checks the context’s cancellation at its own internal checkpoints and stops the request there.
• The request error checked with errors.Is is the wrapped form of the fixed error the context carries (context.Canceled or context.DeadlineExceeded).
• Whether a connection will be reused is measured with the httptrace.GotConnInfo.Reused field; unless the body is fully read and closed, the connection does not return to the pool.
• The effect of a request whose body was not closed also reaches the request after it: a new connection opens because the previous connection is still considered busy.
• The response body is a reader, not a value: once it is fully read, a second read returns empty, without an error.
• This lesson measured the boundary from the sending side; the full distribution of the course’s thirteen boundary crossings is not yet built here, the next lesson builds it.

Next Step

Both lessons measured the boundary’s carrier — the request and the response carry bytes and text; the body can be read once. In neither of these two lessons did the body’s content matter at all; any random text in place of "qty=3" or "qty=7 unit=kg" would have given the same results, because what was measured was the body’s behavior, not the data it carried. But neither has yet asked this question: when the sending side sends a struct, what does it leave behind for the other side to rebuild it? The next lesson measures this question with JSON and produces the course’s binding number: a struct’s field names are carried across with struct tags, but this carrying has limits — a field with no counterpart on the receiving side drops silently, and “it got rebuilt” appearing to be the case does not always mean it was “rebuilt correctly.”