Lesson 12 / 12

The End-to-End Trace of a Request

Joining the entire chain from name resolution to the drawing of the response, the information handed off between stages, and the symptoms of each link's failure.

Contents

A single example was followed throughout the course: a request made to the address example.test. Each lesson opened up one part of this journey — how packets are carried, what addresses mean, how a name is resolved, where the server stands, how the exchange is shaped, and how the channel is secured.

This lesson joins the links together. The goal is not to add a new concept but to make the flow of information between them visible.

The Whole Chain

When https://example.test/ is typed into the address bar and the request is sent, the following steps are carried out.

1. Parsing the address. The written text is split into parts: scheme https, host example.test, path /. The scheme determines that the port will be 443. Handed off to the next stage: the name to resolve.

2. Name resolution. The provider asks the resolver: a cached answer returns directly, otherwise the chain from the root to the authoritative server runs, and the address is stored for its lifetime. Handed off to the next stage: the target IP address.

3. Establishing a connection. A connection is opened to the target address and port. Packets pass through independently operated autonomous systems along the way; no endpoint determines the route. Handed off to the next stage: a byte stream capable of carrying data.

4. Security handshake. The server presents its certificate; the client brings the signature chain to a trust anchor and checks that the certificate’s name matches example.test. The two parties generate the session key. Handed off to the next stage: an encrypted and authenticated channel.

5. Sending the request. The request line, headers, and empty line are written. The Host header carries the host name — since the address is by now reduced to a number, only this header tells the server which site to serve. Handed off to the next stage: the method, path, and headers.

6. The server’s processing. The server selects the site by looking at the Host header, finds the resource by looking at the path, and produces a status code with a body. Handed off to the next stage: the status code, headers, body.

7. Interpreting the response. The client reads the status code; if it is a redirect, it returns to step 1 with a new target. Content-Type states what the body is, Content-Length states where it ends. Handed off to the next stage: the document.

8. Parsing and subresources. The document is turned into a tree of elements. Steps 1–7 repeat for every resource referred to; requests made to the same host use the existing connection.

9. Layout and drawing. The position of elements is computed and the page is drawn.

Two observations about the chain’s structure matter. First, each stage hands off only a single thing to the next and forgets what came before: after parsing, the name matters; after connecting, the address; after the handshake, the certificate becomes irrelevant. Second, the name comes back at step 5, because the address is no longer distinguishing by then — what virtual hosting, established in the Hosting Models lesson, requires.

Running the Chain

All of these steps can be set up locally. The following server is wrapped with TLS and selects a site by the Host header, bringing together the pieces built separately throughout the course.

import ssl
from http.server import BaseHTTPRequestHandler, HTTPServer

SITES: dict[str, bytes] = {
    "example.test": b"<!doctype html>\n<html lang=\"en\"><body><h1>Example Page</h1></body></html>\n",
}


class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def log_message(self, fmt: str, *args: object) -> None:
        print(f"[server] {self.command} {self.path} Host={self.headers.get('Host')}", flush=True)

    def do_GET(self) -> None:
        name = self.headers.get("Host", "").split(":")[0]
        body = SITES.get(name)
        self.log_request(200 if body else 404)
        if body is None:
            body = b"<h1>Unknown host</h1>\n"
            self.send_response_only(404)
        else:
            self.send_response_only(200)
        self.send_header("Content-Type", "text/html; charset=utf-8")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)


context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain("cert.pem", "key.pem")
server = HTTPServer(("127.0.0.1", 8443), Handler)
server.socket = context.wrap_socket(server.socket, server_side=True)
server.serve_forever()

The certificate is the file produced by the command in the Role of HTTPS lesson. The request is made with the following command; --resolve stands in for name resolution, --cacert makes the certificate a trust anchor:

curl -sv --cacert cert.pem --resolve example.test:8443:127.0.0.1 \
  -A "example-client" https://example.test:8443/

The trace on the client side:

* Added example.test:8443:127.0.0.1 to DNS cache
* Hostname example.test was found in DNS cache
*   Trying 127.0.0.1:8443...
* Connected to example.test (127.0.0.1) port 8443
*  CAfile: cert.pem
*  CApath: none
* Server certificate:
*  subject: CN=example.test
*  subjectAltName: host "example.test" matched cert's "example.test"
*  issuer: CN=example.test
*  SSL certificate verify ok.
* using HTTP/1.x
> GET / HTTP/1.1
> Host: example.test:8443
> User-Agent: example-client
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 74
< 
* Connection #0 to host example.test left intact
<!doctype html>
<html lang="en"><body><h1>Example Page</h1></body></html>

Three line types were removed from this output: certificate validity dates, the agreed cipher suite, and the handshake’s intermediate steps. Dates depend on when the certificate was produced and the cipher suite on the parties’ capabilities; both vary by machine. The lines above are what stays unchanged in this setup.

The server’s log is a single line:

[server] GET / Host=example.test:8443

The trace reads as follows: the name became an address (1–2), the connection was established (3), the certificate and name match were verified (4), the request was sent (5), the server answered by the Host header (6), the response was interpreted (7). No subresources were found, so step 8 stayed at a single document.

Symptoms of Failure

Splitting the chain into steps has a practical value: which step a problem is in can be inferred from its symptom. Every step fails in its own characteristic way.

Symptom	Faulty stage	Cause
Name does not resolve	2	No record, authoritative server unresponsive, or a negative answer is cached
Name points to an old address	2	The cached record’s lifetime has not expired
Connection refused	3	The address is correct, but no process is listening on that port
Connection timed out	3	Packets are not reaching the destination, or no response is coming back
Certificate could not be verified	4	The signature chain does not reach a trust anchor
Name on the certificate does not match	4	The certificate was issued for a different name
Unexpected site appeared	6	No site is defined for the corresponding `Host` header
`4xx` status code	6	The request is malformed: path, method, or authorization
`5xx` status code	6	The request was valid, the server could not fulfill it
Page arrives but unstyled	8	One of the subresource requests failed

Using this table means following the order: once a symptom appears, every stage before it succeeded. A certificate error means the name resolved and the connection was established; a 5xx means the secure channel and the request both worked. This narrowing method is the same reasoning as the layered diagnostic method from the Linux Network Administration and Troubleshooting course.

Two points here are most often confused. A refused connection and a timeout differ: a refusal shows the target is reachable and responded explicitly, a timeout means nothing came back at all. Name resolution problems and server problems sit at different layers: a name resolving shows only that the record exists, not that the server is running.

What Stays Constant in the Chain

Three principles repeat throughout the structure built during the course, and they are also the ground for the courses that follow.

Indirection. The name points to an address, the address to a location, the certificate to a key. Each layer stays fixed for the one above it while allowing the layer below it to change.

Layers not knowing about each other. The network does not know the content of the packet it carries; HTTP does not know whether encryption sits underneath it; TLS does not know that the data it carries is HTTP. This separation lets any one layer change without touching the others.

Hard guarantees are left to the endpoints. This principle, established in the What Is a Network lesson, repeats throughout the chain: order correction, cache consistency, authentication, and state management are all done at the endpoints. Intermediate nodes are kept simple.

Summary

A request is carried out in nine stages, and each stage hands off only a single thing to the next, forgetting what came before.
The host name comes back with the Host header after the address has been resolved; virtual hosting requires this.
The entire chain can be set up locally and observed in a single trace.
Fields in the trace such as certificate dates and cipher suite depend on the setup; structural lines do not change.
Every stage has its own characteristic failure symptom; once a symptom is observed, the stages before it are known to have succeeded.
Indirection, layers not knowing about each other, and leaving guarantees to the endpoints are three design principles that repeat throughout the chain.

Course Wrap-Up

This course built, end to end, the turning of a name typed into an address bar into a page on screen. Why networks are built on packet switching, why addresses are hierarchical, why names require a separate layer, how the domain name system scales through caching, why HTTP was designed to be stateless, and what three promises TLS gives — all of it was treated as parts of a single example request.

One question was deliberately left open. Correcting packet order, re-requesting lost packets, and adapting sending speed to what the network can carry were all passed over by saying “this is done at the endpoints”; the protocol that does this was never defined. Connections were spoken of too, but what a connection is, which layer it stands at, and when connectionless operation is preferred were not said.

The Network Models and Protocols course fills this gap. It defines layered models and places the distinctions used intuitively here — which job belongs to which layer — into a systematic framework, takes addressing beyond the prefix arithmetic mentioned here to build subnet design, and justifies the choice between reliable and unreliable transport through the application’s requirements. Every point left here by saying “that is the subject of a later course” is met there.

To keep your progress and take notes, Log in