Skip to content
academia.sh
tr
Log in

Lesson 10 / 12

What the Browser Does

The stages from parsing the text in the address bar to the drawing on screen, subresource requests, and the reuse of connections.

Contents

The previous lesson examined a single request and a single response. Yet when a page is opened, a single request is not made. The document that arrives refers to other resources, and the browser requests those too.

This lesson takes up the browser’s work in order: from parsing the text in the address bar to the drawing on screen. The browser is not a viewer, it is an orchestrator that carries out these stages.

Parsing the Address

The first job is to split the written text into its parts. An address is not a single string, it is a structured value.

from urllib.parse import urlsplit

address = "https://example.test:8443/archive/document?topic=network&page=2#section3"
parts = urlsplit(address)

print("scheme    :", parts.scheme)
print("host      :", parts.hostname)
print("port      :", parts.port)
print("path      :", parts.path)
print("query     :", parts.query)
print("fragment  :", parts.fragment)

scheme    : https
host      : example.test
port      : 8443
path      : /archive/document
query     : topic=network&page=2
fragment  : section3

Each part is used at a different stage:

Part Where it is used
Scheme Which protocol will be spoken; also sets the default port
Host The name given to the resolution in the Domain Name System lesson
Port When the connection is established; the scheme’s default if not written
Path Placed in the request line, sent to the server
Query Placed in the request line together with the path, sent to the server
Fragment Never sent to the server; used only in the browser

The last line is notable. The #section3 part never goes out on the network; the browser uses it itself, after receiving the document, to scroll to the section carrying that name. The server never sees this value.

It was stated in the What Is a Domain Name lesson that a name is case-insensitive. The same is not true of the path part: /Archive and /archive can be two different paths, and the server decides this.

Stages

Once the address is parsed, the browser follows these steps:

  1. Name resolution. The host name is turned into an address. The chain from the Domain Name System lesson runs here, and caches come into play.
  2. Establishing a connection. A connection is made to the target address and port.
  3. Security handshake. If the scheme is https, an encrypted channel is established; the subject of the next lesson.
  4. Sending the request. The request is written in the shape defined in the previous lesson.
  5. Receiving and interpreting the response. What the body is gets determined from the status code and the Content-Type header.
  6. Parsing the document. If the content is a document, its structure is extracted.
  7. Requesting subresources. Steps 1–5 repeat for every resource the document refers to.
  8. Layout and drawing. The position and appearance of elements on screen are computed and drawn.

Part of these steps overlap. The browser starts parsing before the whole document arrives, and sends the request as soon as it sees the first subresource reference.

Subresource Requests

The seventh step explains why an opened page cannot be reduced to a single request. The following server serves three resources; the root document refers to the other two.

from http.server import BaseHTTPRequestHandler, HTTPServer

PAGES: dict[str, tuple[str, bytes]] = {
    "/": ("text/html; charset=utf-8",
          b"<!doctype html>\n<html lang=\"en\">\n"
          b"<head><link rel=\"stylesheet\" href=\"/style.css\"></head>\n"
          b"<body><h1>Example Page</h1><img src=\"/logo.svg\" alt=\"\"></body>\n"
          b"</html>\n"),
    "/style.css": ("text/css; charset=utf-8", b"h1 { color: #333; }\n"),
    "/logo.svg": ("image/svg+xml",
                  b"<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"8\" height=\"8\"></svg>\n"),
}


class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def log_message(self, fmt: str, *args: object) -> None:
        print(f"{self.command} {self.path} -> {args[1]}", flush=True)

    def _respond(self, status: int, kind: str, body: bytes, send_body: bool) -> None:
        self.send_response_only(status)
        self.send_header("Content-Type", kind)
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        if send_body:
            self.wfile.write(body)

    def _dispatch(self, send_body: bool) -> None:
        if self.path in PAGES:
            kind, body = PAGES[self.path]
            self.log_request(200)
            self._respond(200, kind, body, send_body)
        else:
            body = b"<h1>404 Not Found</h1>\n"
            self.log_request(404)
            self._respond(404, "text/html; charset=utf-8", body, send_body)

    def do_GET(self) -> None:
        self._dispatch(send_body=True)

    def do_HEAD(self) -> None:
        self._dispatch(send_body=False)


HTTPServer(("127.0.0.1", 8080), Handler).serve_forever()

The three requests a browser would make, done explicitly with curl:

curl -sv --resolve example.test:8080:127.0.0.1 -A "example-client" -o /dev/null \
  http://example.test:8080/ http://example.test:8080/style.css http://example.test:8080/logo.svg

The server’s log shows three requests:

GET / -> 200
GET /style.css -> 200
GET /logo.svg -> 200

The user typed a single address; the server saw three requests. In real pages this number is far higher, and subresources can live on other domain names — in that case the name resolution and connection steps are carried out again for each new domain name.

The Reuse of Connections

Establishing a separate connection for every request is expensive. Establishing a connection requires at least one round-trip time; an encrypted connection requires more. As set up in the What Is a Network lesson, part of the latency cannot be physically reduced, so cutting the number of round trips is a direct gain.

Examining the connection behavior of the three requests above shows the situation:

*   Trying 127.0.0.1:8080...
* Connected to example.test (127.0.0.1) port 8080
> GET / HTTP/1.1
< HTTP/1.1 200 OK
* Connection #0 to host example.test left intact
* Re-using existing connection with host example.test
> GET /style.css HTTP/1.1
< HTTP/1.1 200 OK
* Connection #0 to host example.test left intact
* Re-using existing connection with host example.test
> GET /logo.svg HTTP/1.1
< HTTP/1.1 200 OK
* Connection #0 to host example.test left intact

A connection was made once, and three requests passed over the same connection. This is persistent connection behavior, and it has become the protocol’s default with HTTP/1.1; in the earlier version, a connection was made and closed for every request.

A problem persistent connections do not solve remains. Requests on the same connection are processed in order: the second cannot be sent before the first response completes. A slow response holds up the ones behind it. This behavior is called head-of-line blocking.

HTTP/2 answers this problem by letting several requests interleave over the same connection; this is called multiplexing. Requests and responses are split into numbered streams, the pieces are sent in mixed order and reassembled on the receiving end. This idea is the same reasoning as the packet numbering in the What Is a Network lesson.

Property Separate connection Persistent connection Multiplexing
Connection setup per request Every request First request First request
Requests pending at once As many as connections One Many
Effect of a slow response Only itself Holds up the ones behind it Does not hold up others

These three behaviors are the technical distinction between the protocol’s versions. Which one is used is decided by the client and server agreeing while establishing the connection.

From Document to Drawing

The document received is text; what appears on screen is elements whose layout and appearance have been computed. There are three jobs in between.

Parsing. The document text is turned into a tree of nested elements. The tree structure from the Data Structures course finds its direct counterpart here: each element is a node, and nesting is the parent–child relationship.

Style computation. Style rules are applied to elements and each element’s appearance properties are determined.

Layout and drawing. Each element’s position and size on screen are computed, then it is drawn.

The way subresources affect this order differs, and this difference produces an observable consequence. Drawing before the style file arrives leads to the page appearing unstyled and then changing; for this reason, drawing is done by waiting for the style file. Scripts, since they can modify the document, can halt parsing. Images, however, are not waited for; they are placed once they arrive.

This means it is not only the number of subresources but also their order that determines how long a page takes to appear.

The Browser Cache

So that fetched resources are not requested again, the browser keeps a cache. The lifetime logic from the Domain Name System lesson applies here too, but control is done with headers.

The server states how long a response can be stored. Once the duration expires, instead of downloading the resource again, the browser makes a conditional request: it sends the identity of the copy it has and says “send it if it changed.” If it has not changed, the server returns a short response with no body, and the body is not carried a second time.

This mechanism is the source of the difference between the first opening and later openings, and it rests on the same idea as the HEAD method’s justification from the previous lesson: getting information about content without carrying the content.

Summary

  • An address is a structured value; its scheme, host, port, path, query, and fragment parts are used at different stages, and the fragment part is never sent to the server.
  • The browser carries out name resolution, connection setup, sending the request, parsing, requesting subresources, and drawing; the steps proceed with overlap.
  • Writing a single address leads to many requests; if subresources are on different domain names, the resolution and connection steps run again.
  • A persistent connection spreads the cost of establishing a connection across requests; multiplexing removes the requirement that requests on the same connection wait on each other.
  • The document is turned into a tree of elements; drawing waits on style, scripts halt parsing, images are processed without being waited on.
  • The browser cache makes a conditional request for resources whose duration has expired, preventing the body from being carried again.

Next Step

This lesson separated out the scheme part of the address and, for the https case, said in passing that a security handshake takes place. That step cannot be skipped over: the exchange must be unreadable and unmodifiable to anyone on the path, and the other party must actually be who it claims to be. The next lesson defines these three promises one by one and shows, with a locally generated certificate, how each of them is tested.

To keep your progress and take notes, Log in

My notes

Log in to take notes.

Start typing to search.

↑↓ ↵ Esc navigate · open · close