The Role of HTTPS

The previous lesson said in passing that when the scheme part of the address is https, a security handshake takes place. This lesson opens up that step.

As seen in the HTTP Request and Response lesson, the request and response are plain text, readable and modifiable by anyone along the way. As established in the What Is a Network lesson, this text passes through numerous intermediate nodes, and, as stated in the Structure of the Internet lesson, these nodes are under the control of different organizations.

HTTPS is HTTP carried over a security layer called TLS. The protocol itself does not change; the channel it is carried over does.

Three Promises

TLS gives three separate promises. Separating them matters, because the common confusion comes from lumping all three under a single “encryption” heading.

Confidentiality. The content of the exchange cannot be read by anyone but the two ends. Nodes along the path carry encrypted bytes; they cannot see their meaning.

Integrity. If the content is modified along the way, that modification is detected. What is at stake is not preventing the modification but noticing it; corrupted data is not accepted silently.

Authentication. It is verified that the party being connected to really is that name.

Without the third promise, the first two are meaningless. A node in between can cut the connection, present itself as the server, and establish an encrypted channel with you; the content stays confidential and intact, but with the wrong party. Encryption without verified identity is whispering without knowing who you are talking to.

This is why the hard part is not encryption but authentication. Encryption is a mathematical problem; authentication is the question of how you decide who a party you have never met really is.

The Certificate

Authentication is done with a certificate. A certificate is a document that carries three things together: a name, a public key belonging to that name, and a signature confirming the pairing of the two.

The bond a certificate establishes is this: this public key belongs to this name. The server proves, during the handshake, that it holds the corresponding private key. This is how “I want to connect to this name” and “I am talking to the owner of this key” become joined.

A certificate can be generated locally. The following command produces a key pair and a certificate for the name example.test:

openssl req -x509 -newkey rsa:2048 -nodes \
  -keyout key.pem -out cert.pem -days 3650 \
  -subj "/CN=example.test" -addext "subjectAltName=DNS:example.test"

Examining the identity fields of the generated certificate:

openssl x509 -in cert.pem -noout -subject -issuer
openssl x509 -in cert.pem -noout -ext subjectAltName

subject=CN=example.test
issuer=CN=example.test
X509v3 Subject Alternative Name: 
    DNS:example.test

The two fields are identical. Subject states who the certificate belongs to, issuer states who confirmed it. The two being the same means the certificate confirmed itself; this is called a self-signed certificate.

A self-signed certificate has no proof value. This document says, “I am example.test, because I say so.” Anyone can produce the same document for any name.

Trust Anchors

The problem becomes: who decides that a certificate is correct?

The answer is to accept an unfamiliar party based on the endorsement of one you know. A certificate authority is an organization that signs a certificate with its own key after verifying that the name really belongs to the applicant.

The client has a pre-loaded set of directly trusted certificates; these are called trust anchors. Verification starts from the server’s certificate and tries to follow its signers to reach one of this set. If it reaches one, the certificate is accepted; if not, it is not.

This reduces trust to a chain, whose last link is something the client already had: trust is not produced from scratch, it is transferred.

A server can be run with this certificate. The server is the same as the one in the HTTP Request and Response lesson; the only difference is that the listening socket is wrapped with TLS:

import ssl
from http.server import BaseHTTPRequestHandler, HTTPServer


class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def log_message(self, fmt: str, *args: object) -> None:
        pass

    def do_GET(self) -> None:
        body = b"<h1>Example Page</h1>\n"
        self.send_response_only(200)
        self.send_header("Content-Type", "text/html; charset=utf-8")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)


context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain("cert.pem", "key.pem")
server = HTTPServer(("127.0.0.1", 8443), Handler)
server.socket = context.wrap_socket(server.socket, server_side=True)
server.serve_forever()

Connecting while the server is running, verification fails:

curl -sS --resolve example.test:8443:127.0.0.1 https://example.test:8443/

curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.se/docs/sslcerts.html

The request came back with nothing; the exit code is 60. The certificate is valid and the name is correct — but its signer is not among the client’s trusted parties. The message text varies by tool; what does not change is that verification fails and the connection is not established.

When the same certificate is given as a trust anchor, the connection is established:

curl -s --cacert cert.pem --resolve example.test:8443:127.0.0.1 https://example.test:8443/

<h1>Example Page</h1>

The server, certificate, and key are the same; only what the client trusts changed. This shows that trust is not a property of the certificate, it is a decision made by the client.

Name Verification

It is not enough for the certificate to be valid; the name it carries must also match the name being connected to. While the certificate above is considered trustworthy, connecting with a different name:

curl -sS --cacert cert.pem --resolve other.test:8443:127.0.0.1 https://other.test:8443/

curl: (60) SSL: no alternative certificate subject name matches target host name 'other.test'
More details here: https://curl.se/docs/sslcerts.html

The certificate carries a trusted signature, but it was issued for the name example.test. A client connecting to the name other.test does not accept it.

Without this check, any party holding a valid certificate for some name could stand in for every other name. A certificate binds a key to a specific name; the bond covers the name as well.

The Handshake

The handshake carries out two jobs together.

Agreeing on a key. The two parties generate a session-specific encryption key, derived from their exchange rather than sent over the network. Even someone who records the entire exchange cannot compute the key.

Proving identity. The server sends its certificate and shows, through an operation only performable with the corresponding private key, that it holds that key. Copying the certificate is not enough; identity cannot be proven without the private key.

Once these are complete, the HTTP exchange begins, in exactly the shape defined in the previous lessons. Encryption does not change HTTP, it wraps it.

Regenerating the session key for every connection has a consequence: even if the private key is later compromised, previously recorded exchanges cannot be decrypted. This property depends on the key agreement method chosen.

What HTTPS Does Not Hide

Knowing the boundaries of the promises given matters as much as knowing the promises themselves.

Name resolution is not hidden. Before the connection is established, the query from the Domain Name System lesson is made, and this query is outside TLS. Which name you want to connect to is known by the parties who see that query.

The destination address is not hidden. Packet headers cannot be encrypted; if they were, they could not be routed. Who you are talking to is seen by nodes along the path.

The size and timing of traffic are not hidden. How much data went and when it went is observable. This information does not give the content by itself, but it is useful for drawing inferences.

The correctness of the content is not guaranteed. TLS verifies that the other party is the name it claims to be; not that what it says is true. A verified site giving false information is not within the protocol’s concern.

This distinction has a practical consequence: a connection being encrypted does not mean the other party is trustworthy. What is verified is identity, not intent.

Summary

• HTTPS is HTTP carried over TLS; the protocol’s shape does not change, the channel does.
• TLS gives the promises of confidentiality, integrity, and authentication together; without authentication, the first two can be established with the wrong party.
• A certificate binds a public key to a specific name and confirms this bond with a signature.
• In a self-signed certificate the subject and issuer are the same and it has no proof value; verification tries to bring the signature chain to one of the trust anchors on the client.
• The certificate being valid is not enough; the name it carries is also checked against the name being connected to.
• HTTPS does not hide the name resolution query, the destination address, or the size and timing of traffic; nor does it give any promise about the correctness of the content.

Next Step

Every link of the chain is now established separately: packet switching, addressing, name resolution, hosting, the HTTP exchange, and the secure channel. The final lesson will join these together, end to end, over a single request, show what information each stage hands off to the next, and list the symptoms that appear when each link breaks.