Errors and Retries

On the network layer an error is not an exception but an everyday event. The connection drops, the server answers late, the response is lost on the way. A network client has to split these errors into two groups: those worth trying again and those that are not. A retry, done right, makes the app stable; done wrong, it places the same order twice.

In this lesson the error classification is a pure function and is measured. The real question lies where the classification stops: what does retrying a request do on the server? The core is the same.

// Replay.swift — classifies an operation's sensitivity to repetition and order
struct Trial<State: Equatable & CustomStringConvertible> {
    let label: String
    let once: State
    let twice: State
    let forward: State
    let backward: State

    var sensitivity: String {
        switch (once != twice, forward != backward) {
        case (false, false): return "robust"
        case (true, false): return "repeat-sensitive"
        case (false, true): return "order-sensitive"
        case (true, true): return "doubly-sensitive"
        }
    }

    func report() -> String {
        print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)")
        return sensitivity
    }
}

func replay<State>(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial<State> {
    Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start)))
}

func countSensitivities(_ results: [String]) {
    for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] {
        print("\(name): \(results.filter { $0 == name }.count)")
    }
}

Error Classification

Four errors are told apart: timeout, no connection, server error, and client error. The first two and the 5xx server errors are retryable; 4xx client errors are not, because the same request gets the same error. The number of attempts is limited by a retry budget, and the budget is an integer. The wait, or the exponential backoff time, is not modeled or measured in this lesson.

// Orders.swift — failure class, order ledger, and retry loop
enum Failure: Equatable {
    case timeout
    case offline
    case server(Int)
    case client(Int)
}

func isRetryable(_ failure: Failure) -> Bool {
    switch failure {
    case .timeout, .offline: return true
    case .server(let code): return code >= 500
    case .client: return false
    }
}

struct Ledger: Equatable, CustomStringConvertible {
    var orders: [String] = []
    var description: String { "[" + orders.joined(separator: ",") + "]" }
}

func place(_ item: String, key: String?) -> (Ledger) -> Ledger {
    { ledger in
        let tag = "\(item)@\(key ?? "-")"
        if key != nil && ledger.orders.contains(tag) { return ledger }
        return Ledger(orders: ledger.orders + [tag])
    }
}

func simulate(budget: Int, key: String?, lostResponses: Int) -> (attempts: Int, ledger: Ledger) {
    var ledger = Ledger()
    var attempts = 0
    while attempts < budget {
        attempts += 1
        ledger = place("book", key: key)(ledger)
        if attempts > lostResponses { break }
    }
    return (attempts, ledger)
}

Failure is an enumeration with four cases. isRetryable looks only at the kind of the error. place builds the server recording the order: if a key is given and a record with the same tag exists, it does not add a second one. simulate builds the client’s loop. The request reaches the server on every attempt and is recorded; the first lostResponses responses, however, are lost on the way. The client sees a lost response as a timeout and retries as long as the budget allows. The model builds what the operation does to the state; it is not the network itself.

// main.swift — classification, retry, two write forms
print("timeout: \(isRetryable(.timeout))")
print("offline: \(isRetryable(.offline))")
print("server-503: \(isRetryable(.server(503)))")
print("client-404: \(isRetryable(.client(404)))")

let keyless = simulate(budget: 3, key: nil, lostResponses: 1)
let keyed = simulate(budget: 3, key: "k-9", lostResponses: 1)
print("keyless: attempts=\(keyless.attempts) ledger=\(keyless.ledger)")
print("keyed: attempts=\(keyed.attempts) ledger=\(keyed.ledger)")

let withoutKey = replay("keyless-order", from: Ledger(), place("book", key: nil), then: place("book", key: nil)).report()
let withKey = replay("keyed-order", from: Ledger(), place("book", key: "k-9"), then: place("book", key: "k-9")).report()
let spending = replay("spend-budget", from: 3, { max($0 - 1, 0) }, then: { max($0 - 1, 0) }).report()

countSensitivities([withoutKey, withKey, spending])

timeout: true
offline: true
server-503: true
client-404: false
keyless: attempts=2 ledger=[book@-,book@-]
keyed: attempts=2 ledger=[book@k-9]
keyless-order | once=[book@-] twice=[book@-,book@-] ab=[book@-,book@-] ba=[book@-,book@-] -> repeat-sensitive
keyed-order | once=[book@k-9] twice=[book@k-9] ab=[book@k-9] ba=[book@k-9] -> robust
spend-budget | once=2 twice=1 ab=1 ba=1 -> repeat-sensitive
robust: 1
repeat-sensitive: 2
order-sensitive: 0
doubly-sensitive: 0

Three of the four errors are retryable; the client error is not. But the second part is the one that stands out. The same lost response gave two different results in the same loop: the keyless request left two records after two attempts, and the keyed request left one record after the same two attempts. What the client sees is the same in both: a timeout followed by a successful response.

keyless-order is repeat-sensitive: the second application adds a second record. keyed-order is robust: the same key prevents the second record. spend-budget is repeat-sensitive; but that is the budget’s job, not an error. The budget exists precisely so that it shrinks as it is repeated. The rule does not say which repetition is an error and which is design.

NW7. The model assumes the server recognizes the key. A real server may ignore the key header, and then the keyed request creates a second record too.

NW8. The model always builds a lost response as a request that reached the server. A request that never arrives is examined separately.

Reading the Error on the Client

In a real client the error is read from the code of a URLError.

import Foundation

func classify(_ error: Error) -> Failure {
    guard let urlError = error as? URLError else { return .client(0) }
    switch urlError.code {
    case .timedOut: return .timeout
    case .notConnectedToInternet: return .offline
    default: return .client(0)
    }
}

This snippet stays SOURCE. URLError carries network errors with a code; .timedOut and .notConnectedToInternet are two of them. The snippet maps only two codes; a real classification covers more. The name of the key header depends on the server; it is usually sent as a request header.

The Cost of Retrying

A retry looks like insurance, but it is not cheap. Every attempt puts one more request on the server, lengthens the user’s wait, and spends battery. If the server is already under load, thousands of clients retrying at the same moment do not reduce the load but increase it. So two limits are used together: the retry budget cuts the total number of attempts, while exponential backoff doubles the wait between attempts each time. The budget limits how many times we try; the backoff limits how often.

A small random share is usually added to the backoff. Otherwise a thousand clients that failed at the same moment retry at the same moment and at the same intervals, and hit the server in waves. The random share spreads those waves out. This lesson does not model the wait, so it does not measure this; but in a real client the budget and the backoff must be treated as two separate decisions.

That the budget comes out repeat-sensitive in the measurement is not a flaw. In the spend-budget line once=2 and twice=1 differ, because the budget exists precisely so that it shrinks with every expenditure. The rule only reports repeat sensitivity; it does not tell whether that is wanted or unwanted. The person reading the measurement makes that distinction.

Where It Shows Up in a Real App

In a shopping app the user taps the “Confirm Order” button. The network is slow, the request reaches the server and the order is recorded, but the response is lost on the way. The client sees a timeout and sends the request again. Without a key, the server counts the second request as a brand new order and the user is charged twice. The ledger=[book@-,book@-] in the keyless line is exactly this situation.

The user mostly experiences this as a separate error: “Why did my order come twice?” The support team sees two records, both look valid, and which one to delete is unclear. The problem is hard to reproduce, because it appears only in the rare case where the response is lost. The developer never sees it in testing, because the network always works there. In production it happens a few times a day among millions of requests.

The same situation is not limited to payments. Any operation that creates a new record on the server, such as sending a message, adding a comment, or putting an item in a cart, carries the same risk. Read operations, on the other hand, are usually retried safely, because they do not change the state on the server.

The Wrong Setup and Its Fix

The common wrong setup is to retry on every error and without limit.

import Foundation

// Wrong: every error is retried, no budget, no wait
func sendWithRetry(_ request: URLRequest, using session: URLSession) async throws -> Data {
    while true {
        do {
            let (data, _) = try await session.data(for: request)
            return data
        } catch {
            continue
        }
    }
}

This snippet stays SOURCE. It has three problems: even a 404 is retried forever, when the server goes down the client sends hundreds of requests a second, and the user cannot understand why the screen never moves on. The fix brings the classification, the budget, and the wait together.

import Foundation

// Fix: only retryable errors, a limited budget, a doubling wait
func sendWithRetry(_ request: URLRequest, using session: URLSession, budget: Int = 3) async throws -> Data {
    var delay: UInt64 = 500_000_000
    for attempt in 1...budget {
        do {
            let (data, response) = try await session.data(for: request)
            if let http = response as? HTTPURLResponse, http.statusCode >= 500, attempt < budget {
                try await Task.sleep(nanoseconds: delay)
                delay *= 2
                continue
            }
            return data
        } catch {
            guard isRetryable(classify(error)), attempt < budget else { throw error }
            try await Task.sleep(nanoseconds: delay)
            delay *= 2
        }
    }
    throw URLError(.unknown)
}

This also stays SOURCE. The function waits only on 5xx responses and retryable errors. A 4xx response returns without waiting; the caller reads it as its own error. The wait doubles with every attempt. When the budget runs out, the last error is thrown to the caller, so nobody keeps waiting without noticing that the error was swallowed. The corrected function does not make retrying safe; it only limits it. For the request to be robust against repetition, an idempotency key is also needed, and that is what the keyed-order line shows.

Carrying the Error to the User

When the budget is spent, the error must not disappear silently. The user is shown two things: what happened and what they can do. If there is no connection, “You have no internet connection” and an offer to retry automatically when the connection returns make sense. For a server error, “We cannot reach the service right now, try again shortly” is enough. For a client error, retrying has no benefit; the user is told to correct what they entered.

When a keyless write request ends with a timeout, the situation is more subtle still. The client does not know whether the request reached the server; this is the uncertainty the bounding measurement shows. The right behavior is to tell the user “The result is uncertain” and not to send a second write without querying the state on the server. If there is a key, the uncertainty disappears, because the second request can be repeated harmlessly with the same key.

Separating Read and Write Requests

The first input to the retry decision is not the error but the kind of the request. A read request does not change the state on the server; even if its response is lost, sending it a second time is safe. A write request changes the state, and a second send means the state changes twice. The protocol carries this distinction to some extent itself: read requests are by definition considered repeatable, requests that create a new record are not. Update and delete requests are also mostly designed to be repeatable, because applying the same update a second time does not change the result.

In practice this means two separate paths in the client. The read path classifies the error, waits, and retries as long as the budget allows. The write path first looks at whether the request carries an idempotency key. If there is a key, it behaves like the read path. If there is no key, the retry decision is left to the user or made by querying the state on the server. The keyless-order and keyed-order lines in the measurement show the difference between these two paths through their results.

What Spends the Budget

Not every error takes the same share of the budget. With no connection the request never leaves; counting that as an attempt wastes the budget. A better setup is to wait until the connection returns and to spend the budget only on requests that are really sent. With server errors, every send puts load on the server, and the budget limits exactly this load. A timeout is in between: the request has left, but its result is unknown.

The size of the budget also depends on the nature of the error. For a brief network outage, three attempts are usually enough. When the server is unreachable for a long time, dozens of attempts only create load; then it is more correct for the client to give up and inform the user.

Common Mistakes

• Retrying every error. A 404 response stays a 404 however many times it is repeated.
• Not putting an upper limit on retries. A loop without a budget wears the server out further.
• Retrying a keyless POST after a timeout.
• Reading a timeout as “the request failed”. A timeout says only that the response did not arrive.
• Trying back to back without a wait. This lesson does not measure the wait, but a real client needs it.

Interpreting the Measured Result

Retrying looks like a classification question but it is a sensitivity question. Classification gives one of two results: try or do not try. Sensitivity asks what the result of the trying will be. The keyless order created a second record; the keyed order did not. The error classification was the same in both.

The Bounding Measurement

When the client sees an error, it cannot know which of two worlds it is in: the request reached the server but the response was lost, or the request never arrived. The measurement below builds the two worlds and prints what the client sees next to the ledger on the server.

// main.swift — two worlds, the same observation
func world(requestArrived: Bool) -> (observed: Failure, ledger: Ledger) {
    let ledger = requestArrived ? place("book", key: nil)(Ledger()) : Ledger()
    return (.timeout, ledger)
}

let lost = world(requestArrived: true)
let never = world(requestArrived: false)
print("client observation equal: \(lost.observed == never.observed)")
print("server ledger equal: \(lost.ledger == never.ledger)")

client observation equal: true
server ledger equal: false

What the client sees is the same in the two worlds; the ledger on the server is different. So the client cannot decide from its observation whether to retry; the decision is given by the request itself, that is, by the key. The rule neither resolves nor measures this uncertainty.

NW9. The measurement builds the two worlds as indistinguishable from the client’s side. A real client can know in some errors (such as a connection that could not be established at all) that the request did not arrive.

Summary

• Network errors are classified as retryable and non-retryable with a pure function; the number of attempts is limited by a budget.
• Retrying a keyless request after a timeout created a second record on the server.
• Under the same retry, the request with an idempotency key left a single record.
• Spending the budget is repeat-sensitive, but that is design, not an error.
• The client cannot tell a lost response from a request that never arrived by its observation.
• This lesson’s sensitivity: repeat-sensitive.

Next Step

A retried request sometimes carries an identity as well. In the next lesson, Authentication will measure how storing the responses of two concurrent token refreshes in a different order can bring back the old token.