--- title: 'Data, Networking and Concurrency' source: 'https://academia.sh/en/courses/data-networking-and-concurrency' language: en updated: '2026-10-21T05:03:08+00:00' license: 'CC BY-SA 4.0' lessons: 19 outcomes: - 'Builds network requests asynchronously and with error handling' - 'Chooses local persistence options according to the data model' - 'Uses the concurrency model while avoiding race conditions' - 'Distinguishes whether an operation is robust, repeat-sensitive, order-sensitive or doubly sensitive' - 'Builds network and disk operations with a small model and recognizes that the model is not the network or the disk itself' --- # Data, Networking and Concurrency A network request is retried, the user taps a button twice, two tasks update the same state in different orders, sync brings back what two devices wrote in a different order. The course measures every mechanism with a single question — if this operation is repeated or its order changes, does the result change? Is the operation robust, repeat-sensitive, order-sensitive, or both? The network client and JSON conversion, errors and retries, authentication and caching, asynchronous functions, tasks and actors, main thread isolation, queues and reactive streams, preference, keychain, file and database storage, cloud sync and push notifications all pass through the same frame. ## The Networking Layer Session configuration and task types, type-safe JSON conversion, classifying network errors and retrying, token management, memory and disk caching, and transfers while the app is in the background. ### The Network Client Source: https://academia.sh/en/courses/data-networking-and-concurrency/network-client Session configuration and the task types; measuring, in four categories, whether the result of the six operations behind the network layer changes if they are repeated or reordered. In the previous course (M30/K04) we compared the value a screen shows with the model's value and checked whether the screen had gone stale. This course goes beneath the screen: to the layer where the value arrives, is stored, and is processed. In this layer the question changes. A screen shows the value or it does not. An operation in the network layer is tested by something else: if the same operation is applied twice, or if two operations swap order, does the result come out different? It is no accident that this question is asked so often on the network. A request can be lost on the way, a response can arrive late, a user can press the same button twice, two responses can arrive in reverse order. None of these raises an error. The state quietly turns into a different value. In this lesson we first get to know the building blocks of a network client: session, configuration, and task. Then we measure six operations with a single core. The measurement axis is **sensitivity**. We apply an operation once and twice; if the results differ, the operation is **repeat-sensitive**. We apply two operations in two different orders; if the results differ, the operation is **order-sensitive**. If both hold it is **doubly sensitive**, and if neither holds it is **robust**. In the output the four names are written in ASCII: `robust`, `repeat-sensitive`, `order-sensitive`, `doubly-sensitive`. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` `Trial` holds four states: `once` is the state after the operation is applied once, `twice` after it is applied twice. `forward` comes from applying `a` first and then `b`; `backward` from applying `b` first and then `a`. `sensitivity` looks only at the two inequalities. `replay` produces these four states from a starting state. **The rule does not see the cause, it sees the inequality:** it does not say why two states differ, only that they do. The lesson says which state is being compared. #### Session and Configuration A network request needs three separate objects. The **configuration** says how the session will behave: the timeout, the cache, extra headers, whether to wait when there is no network. The **session** is the client that works with this configuration; requests that come out of the same session share the same rules. The **task** is a single transfer: it takes a request and returns a response or an error. `URLSessionConfiguration.default` uses a persistent cache and cookie store. `.ephemeral` writes none of these to disk; no trace remains when the session ends. The third configuration is the background session, which is covered in the last lesson of this topic. ```swift import Foundation let configuration = URLSessionConfiguration.ephemeral configuration.timeoutIntervalForRequest = 15 configuration.waitsForConnectivity = true configuration.httpAdditionalHeaders = ["Accept": "application/json"] let session = URLSession(configuration: configuration) ``` The configuration is read when the session is created. Changing the configuration after the session exists does not affect the session; to change anything, a new session is created. For this reason applications usually create a small number of sessions and share them everywhere. #### Task Types Three task types divide the work. A **data task** takes the response into memory; it suits small responses. An **upload task** sends the body of the request as separate data or as a file. A **download task** writes the response directly to a file; it is used so that large bodies do not bloat memory. In the asynchronous form, the first two are the `data(for:)` and `upload(for:from:)` calls; both return a `(Data, URLResponse)` pair. ```swift // profile request — SOURCE import Foundation func fetchProfile(session: URLSession) async throws -> Data { let request = URLRequest(url: URL(string: "https://api.example.com/profile")!) let (data, response) = try await session.data(for: request) guard let http = response as? HTTPURLResponse, http.statusCode == 200 else { throw URLError(.badServerResponse) } return data } ``` This snippet is not run; it reaches out to the network. `data(for:)` is an **asynchronous function**: the call waits at a **suspension point** until the response arrives, and the flow then continues from where it left off. Whether the response is a real response is checked separately; the status code is read through `HTTPURLResponse`. ```swift import Foundation func send(order: Data, session: URLSession) async throws -> Int { var request = URLRequest(url: URL(string: "https://api.example.com/orders")!) request.httpMethod = "POST" let (_, response) = try await session.upload(for: request, from: order) return (response as? HTTPURLResponse)?.statusCode ?? 0 } ``` In an upload task the body is not placed inside the request but passed in the `from:` parameter. This snippet also stays SOURCE. What happens when a `POST` request that creates a new order is repeated is one of this lesson's six operations. #### Six Operations The **model** below builds what each operation does to the state; it is not the network or the disk itself. Each of the six operations corresponds to a familiar job in the network layer: writing to a cache, a send with an idempotency key, an order that is retried, incrementing a badge count, writing a preference, appending a line to a journal. With the core's `replay` function each one is applied once, twice, and in two orders. ```swift // main.swift — six operations, four sensitivities func cached(_ key: String) -> ([String]) -> [String] { { keys in Array(Set(keys + [key])).sorted() } } func submit(_ key: String) -> ([String]) -> [String] { { received in received.contains(key) ? received : received + [key] } } func placeOrder(_ count: Int) -> Int { count + 1 } func badge(_ count: Int) -> Int { count + 1 } func theme(_ value: String) -> (String) -> String { { _ in value } } func log(_ line: String) -> ([String]) -> [String] { { lines in lines + [line] } } let cacheWrite = replay("cache-write", from: [String](), cached("profile"), then: cached("feed")).report() let keyedSend = replay("keyed-send", from: [String](), submit("k-41"), then: submit("k-42")).report() let retry = replay("retry", from: 0, placeOrder, then: placeOrder).report() let push = replay("badge-increment", from: 3, badge, then: badge).report() let preference = replay("preference-write", from: "system", theme("dark"), then: theme("light")).report() let journal = replay("journal-append", from: [String](), log("sign-in"), then: log("sign-out")).report() countSensitivities([cacheWrite, keyedSend, retry, push, preference, journal]) ``` ``` cache-write | once=["profile"] twice=["profile"] ab=["feed", "profile"] ba=["feed", "profile"] -> robust keyed-send | once=["k-41"] twice=["k-41"] ab=["k-41", "k-42"] ba=["k-42", "k-41"] -> order-sensitive retry | once=1 twice=2 ab=2 ba=2 -> repeat-sensitive badge-increment | once=4 twice=5 ab=5 ba=5 -> repeat-sensitive preference-write | once=dark twice=dark ab=light ba=dark -> order-sensitive journal-append | once=["sign-in"] twice=["sign-in", "sign-in"] ab=["sign-in", "sign-out"] ba=["sign-out", "sign-in"] -> doubly-sensitive robust: 1 repeat-sensitive: 2 order-sensitive: 2 doubly-sensitive: 1 ``` `cache-write` is robust: writing the same key twice does not change the key set, and the order of two keys does not change the result, because the set is printed sorted. `keyed-send` is order-sensitive: the idempotency key makes the second arrival of the same request ignored, so `once` and `twice` are the same; but `k-41` and `k-42` are different requests, and the arrival order determines the order of the list. `retry` is repeat-sensitive: the counter grows with every application. `badge-increment` is also repeat-sensitive, for the same reason. `preference-write` is order-sensitive: the last writer wins. `journal-append` is sensitive to both: the line is added twice and the order of the two lines changes. **NW1.** This measurement builds each of the six operations from a single starting state. From a different starting state an operation's class can change. **NW2.** `cache-write` keeps the keys as a sorted set. A cache that preserves insertion order must be measured separately. The distribution is not a finding. One robust, two repeat-sensitive, two order-sensitive, and one doubly sensitive operation came out because we chose the operations. Six other operations would have produced another distribution. #### What Idempotency Solves `keyed-send` is the most instructive line of this lesson, because it shows that what solves repetition does not solve order. In HTTP, methods for which sending the same request one after another does not change the result are called **idempotent**; `GET` and `PUT` are such methods, `POST` is not. If the server accepts an **idempotency key** that gives a request an identity, a `POST` that creates a new order can also be idempotent: a second request arriving with the same key is ignored. In the measurement this came out exactly so. `once` and `twice` are the same: repetition was solved. But when there are two different keys, the order still determines the result: `ab` and `ba` are not the same. **Idempotency solves repetition, not order.** Solving order takes another tool: a version number, a mergeable data structure, or ordering on the server. In later lessons we will measure some of these. #### Where It Shows Up in a Real App The measured order sensitivity is not an artificial laboratory case. A screen sends a profile request as it opens; when the user pulls down to refresh, a second request goes out. If the two responses return from the network in reverse order, the last one to arrive is written to the screen, and it may be the response to the older request. The result is that a change the user made a moment ago appears to be silently undone on screen. There is no trace of this in the error log: no request failed, and each response is correct on its own. Retry, on the other hand, is the typical example of repeat sensitivity. When a request that sends an order times out, the client sees that no response arrived; but the server may have received the request and recorded the order. If the client sends the same request again, a second order is created on the server. The `twice` column of the measurement shows exactly this situation: in the `retry` line the number has grown once more when the operation is repeated. The starting-state issue is more insidious still. The same operation can behave differently in an app that opens with an empty cache and in an app that opens with a cache left full from the previous session. The bounding measurement below shows this: the same pair of operations falls into a different class depending on the starting state. For this reason, whether a request is idempotent cannot be told from the request alone; it is also asked which state the operation runs from. #### The Wrong Setup and Its Fix The most common wrong setup is to apply retry to every failed request without looking at the kind of request. Reading the profile with `GET` breaks nothing when repeated; but if the same rule is applied to a `POST` request that creates an order, every timeout carries the risk of a second order. The fix has two steps. The first is to attach an idempotency key to the repeat-sensitive request: the client generates the key on the first attempt and sends the same one on every retry, and the server ignores a second request that arrives with the same key. The `once` and `twice` columns coming out the same in the `keyed-send` line of the measurement is the counterpart of this. The second is that, for order-sensitive responses, the screen accepts only the response to the most recently sent request; a late-arriving response to an older request is ignored. This does not remove the order problem, but it makes the order visible in a single place; the way an order-sensitive operation becomes safe is, most of the time, that the order can no longer change. For the badge the fix is different. Writing the absolute value the server reports, instead of incrementing the count locally, protects the operation against repetition: even if the same value is written a second time, the result does not change. The absolute value has its own question too: if two values arrive in reverse order, the older one stays. We did not measure this separately in this lesson; we only state its direction. #### Reading the SOURCE Snippets The two SOURCE snippets in the lesson are not run, because both reach out to the network and their results depend on the server. The snippets are still worth reading. `fetchProfile` is a read: sending the same request twice does not change the state on the server, and its counterpart in the model is the `cache-write` line. `send(order:session:)` is a `POST` request that creates a new record, and it corresponds to the `placeOrder` function in the model. The difference between the model and the real thing is that in a real request the repetition can also happen without the client's knowledge: when the connection drops, the client cannot know whether the request reached the server. This difference does not change the measured class; it only says where the repetition can come from. The status code check in the snippets matters too. `fetchProfile` accepts only 200; `send(order:session:)` returns the status code as it is and leaves the decision to the caller. On a timeout no response arrives at all, and the client does not know whether the request was processed. Treating this situation as the same error as a request the server explicitly rejected makes the retry send a repeat-sensitive request a second time. #### Common Mistakes - Creating a new session for every request. The configuration and the connections are not shared. - Not checking whether the response is an `HTTPURLResponse` and not checking the status code. The call does not throw even when the server returns an error code. - Taking a large body into memory with a data task. A download task writes to a file. - Ignoring a repeated user action in the network layer. Pressing the button twice produces two requests; the repetition question is answered separately on the client and on the server. - Using the idempotency key on the assumption that it also solves the order problem. #### Interpreting the Measured Result All six operations were jobs that can show up in a network layer, and they spread across four categories. What matters is not the distribution but that an operation's class comes from two questions: what happens if it is repeated, and what happens if its order changes. The answers to these two questions vary from one operation to another and often cannot be inferred from the operation's name. Although `placeOrder` and `badge` have the same body, one is an order and the other is a counter; for the user their consequences are very different, for the rule they are the same. In the network layer this distinction is useful in three places. In retry: a repeat-sensitive operation produces a second record when retried. In response order: an order-sensitive operation leaves a stale value if two responses arrive reversed. In the cache: a robust operation can be repeated without the risk of error. The next five lessons measure these three places in turn. #### The Bounding Measurement An operation's class depends not only on the operation but also on the starting state. Below, the same two operations, `x * 2` and `x * 3`, are run from two starting points. ```swift // main.swift — the same two operations, two starting points let doubled = replay("from-zero", from: 0, { $0 * 2 }, then: { $0 * 3 }).report() let fromOne = replay("from-one", from: 1, { $0 * 2 }, then: { $0 * 3 }).report() countSensitivities([doubled, fromOne]) ``` ``` from-zero | once=0 twice=0 ab=0 ba=0 -> robust from-one | once=2 twice=4 ab=6 ba=6 -> repeat-sensitive robust: 1 repeat-sensitive: 1 order-sensitive: 0 doubly-sensitive: 0 ``` Starting from zero, both operations leave zero as zero and the line is robust. Starting from one, the second application changes the value and the line is repeat-sensitive. In both starting states the two orders give the same result for this pair of operations, because the multiplications commute. The same operation fell into another class when the starting state changed. **NW3.** This measurement compares only the starting points zero and one. Other starting points were not measured, and the class may change at them too. #### Summary - A network client is built from three parts: the configuration states the behavior, the session works with that behavior, and the task carries out a single transfer. - A data task takes the response into memory, an upload task sends the body separately, and a download task writes the response to a file. - Sensitivity is measured with two questions: if it is repeated, and if its order changes, does the result change. - The six operations fell into four categories: one robust, two repeat-sensitive, two order-sensitive, one doubly sensitive. - Idempotency solves repetition, not order; the same operation can fall into another class when the starting state changes. - This lesson's sensitivity: **all four**. #### Next Step The response that comes back from the network is a byte sequence; what the application needs is a typed record. In the next lesson, JSON Encoding and Decoding will measure whether turning a response into a record can itself be order-sensitive: the order in which a two-version record is written determines the value in the store. ### JSON Encoding and Decoding Source: https://academia.sh/en/courses/data-networking-and-concurrency/json-encoding-and-decoding Turning a network response into a typed record and writing the record to a store; measuring how the order in which a two-version record is written changes the result. In the previous lesson we measured the three parts of a network client and the four sensitivities of six operations. What comes back from the network, however, is a byte sequence. The app turns this sequence into a typed record with **encoding and decoding**. The conversion itself looks safe: if the type does not fit, decoding raises an error, and if it fits, the record is ready. The trouble starts after decoding. The decoded record is written to a store, and if two responses carry two versions of the same record, which one is written last determines the value in the store. The core is the same. As in every lesson, the first measured Swift block is `Replay.swift`. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` Three operations will be measured in this lesson: appending the record to a list, writing the record to the store by its identifier, and encoding and decoding the record. Each works on a record type and a small shelf. #### Record and Shelf `Record` is both `Codable` and carries a short `description` so that it can be read in the measurement output. `Shelf` holds the state of the records; `description` writes the records sorted by identifier, because dictionary order is not known. ```swift // Records.swift — record, shelf, and three write forms import Foundation struct Record: Codable, Equatable, CustomStringConvertible { let id: Int let name: String let version: Int var description: String { "\(id):\(name):v\(version)" } } struct Shelf: Equatable, CustomStringConvertible { var records: [Record] = [] var description: String { records.sorted { $0.id < $1.id }.map(\.description).joined(separator: ",") } } func decodeRecord(_ text: String) -> Record { try! JSONDecoder().decode(Record.self, from: Data(text.utf8)) } func append(_ record: Record) -> (Shelf) -> Shelf { { shelf in Shelf(records: shelf.records + [record]) } } func upsert(_ record: Record) -> (Shelf) -> Shelf { { shelf in var next = shelf.records if let index = next.firstIndex(where: { $0.id == record.id }) { next[index] = record } else { next.append(record) } return Shelf(records: next) } } func roundTrip(_ record: Record) -> Record { let encoder = JSONEncoder() encoder.outputFormatting = [.sortedKeys] let data = try! encoder.encode(record) return try! JSONDecoder().decode(Record.self, from: data) } func rename(_ name: String) -> (Record) -> Record { { record in Record(id: record.id, name: name, version: record.version) } } ``` We test the three operations with the text of two responses. `first` and `second` are two versions carrying the same identifier. ```swift // main.swift — two versions, three write forms let first = decodeRecord(#"{"id":7,"name":"notebook","version":1}"#) let second = decodeRecord(#"{"id":7,"name":"notebook-new","version":2}"#) let listed = replay("append-to-list", from: Shelf(), append(first), then: append(second)).report() let stored = replay("write-by-id", from: Shelf(), upsert(first), then: upsert(second)).report() let travel = replay("round-trip", from: first, roundTrip, then: rename("notebook-new")).report() countSensitivities([listed, stored, travel]) ``` ``` append-to-list | once=7:notebook:v1 twice=7:notebook:v1,7:notebook:v1 ab=7:notebook:v1,7:notebook-new:v2 ba=7:notebook-new:v2,7:notebook:v1 -> doubly-sensitive write-by-id | once=7:notebook:v1 twice=7:notebook:v1 ab=7:notebook-new:v2 ba=7:notebook:v1 -> order-sensitive round-trip | once=7:notebook:v1 twice=7:notebook:v1 ab=7:notebook-new:v1 ba=7:notebook-new:v1 -> robust robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 1 ``` `append-to-list` is doubly sensitive. Appending the same response twice duplicates the record, and the order of the two versions changes the order of the list. `write-by-id` is robust against repetition: `once` and `twice` are the same, because a record with the same identifier is replaced by the new one, not appended. But it stays order-sensitive: `ab` and `ba` differ, because the last version written wins. `round-trip` is robust; when the record is encoded and decoded the same record comes back, and even if the order is changed by renaming, the result is the same. **NW4.** The measurement assumes the identifier of the two versions is the same. Records with different identifiers are written to separate rows, and the order becomes invisible only in the sorted display. **NW5.** `rename` changes only the name and does not touch the version number. An operation that also changes the version must be measured separately. #### The Response from the Network In a real app these two texts are the bodies of two separate responses. The client receives the response as a byte sequence and decodes it. ```swift import Foundation func loadRecord(session: URLSession, url: URL) async throws -> Record { let (data, _) = try await session.data(from: url) let decoder = JSONDecoder() decoder.keyDecodingStrategy = .convertFromSnakeCase return try decoder.decode(Record.self, from: data) } ``` This snippet stays SOURCE; it reaches out to the network. The model above builds what the write operation does to the state; it is not the network itself. `convertFromSnakeCase` maps the server's keys such as `created_at` to field names such as `createdAt`. Decoding can throw an error; for this reason the call is made with `try`, and the caller either propagates or catches the error. The client cannot choose in which order the two responses will arrive. If two requests leave at the same time, the one sent second can come back first. So the order sensitivity in the `write-by-id` line is not a theory but a normal behavior of the network. #### Why Encoding Is Safe The `round-trip` line came out robust because encoding and decoding are written to be each other's inverse. `Record` carries all three fields with fixed types; the encoder writes each field to a text under its name, and the decoder reads it back under the same names. There is no decision in between: which field goes where comes from the type, not from the order of operations or the number of times they run. So encoding and decoding the same record twice gives the same record, and whether the round trip or the renaming comes first does not affect the result. This safety has two conditions. The first is that the encoded value itself must be lossless. If a `Double` field carries a very long decimal fraction, its last digits can change when it is written to text and read back; the integer and string fields of `Record` do not carry this danger. The second is that the names the decoder expects must be the same as the names the encoder writes. If the server writes `created_at` and the client expects `createdAt`, decoding throws an error on the first line. The measurement treats both as met; so its coming out robust does not say the conversion is flawless, it says the measured record is one for which the conversion is easy. #### Where It Shows Up in a Real App A notes app receives every record from the server as JSON. The user opens a note, the app decodes the note and updates the list on the screen. The same note arrives once more when the user refreshes the screen. The code that appends to the list takes this second response for a new note, and the same note appears as two rows on the screen. The `append-to-list` line in the measurement is exactly this: in the `twice` column the same record stands twice. The user sees this as a visual flaw, and the developer mostly starts looking for something in the network layer. Yet the network layer has worked correctly; both responses are valid and were decoded correctly. The error is in the way the decoded record is written to the store. Decoding being robust and writing being sensitive is the dividing line of this lesson: the place to look for the problem is not the two sides of the conversion but the operation performed with the conversion's output. A second case is order. The user edits a note twice in quick succession, and each edit sends a separate save request. The server processes the two requests, and then the app reads the note again with two responses. The arrival order of the responses does not have to match the sending order. If the old version comes later, the old text stays on the screen and the user thinks the text they wrote has been lost. The difference between the `ab` and `ba` columns of the `write-by-id` line is exactly this. #### The Wrong Setup and Its Fix The common wrong setup is to write every response to the store as soon as it arrives. ```swift import Foundation // Wrong: every incoming response overwrites the record, without looking at its version func receive(_ record: Record, into shelf: Shelf) -> Shelf { upsert(record)(shelf) } ``` This snippet stays SOURCE. Writing by identifier solves repetition, but a late-arriving old response overwrites the new version. The fix is to accept only the larger version. ```swift import Foundation // Fix: only a newer version enters the store func receiveNewer(_ record: Record, into shelf: Shelf) -> Shelf { if let current = shelf.records.first(where: { $0.id == record.id }), current.version >= record.version { return shelf } return upsert(record)(shelf) } ``` This also stays SOURCE. For a record with the same identifier, the function compares the version in the store with the one in the response. If the store has an equal or larger version, the incoming response is ignored. This removes the problem of the two responses' order: whichever order they arrive in, the one with the larger version stays in the store. Repetition is solved too, because the second arrival of the same version is ignored. This function is not in the measurement; the measured `write-by-id` is still order-sensitive, and the effect of the fix is described only in this explanation. The fix has one condition: the server must provide the version number. A client's own counter cannot be compared between two clients. If the server provides no version, the last modification time does a similar job, but clock skew makes this path fragile too. #### Key and Date Strategies The naming habit of the server and the client is often different. The server separates words with underscores, Swift code with capital letters. Giving the decoder a key strategy closes this gap in one place. ```swift import Foundation let decoder = JSONDecoder() decoder.keyDecodingStrategy = .convertFromSnakeCase decoder.dateDecodingStrategy = .iso8601 ``` This snippet stays SOURCE. The date strategy tells the decoder which format the date text sent by the server is in. If the format is chosen wrong, decoding throws a `DecodingError` and the whole response cannot be read. Setting the strategy up in a single place in the network layer, instead of rebuilding it on every call, prevents two screens from reading the same field differently. Missing and extra fields are also a matter decided in the same place. If the response has a field that is not in the type, the decoder ignores it; this makes it possible for the server to add a new field without breaking the client. A required field that is in the type but missing from the response gives a `keyNotFound` error. If you know that the field can really be missing at times, you define it as optional in the type. Swallowing the error without doing this leaves the record silently half complete. #### What the Measurement Shows and Does Not Show The three classes of the three lines separate where the conversion is safe from where it is sensitive. What is safe is the transformation between text and record; what is sensitive is the way the record joins a collection. This distinction saves time when looking for a bug. In an app that gets a decoding error, the problem is in the shape of the response: a field is missing or its type is different. In an app that sees records multiplying or reverting to an old value, decoding has worked correctly and the problem is in the write rule. There are also things the measurement does not show. If the record had a nested object or an array, decoding could fail in more places. If it had a date field, the choice of format would change the result. Large numbers, optional fields, and empty values in the response produce separate behaviors. The record measured in this lesson was deliberately kept small, so that the sensitivity question could be seen without getting mixed up with the details of the conversion. In a real app a decoding error is also an error class, and it falls within the classification described in the third lesson: retrying a response that cannot be decoded mostly gives the same result. #### Facing a Decoding Error If a decoding error is left silent, it becomes one of the most expensive errors. When the server changes the name or type of a field, the whole response becomes unreadable and the app shows an empty screen. The code that catches the error should log in which field and on which path the `DecodingError` occurred; saying only "could not be decoded" stretches finding the problem into hours. The user, on the other hand, is shown not the details but an understandable message. Whether the error is retried is also a decision. Requesting a response that cannot be decoded from the same server again mostly gives the same result; the error is in the shape of the response, not in the network. For this reason a decoding error falls among the non-retryable errors in the third lesson's classification. One exception is when the response has been cut off halfway; then the body is incomplete and requesting it again can fix the result. To tell the two apart, the status code and the length information of the response are read together. #### Common Mistakes - Decoding the response and appending it straight to the list. The same record multiplies when it arrives with two responses. - Treating two responses as "the same" by comparing raw text. Two texts with a different key order can carry the same record. - Swallowing a decoding error. When the server changes a field, the app silently works with the old value. - Assuming the record has no version number and thinking the last response to arrive is the newest. #### Interpreting the Measured Result The three operations had three classes: doubly sensitive, order-sensitive, and robust. Encoding and decoding were robust; the problem was not in the conversion itself but in how the converted record was written to the store. Writing by identifier solved repetition; the order problem remained. To solve order, a version number must be carried in the record and only the larger version written. This rule was not measured; a similar storage rule will be measured in the fourth lesson. #### The Bounding Measurement The rule does not say whether it looks at the raw text or the decoded value; which one is compared determines the result. Below, two texts carrying the same record with a different key order are compared in two ways. ```swift // main.swift — the same record, two texts import Foundation let textA = #"{"id":7,"name":"notebook","version":1}"# let textB = #"{"version":1,"name":"notebook","id":7}"# print("raw text equal: \(textA == textB)") print("decoded record equal: \(decodeRecord(textA) == decodeRecord(textB))") ``` ``` raw text equal: false decoded record equal: true ``` The raw texts are different, the decoded records are the same. A client that compares two responses as raw text takes a repeated response for a new one; a client that compares the decoded value ignores it. **The rule sees the inequality, it does not see what was compared.** **NW6.** This measurement shows the difference of key order. A difference in whitespace or number format was not measured. #### Summary - Encoding and decoding turn a typed record into a byte sequence and back; the round trip came out robust in the measurement. - Appending the decoded record to a list is sensitive to both repetition and order. - Writing by identifier solves repetition, not the arrival order of two versions. - The arrival order of two responses is a behavior of the network; the client cannot choose it. - Raw text equality and decoded value equality do not give the same decision. - This lesson's sensitivity: **order-sensitive**. #### Next Step A response may never arrive, or may be cut off halfway. In the next lesson, Errors and Retries will measure the classification of an error and why an order retried after a timeout can create a second record. ### Errors and Retries Source: https://academia.sh/en/courses/data-networking-and-concurrency/errors-and-retries Classifying network errors, and measuring whether an order retried after a timeout creates a second record on the server. On the network layer an error is not an exception but an everyday event. The connection drops, the server answers late, the response is lost on the way. A network client has to split these errors into two groups: those worth trying again and those that are not. A **retry**, done right, makes the app stable; done wrong, it places the same order twice. In this lesson the error classification is a pure function and is measured. The real question lies where the classification stops: what does retrying a request do on the server? The core is the same. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` #### Error Classification Four errors are told apart: timeout, no connection, server error, and client error. The first two and the 5xx server errors are retryable; 4xx client errors are not, because the same request gets the same error. The number of attempts is limited by a **retry budget**, and the budget is an integer. The wait, or the **exponential backoff** time, is not modeled or measured in this lesson. ```swift // Orders.swift — failure class, order ledger, and retry loop enum Failure: Equatable { case timeout case offline case server(Int) case client(Int) } func isRetryable(_ failure: Failure) -> Bool { switch failure { case .timeout, .offline: return true case .server(let code): return code >= 500 case .client: return false } } struct Ledger: Equatable, CustomStringConvertible { var orders: [String] = [] var description: String { "[" + orders.joined(separator: ",") + "]" } } func place(_ item: String, key: String?) -> (Ledger) -> Ledger { { ledger in let tag = "\(item)@\(key ?? "-")" if key != nil && ledger.orders.contains(tag) { return ledger } return Ledger(orders: ledger.orders + [tag]) } } func simulate(budget: Int, key: String?, lostResponses: Int) -> (attempts: Int, ledger: Ledger) { var ledger = Ledger() var attempts = 0 while attempts < budget { attempts += 1 ledger = place("book", key: key)(ledger) if attempts > lostResponses { break } } return (attempts, ledger) } ``` `Failure` is an **enumeration** with four cases. `isRetryable` looks only at the kind of the error. `place` builds the server recording the order: if a key is given and a record with the same tag exists, it does not add a second one. `simulate` builds the client's loop. The request reaches the server on every attempt and is recorded; the first `lostResponses` responses, however, are lost on the way. The client sees a lost response as a timeout and retries as long as the budget allows. **The model builds what the operation does to the state; it is not the network itself.** ```swift // main.swift — classification, retry, two write forms print("timeout: \(isRetryable(.timeout))") print("offline: \(isRetryable(.offline))") print("server-503: \(isRetryable(.server(503)))") print("client-404: \(isRetryable(.client(404)))") let keyless = simulate(budget: 3, key: nil, lostResponses: 1) let keyed = simulate(budget: 3, key: "k-9", lostResponses: 1) print("keyless: attempts=\(keyless.attempts) ledger=\(keyless.ledger)") print("keyed: attempts=\(keyed.attempts) ledger=\(keyed.ledger)") let withoutKey = replay("keyless-order", from: Ledger(), place("book", key: nil), then: place("book", key: nil)).report() let withKey = replay("keyed-order", from: Ledger(), place("book", key: "k-9"), then: place("book", key: "k-9")).report() let spending = replay("spend-budget", from: 3, { max($0 - 1, 0) }, then: { max($0 - 1, 0) }).report() countSensitivities([withoutKey, withKey, spending]) ``` ``` timeout: true offline: true server-503: true client-404: false keyless: attempts=2 ledger=[book@-,book@-] keyed: attempts=2 ledger=[book@k-9] keyless-order | once=[book@-] twice=[book@-,book@-] ab=[book@-,book@-] ba=[book@-,book@-] -> repeat-sensitive keyed-order | once=[book@k-9] twice=[book@k-9] ab=[book@k-9] ba=[book@k-9] -> robust spend-budget | once=2 twice=1 ab=1 ba=1 -> repeat-sensitive robust: 1 repeat-sensitive: 2 order-sensitive: 0 doubly-sensitive: 0 ``` Three of the four errors are retryable; the client error is not. But the second part is the one that stands out. The same lost response gave two different results in the same loop: the keyless request left two records after two attempts, and the keyed request left one record after the same two attempts. What the client sees is the same in both: a timeout followed by a successful response. `keyless-order` is repeat-sensitive: the second application adds a second record. `keyed-order` is robust: the same key prevents the second record. `spend-budget` is repeat-sensitive; but that is the budget's job, not an error. The budget exists precisely so that it shrinks as it is repeated. The rule does not say which repetition is an error and which is design. **NW7.** The model assumes the server recognizes the key. A real server may ignore the key header, and then the keyed request creates a second record too. **NW8.** The model always builds a lost response as a request that reached the server. A request that never arrives is examined separately. #### Reading the Error on the Client In a real client the error is read from the code of a `URLError`. ```swift import Foundation func classify(_ error: Error) -> Failure { guard let urlError = error as? URLError else { return .client(0) } switch urlError.code { case .timedOut: return .timeout case .notConnectedToInternet: return .offline default: return .client(0) } } ``` This snippet stays SOURCE. `URLError` carries network errors with a code; `.timedOut` and `.notConnectedToInternet` are two of them. The snippet maps only two codes; a real classification covers more. The name of the key header depends on the server; it is usually sent as a request header. #### The Cost of Retrying A retry looks like insurance, but it is not cheap. Every attempt puts one more request on the server, lengthens the user's wait, and spends battery. If the server is already under load, thousands of clients retrying at the same moment do not reduce the load but increase it. So two limits are used together: the retry budget cuts the total number of attempts, while **exponential backoff** doubles the wait between attempts each time. The budget limits how many times we try; the backoff limits how often. A small random share is usually added to the backoff. Otherwise a thousand clients that failed at the same moment retry at the same moment and at the same intervals, and hit the server in waves. The random share spreads those waves out. This lesson does not model the wait, so it does not measure this; but in a real client the budget and the backoff must be treated as two separate decisions. That the budget comes out repeat-sensitive in the measurement is not a flaw. In the `spend-budget` line `once=2` and `twice=1` differ, because the budget exists precisely so that it shrinks with every expenditure. The rule only reports repeat sensitivity; it does not tell whether that is wanted or unwanted. The person reading the measurement makes that distinction. #### Where It Shows Up in a Real App In a shopping app the user taps the "Confirm Order" button. The network is slow, the request reaches the server and the order is recorded, but the response is lost on the way. The client sees a timeout and sends the request again. Without a key, the server counts the second request as a brand new order and the user is charged twice. The `ledger=[book@-,book@-]` in the `keyless` line is exactly this situation. The user mostly experiences this as a separate error: "Why did my order come twice?" The support team sees two records, both look valid, and which one to delete is unclear. The problem is hard to reproduce, because it appears only in the rare case where the response is lost. The developer never sees it in testing, because the network always works there. In production it happens a few times a day among millions of requests. The same situation is not limited to payments. Any operation that creates a new record on the server, such as sending a message, adding a comment, or putting an item in a cart, carries the same risk. Read operations, on the other hand, are usually retried safely, because they do not change the state on the server. #### The Wrong Setup and Its Fix The common wrong setup is to retry on every error and without limit. ```swift import Foundation // Wrong: every error is retried, no budget, no wait func sendWithRetry(_ request: URLRequest, using session: URLSession) async throws -> Data { while true { do { let (data, _) = try await session.data(for: request) return data } catch { continue } } } ``` This snippet stays SOURCE. It has three problems: even a `404` is retried forever, when the server goes down the client sends hundreds of requests a second, and the user cannot understand why the screen never moves on. The fix brings the classification, the budget, and the wait together. ```swift import Foundation // Fix: only retryable errors, a limited budget, a doubling wait func sendWithRetry(_ request: URLRequest, using session: URLSession, budget: Int = 3) async throws -> Data { var delay: UInt64 = 500_000_000 for attempt in 1...budget { do { let (data, response) = try await session.data(for: request) if let http = response as? HTTPURLResponse, http.statusCode >= 500, attempt < budget { try await Task.sleep(nanoseconds: delay) delay *= 2 continue } return data } catch { guard isRetryable(classify(error)), attempt < budget else { throw error } try await Task.sleep(nanoseconds: delay) delay *= 2 } } throw URLError(.unknown) } ``` This also stays SOURCE. The function waits only on 5xx responses and retryable errors. A 4xx response returns without waiting; the caller reads it as its own error. The wait doubles with every attempt. When the budget runs out, the last error is thrown to the caller, so nobody keeps waiting without noticing that the error was swallowed. The corrected function does not make retrying safe; it only limits it. For the request to be robust against repetition, an idempotency key is also needed, and that is what the `keyed-order` line shows. #### Carrying the Error to the User When the budget is spent, the error must not disappear silently. The user is shown two things: what happened and what they can do. If there is no connection, "You have no internet connection" and an offer to retry automatically when the connection returns make sense. For a server error, "We cannot reach the service right now, try again shortly" is enough. For a client error, retrying has no benefit; the user is told to correct what they entered. When a keyless write request ends with a timeout, the situation is more subtle still. The client does not know whether the request reached the server; this is the uncertainty the bounding measurement shows. The right behavior is to tell the user "The result is uncertain" and not to send a second write without querying the state on the server. If there is a key, the uncertainty disappears, because the second request can be repeated harmlessly with the same key. #### Separating Read and Write Requests The first input to the retry decision is not the error but the kind of the request. A read request does not change the state on the server; even if its response is lost, sending it a second time is safe. A write request changes the state, and a second send means the state changes twice. The protocol carries this distinction to some extent itself: read requests are by definition considered repeatable, requests that create a new record are not. Update and delete requests are also mostly designed to be repeatable, because applying the same update a second time does not change the result. In practice this means two separate paths in the client. The read path classifies the error, waits, and retries as long as the budget allows. The write path first looks at whether the request carries an idempotency key. If there is a key, it behaves like the read path. If there is no key, the retry decision is left to the user or made by querying the state on the server. The `keyless-order` and `keyed-order` lines in the measurement show the difference between these two paths through their results. #### What Spends the Budget Not every error takes the same share of the budget. With no connection the request never leaves; counting that as an attempt wastes the budget. A better setup is to wait until the connection returns and to spend the budget only on requests that are really sent. With server errors, every send puts load on the server, and the budget limits exactly this load. A timeout is in between: the request has left, but its result is unknown. The size of the budget also depends on the nature of the error. For a brief network outage, three attempts are usually enough. When the server is unreachable for a long time, dozens of attempts only create load; then it is more correct for the client to give up and inform the user. #### Common Mistakes - Retrying every error. A `404` response stays a `404` however many times it is repeated. - Not putting an upper limit on retries. A loop without a budget wears the server out further. - Retrying a keyless `POST` after a timeout. - Reading a timeout as "the request failed". A timeout says only that the response did not arrive. - Trying back to back without a wait. This lesson does not measure the wait, but a real client needs it. #### Interpreting the Measured Result Retrying looks like a classification question but it is a sensitivity question. Classification gives one of two results: try or do not try. Sensitivity asks what the result of the trying will be. The keyless order created a second record; the keyed order did not. The error classification was the same in both. #### The Bounding Measurement When the client sees an error, it cannot know which of two worlds it is in: the request reached the server but the response was lost, or the request never arrived. The measurement below builds the two worlds and prints what the client sees next to the ledger on the server. ```swift // main.swift — two worlds, the same observation func world(requestArrived: Bool) -> (observed: Failure, ledger: Ledger) { let ledger = requestArrived ? place("book", key: nil)(Ledger()) : Ledger() return (.timeout, ledger) } let lost = world(requestArrived: true) let never = world(requestArrived: false) print("client observation equal: \(lost.observed == never.observed)") print("server ledger equal: \(lost.ledger == never.ledger)") ``` ``` client observation equal: true server ledger equal: false ``` What the client sees is the same in the two worlds; the ledger on the server is different. So the client cannot decide from its observation whether to retry; the decision is given by the request itself, that is, by the key. The rule neither resolves nor measures this uncertainty. **NW9.** The measurement builds the two worlds as indistinguishable from the client's side. A real client can know in some errors (such as a connection that could not be established at all) that the request did not arrive. #### Summary - Network errors are classified as retryable and non-retryable with a pure function; the number of attempts is limited by a budget. - Retrying a keyless request after a timeout created a second record on the server. - Under the same retry, the request with an idempotency key left a single record. - Spending the budget is repeat-sensitive, but that is design, not an error. - The client cannot tell a lost response from a request that never arrived by its observation. - This lesson's sensitivity: **repeat-sensitive**. #### Next Step A retried request sometimes carries an identity as well. In the next lesson, Authentication will measure how storing the responses of two concurrent token refreshes in a different order can bring back the old token. ### Authentication Source: https://academia.sh/en/courses/data-networking-and-concurrency/authentication Storing and refreshing bearer tokens; measuring whether storing two refresh responses in a different order brings back the old token. Most backend requests require **authentication**. The client puts a **bearer token** on every request: the server treats whoever carries the token as authorized. The token is short-lived; when it expires, the client asks for a new one with a long-lived **refresh token**. The short-lived one of the tokens is called the **access token**. This mechanism is sensitive in two places. The first is storage: if refresh responses arrive in reverse order, which one is stored matters. The second is the server: a refresh token is usually single-use, and its second use is rejected. The core is the same. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` #### Token Storage The model stamps the token with a counter: `serial` says which refresh the token came from. Expiry is modeled with this counter instead of a real clock, so the measurement does not depend on the clock. ```swift // Vault.swift — token, vault, and the server's refresh state struct Token: Equatable { let value: String let serial: Int } struct Vault: Equatable, CustomStringConvertible { var current: Token? var description: String { current.map { "\($0.value)#\($0.serial)" } ?? "empty" } } func store(_ token: Token) -> (Vault) -> Vault { { _ in Vault(current: token) } } func storeNewer(_ token: Token) -> (Vault) -> Vault { { vault in guard let held = vault.current, held.serial >= token.serial else { return Vault(current: token) } return vault } } struct Issuer: Equatable, CustomStringConvertible { var valid: String var issued: Int = 1 var rejected: Int = 0 var description: String { "valid=\(valid) issued=\(issued) rejected=\(rejected)" } } func redeem(_ refresh: String) -> (Issuer) -> Issuer { { issuer in guard refresh == issuer.valid else { return Issuer(valid: issuer.valid, issued: issuer.issued, rejected: issuer.rejected + 1) } return Issuer(valid: "r\(issuer.issued + 1)", issued: issuer.issued + 1, rejected: issuer.rejected) } } ``` `store` stores the incoming token unconditionally; the last writer wins. `storeNewer` stores only a token with a larger stamp; one that arrives with an equal or smaller stamp is ignored. In the measurement below there are two refresh responses: the old one, `a1`, comes from the first refresh, and the new one, `a2`, from the second. If two concurrent refresh requests leave at the same moment, the order in which the responses return is not certain. ```swift // main.swift — storing two refresh responses let stale = Token(value: "a1", serial: 1) let fresh = Token(value: "a2", serial: 2) let lastWins = replay("last-write-wins", from: Vault(), store(stale), then: store(fresh)).report() let newerWins = replay("larger-serial-wins", from: Vault(), storeNewer(stale), then: storeNewer(fresh)).report() countSensitivities([lastWins, newerWins]) ``` ``` last-write-wins | once=a1#1 twice=a1#1 ab=a2#2 ba=a1#1 -> order-sensitive larger-serial-wins | once=a1#1 twice=a1#1 ab=a2#2 ba=a2#2 -> robust robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` `last-write-wins` is order-sensitive. If the old response comes later, `a1` stays in the vault as in the `ba` column; the app keeps sending requests with an old token that has expired or been replaced by a new one. `larger-serial-wins` is robust: `a2` stays in both orders, and storing the same response twice changes nothing. **NW10.** The model assumes the server issues an increasing stamp on every refresh. With a server that issues no stamp, the client cannot establish the order on its own. **NW11.** The old token may or may not still be valid on the server. The measurement looks only at what the client stores. #### The Token on the Request and Single Flight The bearer token is added to the request with the `Authorization` header. ```swift import Foundation func authorize(_ request: URLRequest, token: String) -> URLRequest { var signed = request signed.setValue("Bearer \(token)", forHTTPHeaderField: "Authorization") return signed } ``` This snippet stays SOURCE. Another way to solve the order problem before storage is **single flight**: only one refresh runs at a time, and the other callers wait for the result of that refresh. An **actor** builds this naturally. ```swift // token provider — SOURCE actor TokenProvider { private var refreshTask: Task? func validToken(refresh: @escaping @Sendable () async throws -> String) async throws -> String { if let task = refreshTask { return try await task.value } let task = Task { try await refresh() } refreshTask = task defer { refreshTask = nil } return try await task.value } } ``` This snippet is not measured either. The second caller finds `refreshTask` filled and does not start a new refresh. However, the actor's state is consistent only between these `await` points; if the vault is written from somewhere else while a refresh is in progress, the same order problem comes back. #### The Lifetime of the Access and Refresh Tokens The two tokens do different jobs. The access token goes with every request; its lifetime is short, because if it is stolen the damage is short too. The refresh token is used only in a refresh request; its lifetime is long, because it saves the user from signing in again every half hour. The long-lived one is kept in fewer places and more securely. In apps the refresh token usually lives in the system's keychain; the access token can be kept in memory, because if it is lost it is obtained again by refreshing. This distinction explains why the measurement looks at two refresh responses. When the access token expires, the client asks for a new pair with the refresh token. If several requests have left at the same time with the expired token, all of them get a 401 and all of them want to refresh. Two requests with the same refresh token reach the server, and one of the responses returns before the other. The two responses carry two separate pairs; depending on which one it stored, the client continues with either the fresh or the stale token. #### Where It Shows Up in a Real App When a home page opens, the app sends three requests at the same time, for the profile, the notification count, and the recommendation list. If the access token has expired at that moment, all three return 401. A naive client sends three separate refresh requests. If the server keeps refresh tokens single-use, the first succeeds, the other two are rejected, and the user sees their session close. This is the everyday form of the `token-rotation` line in the measurement: the first use consumes the token and the second is rejected. The problem may not show to the user as an error. The app may have silently returned to the sign-in screen, and the user complains that it "signed out by itself". In the logs only a "refresh rejected" line appears. The real cause is that the refresh was started more than once. So a rule is needed that makes sure only one refresh runs at a time. This rule is called **single flight**: while a refresh is in progress, every request that arrives does not start a new refresh, it waits for the result of the one in progress. #### The Wrong Setup and Its Fix In the wrong setup every failed request starts its own refresh. ```swift import Foundation // Wrong: three concurrent 401s start three separate refreshes func refreshAccessToken(using session: URLSession, refresh: String) async throws -> String { var request = URLRequest(url: URL(string: "https://api.example.com/refresh")!) request.httpMethod = "POST" request.httpBody = Data(refresh.utf8) let (data, _) = try await session.data(for: request) return String(decoding: data, as: UTF8.self) } ``` This snippet stays SOURCE. The fix is to gather the refresh in a single place: an actor holds the refresh in progress as a task and gives every call that reaches it the result of the same task. ```swift import Foundation actor RefreshCoordinator { private var inFlight: Task? func freshToken(using perform: @escaping () async throws -> String) async throws -> String { if let inFlight { return try await inFlight.value } let task = Task { try await perform() } inFlight = task defer { inFlight = nil } return try await task.value } } ``` This also stays SOURCE. The actor guarantees that only one call touches its state at a time; so `inFlight` does not race between two calls. The first call starts the task, the later ones wait, and all continue with the same token. The solution resets repetition: a single refresh reaches the server. The `token-rotation` line in the measurement provides no protection for this, because the model builds what the server does; the side that applies the protection is the client. This solution still has a limit. Single flight prevents concurrent refreshes; it does not prevent the storage order of two responses. A late response left over from a previous refresh can still write the stale token to the vault if it is stored after the new one. So a stamp comparison like `storeNewer` is also needed: the two protections solve two separate problems. #### Where the Token Is Stored The `Vault` in the model is a vault; in a real app its counterpart has two layers. Memory gives fast access while the app is running and is lost when the app closes. The system's keychain stays after closing and keeps its data encrypted. Writing the token to a plain store such as user preferences is a common and serious mistake: a plain store goes into backups and file dumps as it is. The keychain will be covered separately in the next topic; what matters here is that storing is a write operation, and a write operation is subject to this whole course's measure. The subtle point in storing is that writing to the keychain can be an asynchronous job. If two refresh responses arrive back to back, both writes are sent, and which one reaches the store last may not be the same as which one was called last. That is why the `last-write-wins` line in the measurement is realistic: the model builds the write itself as an order-sensitive operation. The fix is to carry the order information in the value itself; that is why the `larger-serial-wins` line came out robust. #### Signing Out When the user signs out, the client does two things: it deletes the tokens in the vault and has the server revoke the refresh token. Deleting is a local operation and is robust against repetition: emptying an empty vault once more changes nothing. The revoke request goes to the server and must be retryable; for this reason revoking is designed as an idempotent operation. If a refresh is still in progress at the time of sign-out, the late-returning response can fill the vault again. The session coming back while the user has signed out is the most uncomfortable form of the order problem. To prevent this, sign-out cancels the refresh in progress, and before storing a returning response it is checked whether the session is still open. #### If the Refresh Itself Fails The refresh request is also a request and can fail. Two cases need to be told apart here. If the refresh token has been rejected, that is, the server responded and treated the token as invalid, the session has really ended and the user has to sign in again. If the refresh request has dropped because of a network error, the session is intact; only the connection is temporary. A client that mixes up these two cases throws a user whose connection dropped briefly out of the session unnecessarily. The right behavior is to apply the classification in the third lesson here too. Timeout and no connection are retryable errors; the refresh is retried within the budget and the token in the vault is not deleted. The server's explicit rejection, on the other hand, empties the vault and sends the user to the sign-in screen. The rejected counter in the `token-rotation` line is the model of this second case. A counter greater than zero shows that the server rejected a refresh token; when the client sees this, it should end the session instead of retrying. #### Refreshing Before Expiry Refreshing can be done at two times. In the reactive approach, the token is refreshed when a request gets a `401`; the code is simple, but at every expiry one request is wasted and all the requests that left at the same time fail. In the proactive approach, the access token is refreshed shortly before it expires; so a `401` is never seen. This requires the server to give the expiry information in the response and relies on the client clock. If the clock has drifted, the token is refreshed while it is actually still valid, or is thought valid when it has expired. In practice the two are used together: proactive refresh prevents most errors, and reactive refresh catches the cases that slip through, such as clock drift. Both paths must go through the single flight rule; otherwise proactive and reactive refresh leave at the same time and use the same refresh token twice. #### Common Mistakes - Storing the refresh response unconditionally. A late-arriving old response overwrites the new token. - Starting a separate refresh after each failed request. If requests get `401` at the same time, several refreshes race. - Using the refresh token a second time. The server may reject it and may close the session as well. - Writing the token to a log or putting it in an error message. - Tying expiry to the client clock. Clock drift produces wrong decisions. #### Interpreting the Measured Result Two storage rules received the same two responses and gave different results. The difference is not in the responses but in the rule: one looked at the order, the other at the stamp. Because the stamp carried the order into the response, the arrival order became unimportant. This is a pattern often used on the network: writing the order into the data instead of expecting it from the network. #### The Bounding Measurement Even if storage is robust, the refresh token is still unprotected against repetition. Below, the same refresh token is used twice on the server. ```swift // main.swift — the second use of the same refresh token let reuse = replay("token-rotation", from: Issuer(valid: "r1"), redeem("r1"), then: redeem("r1")).report() countSensitivities([reuse]) ``` ``` token-rotation | once=valid=r2 issued=2 rejected=0 twice=valid=r2 issued=2 rejected=1 ab=valid=r2 issued=2 rejected=1 ba=valid=r2 issued=2 rejected=1 -> repeat-sensitive robust: 0 repeat-sensitive: 1 order-sensitive: 0 doubly-sensitive: 0 ``` The first use consumes `r1` and makes `r2` valid; the second use finds `r1` invalid and rejects it. This is **token rotation**, and it came out repeat-sensitive in the measurement. The two orders gave the same result, because the two operations were the same. **The rule does not say whether this rejection is a security feature or a bug.** **NW12.** The server is assumed to reject the old refresh token immediately. Some servers allow a short grace period; this behavior was not measured. #### Summary - The access token is short-lived; the refresh token is long-lived and serves to obtain a new one. - Storing refresh responses unconditionally is order-sensitive; a late-arriving old response overwrites the new token. - Storage that keeps the one with the larger stamp left the same token in both orders. - Single flight solves the problem from another place by running one refresh at a time. - The second use of the refresh token was rejected, and this came out repeat-sensitive. - This lesson's sensitivity: **order-sensitive**. #### Next Step After the token and the responses arrive, some of them need to be stored. In the next lesson, Image and Data Caching will measure whether the same pair of operations comes out robust in an empty cache and order-sensitive in a full one. ### Image and Data Caching Source: https://academia.sh/en/courses/data-networking-and-concurrency/image-and-data-caching The eviction rule of memory and disk caches; measuring why the same pair of operations comes out robust in an empty cache and order-sensitive in a full one. Downloading the same image again every time a screen opens wastes both data and battery. This is why the **cache** exists: it keeps a response that was already fetched close at hand. Two kinds of cache are told apart. The **memory cache** is fast but disappears when the app closes. The **disk cache** is slower but stays after the app closes. Both are limited; when the limit is reached, an entry is **evicted**. The question is this: when is writing to the cache safe? In an empty cache, almost every write is safe. In a full cache, which entry is discarded depends on the order of the operations. The core is the same. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` #### A Cache with a Capacity The model is a cache with a limited capacity that discards the entry unused for the longest time. Entries are "key=value" strings; the entry at the head of the list is the one unused for the longest time. The state comparison and the output show the entries in sorted order. ```swift // Cache.swift — a cache with a capacity that discards the least recently used entry struct Cache: Equatable, CustomStringConvertible { var capacity: Int var entries: [String] = [] static func == (lhs: Cache, rhs: Cache) -> Bool { lhs.capacity == rhs.capacity && lhs.entries.sorted() == rhs.entries.sorted() } var description: String { "[" + entries.sorted().joined(separator: ",") + "]" } } func entryKey(_ entry: String) -> String { String(entry.split(separator: "=", maxSplits: 1).first ?? "") } func put(_ key: String, _ value: String) -> (Cache) -> Cache { { cache in var next = cache next.entries.removeAll { entryKey($0) == key } next.entries.append("\(key)=\(value)") while next.entries.count > next.capacity { next.entries.removeFirst() } return next } } func get(_ key: String) -> (Cache) -> Cache { { cache in var next = cache if let index = next.entries.firstIndex(where: { entryKey($0) == key }) { let entry = next.entries.remove(at: index) next.entries.append(entry) } return next } } ``` `put` removes the old entry with the same key, adds the new one at the end, and if the capacity is exceeded, discards the one at the head. `get` moves the entry it finds to the end; reading is also a use. **The model builds what the operation does to the state; it is not the system's cache or the disk itself.** `==` compares only the contents of the entries; how old an entry is does not show. #### Three Operations in an Empty Cache ```swift // main.swift — three operations in an empty cache let empty = Cache(capacity: 2) let sameKey = replay("same-key", from: empty, put("profile", "v1"), then: put("profile", "v1")).report() let twoKeys = replay("two-keys", from: empty, put("profile", "v1"), then: put("feed", "v1")).report() let writeRead = replay("write-read", from: empty, put("image", "v1"), then: get("profile")).report() countSensitivities([sameKey, twoKeys, writeRead]) ``` ``` same-key | once=[profile=v1] twice=[profile=v1] ab=[profile=v1] ba=[profile=v1] -> robust two-keys | once=[profile=v1] twice=[profile=v1] ab=[feed=v1,profile=v1] ba=[feed=v1,profile=v1] -> robust write-read | once=[image=v1] twice=[image=v1] ab=[image=v1] ba=[image=v1] -> robust robust: 3 repeat-sensitive: 0 order-sensitive: 0 doubly-sensitive: 0 ``` All three operations are robust. Writing the same data to the same key twice does not change the content. Two different keys leave the same content in both orders, as long as they do not exceed the capacity. Reading a key that does not exist does nothing. In this section the cache is an example of robustness, and most of the time it is; this is why writing to a cache is thought of as an easy job. **NW13.** The measurement takes the capacity as 2. With other capacities, which pairs of operations fill it changes. **NW14.** The state comparison does not see the age order of the entries. Two caches with a different order but the same content are counted as equal. #### The System's Caches `NSCache` is used for the memory cache, and `URLCache` for the disk and memory cache of network responses. ```swift import Foundation let memory = NSCache() memory.countLimit = 100 memory.setObject(Data([1, 2, 3]) as NSData, forKey: "profile" as NSString) let entry = memory.object(forKey: "profile" as NSString) ``` ```swift import Foundation let urlCache = URLCache(memoryCapacity: 20_000_000, diskCapacity: 100_000_000, directory: nil) let configuration = URLSessionConfiguration.default configuration.urlCache = urlCache configuration.requestCachePolicy = .returnCacheDataElseLoad ``` Both snippets stay SOURCE. `NSCache` can discard entries by itself under memory pressure and does not document in what order it will discard them; for this reason the model above does not imitate its eviction policy, it builds the lesson's own capacity rule. `URLCache` keeps responses in memory and on disk; the server's headers also affect which response is cached. `returnCacheDataElseLoad` returns what is in the cache without going to the network, if there is something there. #### Why the Cache Looks Robust All three of the three operations came out robust in an empty cache. This does not mean a cache is always robust; it only means there is no eviction, because while it is empty the capacity never comes into play. An empty cache holds a single entry when the same key is written twice, and gives the same set whichever order you write two different keys. When the capacity fills, a new entry discards an old one, and the entry discarded changes with the order of the operations. The lesson's bounding measurement shows exactly this: the same pair of operations comes out order-sensitive in a full cache. Knowing the difference between these two situations also determines a test strategy. A test that checks the cache from an empty start does not see that the problem begins when the capacity is exceeded. A real bug usually appears after a long use, when the cache is full: the user has browsed about a hundred images, the first image they opened has been evicted, and it is downloaded again when they go back. The nature of the bug is performance, not correctness; but order sensitivity determines which entry is lost. #### Where It Shows Up in a Real App A feed screen downloads a small image for each row. As the user scrolls down, images are evicted; when they scroll back up, the same images are needed again. Without a cache every scroll is a re-download: data is spent, the screen flickers. With a cache, going back up is instant. The capacity is the balance of two forces. A very large cache bloats memory and the system terminates the app; a very small cache downloads half the screen again. Staleness is a second problem. A user who changes their profile picture and sees the old picture still sitting in the cache thinks the change was not saved. There are two common solutions for this: adding a version to the address of the picture, so that when the address changes the old entry is never used; or explicitly deleting the relevant entry at the moment of change. Both rely on the `put` operation in the measurement being robust against repetition: writing the same key again replaces the old entry, it does not duplicate it. #### The Wrong Setup and Its Fix A common wrong setup is to use the image address as the cache key and not take the query parameters into account. ```swift import Foundation // Wrong: the key is only the path; different sizes going to the same path collide func cacheKey(for url: URL) -> String { url.path } // Fix: the query also goes into the key; the key changes when the size or version changes func cacheKey(for url: URL, size: Int) -> String { "\(url.path)?\(url.query ?? "")#\(size)" } ``` This snippet stays SOURCE. The first function maps the addresses `avatar.png?size=64` and `avatar.png?size=512` to the same entry, and the small image looks blurry on a large screen. The second function adds the query and the size to the key. The `two-keys` line in the measurement is the model of this: different keys do not affect each other, and whichever order they are written in, the cache holds the same set. The second mistake is to put every response that comes from the network into the cache as it is. An error page is also a response; if a 404 response enters the cache, the client keeps showing the error even after the image is fixed on the server. Only successful responses and images that can be decoded should be stored. #### The Limit of the System Caches The system has two caches, and each does a different job. The memory cache, that is `NSCache`, discards entries on its own under memory pressure; which entry is discarded and when is the system's decision. The HTTP cache, that is `URLCache`, stores responses according to the headers the server sends and reuses them until they expire. The rule the model builds, discarding the entry unused for the longest time, is not an exact counterpart of either. Because the system's eviction decision was not measured, this behavior goes into an assumption block. #### What the Eviction Rule Hides The rule in the model is to discard the entry unused for the longest time, and this rule has two sides. Writing puts a new entry at the end of the list; reading moves an existing entry to the end. For this reason reading, even though it looks like an operation that changes nothing, changes the state of the cache. The `write-read` line did not show this in an empty cache, because the key that was read did not exist. In a full cache, reading determines which entry escapes the next eviction. The `ba` column in the bounding measurement shows exactly this: because `a` is read first it becomes the newest, and the one discarded is `b`. The counterpart of this behavior in a real app is this: showing an image also keeps it alive in the cache. A profile picture the user looks at often lives longer than feed images shown only once. The rule is simple and gives good results in most workloads. But a job that reads every entry only once, like a scan, can also push out the frequently used entries. A user who browses hundreds of images in a row evicts the profile picture and the icons that were in the cache at the start; when they return, all of them are downloaded again. #### Choosing the Capacity The capacity is a compromise between two resources. If it is chosen large, memory consumption rises; the system may terminate the app under memory pressure, and the gain the cache provides turns into a crash. If it is chosen small, the hit rate falls and the cache does almost no work. A good start is to limit the total size, not the number of entries: a hundred small icons and a hundred large photos do not use the same memory. The system's memory cache takes both a count limit and a cost limit; the capacity in the model, which depends only on the count, is a simple counterpart of this. A separate limit is set for the disk cache too. The disk looks large and cheap, but the space on the user's phone is a shared resource, and the system can delete cache files to free the disk. For this reason the cache on disk should hold data whose loss does not break the app; images that can be downloaded again suit this, content created by the user does not. #### Stale Entries and Validity The least reliable side of a cache is that the data it holds can fall behind the truth on the server. There are three common ways to manage this. The first is a lifetime limit: each entry is given an expiry time, and an expired entry is not read. The second is validation: before using the entry, the client asks the server whether it has changed, and if it has not, the response returns as a short confirmation. The third is adding a version to the key; when the content changes the address changes too, the old entry is left without an owner, and is eventually deleted by eviction. Which one to choose depends on how often the data changes. For an image that never changes, a versioned address is enough. For a list that changes often, a short lifetime limit and validation are used together. None is enough alone: a lifetime limit makes the client download data unnecessarily, validation sends a request every time, and a versioned address requires the server to generate the addresses. #### Testing the Cache Testing code with a cache requires not being content with an empty cache. This is the warning of a measurement that came out robust: in the empty state every operation is safe. A good test fills the cache up to its capacity, then adds an operation that triggers eviction, and checks which entry remains. It is necessary to verify separately that reading keeps an entry alive and that writing discards the oldest. A team that leaves these two behaviors untested learns of a bug in eviction only in production, when a user complains. For rules that depend on time, a fake clock is used instead of the real clock; to test an entry whose lifetime limit has expired, the test does not have to wait for seconds. That the lesson's model builds time with a counter or with ordering is, in this respect too, a convenience for testing. #### Common Mistakes - Leaving the cache unbounded. The memory cache bloats and the system can terminate the app. - Treating the value in the cache as the single source of truth. A cache can always be stale. - Keeping the files in the disk cache as important data the user cannot delete. The system deletes them to free the disk. - Relying on the eviction order of the system cache. The order is undocumented. - Putting only the address in the key and not including the version or the size. Two sizes of the same address get mixed up. #### Interpreting the Measured Result All three operations came out robust. This is not a result, it is a starting point: none of the operations touched the limit of the cache. The distinguishing feature of a cache is its limit, and when the limit is reached, a write is no longer only an addition, it is also a removal. #### The Bounding Measurement Below, the same two operations, writing the key `c` and reading the key `a`, are applied to two caches. One is empty, the other is full to its capacity, and `a` is the entry unused for the longest time. ```swift // main.swift — the same pair of operations, empty and full cache let full = put("b", "2")(put("a", "1")(Cache(capacity: 2))) let onEmpty = replay("empty-cache", from: Cache(capacity: 2), put("c", "3"), then: get("a")).report() let onFull = replay("full-cache", from: full, put("c", "3"), then: get("a")).report() countSensitivities([onEmpty, onFull]) ``` ``` empty-cache | once=[c=3] twice=[c=3] ab=[c=3] ba=[c=3] -> robust full-cache | once=[b=2,c=3] twice=[b=2,c=3] ab=[b=2,c=3] ba=[a=1,c=3] -> order-sensitive robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` In the empty cache `a` does not exist; the read does nothing and the two orders give the same result. In the full cache the order is decisive. If the write comes first, `c` discards `a`, the oldest entry, and the read is wasted. If the read comes first, `a` becomes the newest and the write discards `b`. The two orders leave different contents. **The rule does not know the cache is full, it sees only that the two results are different.** **NW15.** This order behavior is not assumed for the system's memory cache. A real `NSCache` discards entries with its own policy, and the measured order is valid only for the lesson's model. #### Summary - The memory cache is fast and temporary; the disk cache is slower but persistent; both are limited and evict when full. - Writing the same data with the same key and writing two different keys came out robust in an empty cache. - In a full cache, the order of the write and the read determined which entry was discarded. - The same pair of operations is robust in an empty cache and order-sensitive in a full one. - The eviction order of the system cache is undocumented; the model is the lesson's own capacity rule. - This lesson's sensitivity: **robust**. #### Next Step The cache is a mechanism that works while the app is open. In the next lesson, Background Transfers will measure why putting downloads that the system keeps running while the app is in the background back in the queue can download the same file twice. ### Background Transfers Source: https://academia.sh/en/courses/data-networking-and-concurrency/background-transfers Downloads the system keeps running while the app is in the background; measuring why re-queuing on relaunch without checking the list downloads the same file twice. A user can start downloading a large file and leave the app. In an ordinary session the transfer is cut off when the app moves to the background. A **background transfer** changes this: the system runs the transfer, and it continues even if the app is terminated by the system. When it finishes, or when needed, it wakes the app. This arrangement has a consequence: when the app is relaunched, some transfers are already running. The app's own list does not know this. Re-queuing the list as it is can mean starting a second time a transfer the system is already running. The core is the same. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` #### The Transfer Registry The model builds the transfer registry that the app knows. Each transfer is referred to by an identifier, and the registry is kept in sorted order. Two ways of queuing are compared: one that adds without looking at the identifier and one that looks first. ```swift // Transfers.swift — the transfer registry and two ways of queuing struct Registry: Equatable, CustomStringConvertible { var tasks: [String] = [] var description: String { "[" + tasks.joined(separator: ",") + "]" } } func enqueueAll(_ files: [String]) -> (Registry) -> Registry { { registry in Registry(tasks: (registry.tasks + files).sorted()) } } func enqueueIfMissing(_ files: [String]) -> (Registry) -> Registry { { registry in let missing = files.filter { !registry.tasks.contains($0) } return Registry(tasks: (registry.tasks + missing).sorted()) } } func finish(_ file: String) -> (Registry) -> Registry { { registry in Registry(tasks: registry.tasks.filter { $0 != file }) } } ``` `enqueueAll` adds a new transfer for every file in the list. `enqueueIfMissing` adds only those that are not in the registry. `finish` builds the completion of a transfer. **The model builds only the app's registry; it is not the system's transfer execution or the disk itself.** The starting state in the measurement is a relaunch in which the system kept the download of `a.zip` running while the app was closed. ```swift // main.swift — two ways of queuing on relaunch let reopened = Registry(tasks: ["a.zip"]) let wanted = ["a.zip"] let blind = replay("queue-without-checking", from: reopened, enqueueAll(wanted), then: enqueueAll(wanted)).report() let byIdentifier = replay("by-identifier", from: reopened, enqueueIfMissing(wanted), then: enqueueIfMissing(wanted)).report() countSensitivities([blind, byIdentifier]) ``` ``` queue-without-checking | once=[a.zip,a.zip] twice=[a.zip,a.zip,a.zip] ab=[a.zip,a.zip,a.zip] ba=[a.zip,a.zip,a.zip] -> repeat-sensitive by-identifier | once=[a.zip] twice=[a.zip] ab=[a.zip] ba=[a.zip] -> robust robust: 1 repeat-sensitive: 1 order-sensitive: 0 doubly-sensitive: 0 ``` `queue-without-checking` is repeat-sensitive. The system was already running `a.zip`; when the list was re-queued a second `a.zip` record appeared, and when it was queued once more, a third. The same file is downloaded twice in a row. `by-identifier` is robust: because the identifier is in the registry, queuing adds nothing, however many times it is done. **NW16.** The model counts the transfer the system kept running as present in the app's registry. In reality the app builds the registry by querying the system itself. **NW17.** The model counts two records of the same file as two separate downloads. In a real app, if the two records write to the same target file, one may overwrite the other. #### The Background Session A background transfer is set up with a special configuration that has an identifier. ```swift import Foundation let configuration = URLSessionConfiguration.background(withIdentifier: "downloads") configuration.sessionSendsLaunchEvents = true let session = URLSession(configuration: configuration, delegate: downloader, delegateQueue: nil) let task = session.downloadTask(with: URL(string: "https://files.example.com/a.zip")!) task.taskDescription = "a.zip" task.resume() ``` This snippet stays SOURCE. The configuration's identifier makes it possible to reconnect to the same session when the app is relaunched. The reconnected session can query the tasks that are in progress. ```swift import Foundation func pendingFiles(of session: URLSession) async -> Set { let tasks = await session.allTasks return Set(tasks.compactMap(\.taskDescription)) } ``` This also stays SOURCE. The model represents the result of this query as a `Registry`; `enqueueIfMissing` compares against this set before queuing. When the transfer finishes, a **delegate** receives the result, and the file must be moved from the temporary location to a permanent one. ```swift import UIKit final class AppDelegate: UIResponder, UIApplicationDelegate { var completionHandlers: [String: () -> Void] = [:] func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) { completionHandlers[identifier] = completionHandler } } ``` If the system wakes the app in the background, it calls this **app delegate** handler. The app tells the system it has finished its work by calling the stored **closure** when the events are done. #### The System Runs the Transfer The basic idea of a background session is that the transfer does not depend on the lifetime of the app. In an ordinary session a download is cut off shortly after the app moves to the background, because the system suspends the app. In a background session, a system process separate from the app runs the download. Even if the app is suspended, or even closed by the system without being terminated by the user, the transfer can continue. When the job is done, the system wakes the app if needed and delivers the result. The cost of this arrangement is that the app is no longer the only owner of the transfer. The system keeps the state of the transfer; the list in the app's hands is only a copy. While the app is closed, the transfer may have completed, failed, or still be running, and when the app is relaunched it cannot know which of these three it is without asking the system. The `Registry` in the model builds this copy of the app; the `queue-without-checking` line shows what happens in a relaunch where the copy does not match reality. #### Where It Shows Up in a Real App In a podcast app, the user queues ten episodes for download and puts the phone in their pocket. The app moves to the background and the system continues the downloads. When the user opens the app an hour later, some of the episodes have been downloaded and some are still in progress. If the first thing the app does at launch is to re-queue the "to download" list, the episodes in progress are started a second time. The same file is downloaded twice; the data plan is spent, the battery drains, and if two transfers try to write to the same target file, the file can be corrupted. A second case is the user closing the app themselves. If the user terminates the app from the multitasking screen, the system cancels the session's background transfers and does not launch the app on its own until the user opens it again. Because this behavior takes place in the system, it is not measured here; asking which transfers are still running when the app is relaunched is the only safe way to tell these two cases apart. #### The Wrong Setup and Its Fix The wrong setup is to re-queue the saved list directly at launch. ```swift import Foundation // Wrong: at launch the list is re-queued as it is func resumeDownloads(_ files: [URL], using session: URLSession) { for file in files { let task = session.downloadTask(with: file) task.taskDescription = file.lastPathComponent task.resume() } } ``` This snippet stays SOURCE. The fix is to first ask the system for the tasks in progress and queue only the missing ones. ```swift import Foundation // Fix: ask for the running ones, queue the missing ones func resumeDownloads(_ files: [URL], using session: URLSession) async { let running = Set(await session.allTasks.compactMap(\.taskDescription)) for file in files where !running.contains(file.lastPathComponent) { let task = session.downloadTask(with: file) task.taskDescription = file.lastPathComponent task.resume() } } ``` This also stays SOURCE. `taskDescription` is used here as the task identifier: it is written when queuing and read at launch. The `by-identifier` line in the measurement is the model of this: if the identifier is in the registry, queuing adds nothing, and however many times the operation is repeated, the registry stays the same. The robustness of the corrected function is against repetition; it is not against order. If a transfer is queued at the same moment it finishes, the result, as we saw in the measurement, depends on the order of the operations. #### The Session Lifecycle A background session is referred to by its identifier, and this identifier is the session's identity. When the app sets up a new session with the same identifier, the system attaches the tasks in progress that belong to that identifier to the new session. If the identifier is generated differently at every launch, this link cannot be made; the old tasks are left without an owner and the app starts with a new set of tasks. The `by-identifier` line in the measurement therefore depends on a condition: the registry matching the system's real state and the identifier being persistent. The lifecycle works like this. The app queues the task and moves to the background. The system runs the task. If the task finishes, or an event such as authentication is needed, the system launches the app in the background and delivers the events to the session's delegate. After the delegate processes the events, it calls the closure the system gave. After that the system may suspend the app again. When the user opens the app themselves, a session is set up with the same identifier and the pending events are delivered. #### The Check on Relaunch The first job of a relaunch is to ask what the system has been doing. The list of tasks in progress is obtained, results that have completed but have not yet been processed are delivered to the delegate, and the registry is updated with this information. Only after that are new tasks queued for the files that are still missing. This order is the practical counterpart of the difference between `queue-without-checking` and `by-identifier`: the first considers the list correct by itself, the second asks for the system's truth. Checking once is not enough. While the app stays open, tasks also complete, fail, or are cancelled. If the registry is not kept current with these events, the next launch checks with old information. This is the reason `finish` was put into the model: a finished transfer needs to leave the registry, otherwise the registry assumes a transfer that does not exist. The order sensitivity we see in the bounding measurement arises from the timing of this update: the finish notification can come before or after queuing, and it changes the result. #### Errors and Cancellation A background transfer can fail as well: the server may not respond, the connection may drop, or the file may be cut off halfway. A failed task is reported to the delegate with an error. In some errors the system gives, inside the error, the data needed to continue from where it was cut off. If the app stores this data and starts the task from where it left off, the same bytes are not downloaded again. An app that does not do this downloads a large file left half done from the beginning. When the user cancels a transfer, it also has to be removed from the registry. If the cancelled task stays in the registry, at the next launch the check thinks it is running and does not queue it again; the user asks when the file they cancelled will start downloading, or sees that the file never downloaded. This is another face of the principle that the record must reflect the state of the system. #### Verifying the Transferred File A download finishing does not mean the file is correct. The server may have returned an error page with a successful response, or the file may have been left incomplete. Before moving it, the response's status code is checked, the file size is compared with the expected one, and if possible a digest value is verified. If an unverified file is taken to the permanent location, corrupt data stays for a long time without being noticed. When the user opens a video and finds that it cannot be played, they cannot understand why the app downloaded the file corrupt. #### Showing Progress While a background transfer is running, the user may open the app and want to see the state of the transfer. The delegate reports the number of downloaded bytes at regular intervals; the app connects this information to a progress bar. Here too it should be kept in mind that the registry is not the single source of truth: when the app is relaunched, the progress information should be asked from the system again, because the bytes reported while the app was closed never reached the registry. Progress information is not always exact. If the server does not report the total size, a percentage cannot be calculated and the bar turns into an indeterminate indicator. In this case showing a made-up percentage misleads the user; saying "downloading" is more honest. Sending a notification when the transfer completes is also common, but permission for notifications belongs to the user, and the app should keep working in a situation where it cannot show notifications. #### Constraints Tied to Network Conditions The configuration of a background session also determines under what conditions the transfer will run. It is possible to choose downloading only on a wireless network, downloading on a connection that is not expensive, or downloading while charging. The tighter the constraint, the later the transfer starts. An app that promises a user a large file should state these conditions clearly; otherwise the user cannot understand why the transfer has not started. The result of the constraint also shows in the measurement: because it is not known when the transfer will finish, the registry can stay in the "running" state for a long time. This is not a faulty record, it is an expected situation. Reading the record as an error leads the user to queue the same file again, and to a duplicate transfer. #### Common Mistakes - Queuing the list on relaunch without asking. The transfer the system kept running is duplicated. - Not moving the downloaded file before the delegate method returns. The temporary location is deleted. - Generating the identifier differently at every launch. The app cannot reconnect to the old transfers. - Relying on values held in a closure in a background session. The app may have been terminated. - Predicting when the transfer will finish. That decision belongs to the system. #### Interpreting the Measured Result The same two approaches were compared in the same relaunch, and a single check created the difference: looking at the identifier. Note that the queuing operation did not raise an error anywhere; three transfer records are a legal state. The duplication problem, as we often see in the networking lessons, is silent. Keeping the registry by identifier appeared as a one-line rule that solves the duplication; but for this rule to work, the record must be kept consistent with the system's real state. #### The Bounding Measurement The model nowhere calculates when the system will finish the transfer or when it will wake the app. In the measurement below, the finish and queue operations are applied in two orders; the order is given to the model from outside. ```swift // main.swift — finish and queue, two orders let running = Registry(tasks: ["a.zip"]) let race = replay("finish-and-queue", from: running, finish("a.zip"), then: enqueueIfMissing(["a.zip"])).report() countSensitivities([race]) ``` ``` finish-and-queue | once=[] twice=[] ab=[a.zip] ba=[] -> order-sensitive robust: 0 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` It is order-sensitive: if the transfer finishes first and is queued afterward, the registry stays full; if it is queued first and finishes afterward, the registry empties. Which order will happen is not measured; in the model the order is an input, not a calculation. **The rule sees the order of the operations but does not see who determines the order.** **NW18.** When the system will continue the transfer, when it will wake the app, and under what conditions it will defer were not measured. This behavior depends on system conditions. #### Summary - In a background session the system runs the transfer; it continues even if the app is terminated by the system. - Re-queuing the list on relaunch without checking duplicated a transfer that was in progress. - Keeping the registry by task identifier stayed robust in the same relaunch. - The order of the finish and queue operations determined the result; the system determines the order, and it was not measured. - When the transfer finishes, a delegate receives the result, and the file must be moved from the temporary location. - This lesson's sensitivity: **repeat-sensitive**. #### Next Step The six lessons of the networking layer end here. The next topic moves to the concurrency layer and measures what running the same job twice, and interleaving two jobs, changes in a shared state. ## Concurrency Asynchronous functions, tasks and task groups in structured concurrency, actors that isolate shared state, the safety of interface updates on the main thread, the older queue-based model and reactive streams. ### Asynchronous Functions Source: https://academia.sh/en/courses/data-networking-and-concurrency/asynchronous-functions A measurement showing that when two operations that can suspend are awaited in sequence, the remaining stock depends on the order of reservation, while a read operation does not see the order. In the previous lesson, Background Transfers, we saw that when the app reopened and the download list was requeued without being checked, the same file was downloaded twice. The opening measurement of the networking layer had distributed six operations over four classes: one robust, two repeat-sensitive, two order-sensitive, one doubly sensitive. This topic carries the same question into concurrent code: if two operations are repeated or reordered, does the result change? The order is no longer only the order the programmer wrote; while one operation waits, another piece of work can run. The measurement uses the core file. This file is the same in every lesson of the topic. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` The core runs an operation four times: once, twice, `a` then `b`, and `b` then `a`. A difference between once and twice shows repeat sensitivity; a difference between the two orders shows order sensitivity. `replay` is not used for an asynchronous operation, because the functions passed to it cannot `await`. The four runs are built by hand with `await`, and the results go straight into `Trial`. The model builds what the operation does to the state; it is not the network or the runtime itself. #### A Function That Can Suspend An **asynchronous function** (an `async` function) is a function that, while running, accepts that it may stop at some point and wait for a result. The caller calls it with `await`. Every place where `await` is written is a **suspension point**: the function may stop there, give up control, and continue from where it left off once the result is ready. While it is suspended, the thread is not blocked; the system can run other work in the meantime. On the SOURCE side, a suspension point is typically on the network. The fragment below is not measured, because a real request goes out to the network. ```swift import Foundation func fetchStock(for sku: String) async throws -> Int { let url = URL(string: "https://example.com/stock/\(sku)")! let (data, _) = try await URLSession.shared.data(from: url) return Int(String(decoding: data, as: UTF8.self)) ?? 0 } ``` The single `await` in the fragment waits for the result of the request; `try` says that an error can propagate to the caller. The measured layer does not build this call. Instead it looks at what happens to the state: one shelf, two reservation requests, and the rule that no reservation is made when stock is insufficient. ```swift // Stock.swift — shelf stock: no reservation is made when stock is insufficient actor StockShelf { private(set) var quantity: Int private(set) var accepted: [Int] = [] init(_ quantity: Int) { self.quantity = quantity } func reserve(_ amount: Int) -> Int { if quantity >= amount { quantity -= amount accepted.append(amount) } return quantity } func remaining() -> Int { quantity } } func runReserve(_ amounts: [Int]) async -> Int { let shelf = StockShelf(10) var left = 10 for amount in amounts { left = await shelf.reserve(amount) } return left } func runReads(_ times: Int) async -> Int { let shelf = StockShelf(10) var seen = 0 for _ in 0.. order-sensitive stock-read | once=10 twice=10 ab=10 ba=10 -> robust robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` In the reservation line, `once=4`: six units were reserved and four remained. `twice=4`: the second six did not fit and was rejected, so repetition did not change the stock. `ab=4` and `ba=5`: when six was reserved first, five did not fit and four remained; when five was reserved first, six did not fit and five remained. The two orders left two different stocks, so the operation is **order-sensitive**. In the read line, all four values are ten: because a read does not touch the state, neither repetition nor order changes anything, and the operation is **robust**. The count is one robust, one order-sensitive. **CC1.** This measurement assumes that no other work comes between the two `await`s. Each run starts from its own shelf and the requests are processed in the order they are written; whether the runtime would change the order was not measured. #### Separate Shelves Order sensitivity is a property not only of the operation but of the state the operations share. To see this, we send the same two requests, six and five units, to two separate shelves this time. The pair gives which shelf each request goes to. ```swift // main.swift — the same two requests, two separate shelves func runShelves(_ requests: [(shelf: Int, amount: Int)]) async -> [Int] { let shelves = [StockShelf(10), StockShelf(10)] for request in requests { _ = await shelves[request.shelf].reserve(request.amount) } let firstLeft = await shelves[0].remaining() let secondLeft = await shelves[1].remaining() return [firstLeft, secondLeft] } let apartOnce = await runShelves([(0, 6)]) let apartTwice = await runShelves([(0, 6), (0, 6)]) let apartForward = await runShelves([(0, 6), (1, 5)]) let apartBackward = await runShelves([(1, 5), (0, 6)]) let apart = Trial(label: "separate-shelves", once: apartOnce, twice: apartTwice, forward: apartForward, backward: apartBackward).report() countSensitivities([apart]) ``` ``` separate-shelves | once=[4, 10] twice=[4, 10] ab=[4, 5] ba=[4, 5] -> robust robust: 1 repeat-sensitive: 0 order-sensitive: 0 doubly-sensitive: 0 ``` The second six is again rejected because it does not fit on the first shelf, and `once` and `twice` stay the same. `ab` and `ba` leave the same two shelves in the same state, because neither request changes something the other would see. The operation is **robust**. When the same six-and-five pair went to a single shelf it was order-sensitive; the difference is not in the requests but in whether they touch a shared state. In practice this tells you where to pay the cost of fixing the order: only for state that is truly shared. The count is one robust. **CC2.** The separate shelves were treated as completely independent. If there is a common warehouse or a total stock limit behind the two shelves, the separation would be misleading; no such common limit was measured. #### What Happens at a Suspension Point What the measurement does not see happens inside `await`. When a function is suspended, it is possible for another piece of work to reach the same actor or the same data. The work that ran first may have changed the state that the work running later sees; code that reads state before an `await` and uses it after the `await` needs care for this reason. **In which order** this interleaving happens depends on the system's scheduling, and it does not have to be the same in every run. This lesson does not run it: two sequential calls, with no interleaving, make a deterministic experiment. The practical rule is this: code that reads state and decides on it should not leave an `await` between the decision and the write. Otherwise the decision may no longer be true by the time it is written. In the shelf example, the `reserve` function does the check and the write in the same step and there is no `await` in between; so it is consistent on its own. Being consistent does not mean that the order of the two requests does not affect the result; the measurement showed exactly that. #### The Gap Between Decision and Write A setup that is not measured but is often seen is doing the check and the write with two separate `await`s. The function below first asks how many units are left on the shelf, then asks for the reservation. ```swift // Wrong setup: there is an await between the decision and the write func reserveCarelessly(_ amount: Int, on shelf: StockShelf) async -> Bool { let available = await shelf.remaining() guard available >= amount else { return false } _ = await shelf.reserve(amount) return true } ``` Between the two `await`s the shelf may have been changed by another request. Then `available` is no longer true, `reserve` silently rejects the request, and the function still returns `true`. The caller thinks the reservation was made, yet the stock never dropped. The fix is to gather the decision and the write in a single routine of the actor and return the result from that routine. ```swift // Routine added to StockShelf: decision and write in one step func tryReserve(_ amount: Int) -> Bool { guard quantity >= amount else { return false } quantity -= amount accepted.append(amount) return true } ``` The routine is not `async` and has no `await` inside; the actor finishes it without splitting it into another piece of work. The returned value says that the reservation was actually made. In sequential runs the two setups give the same result, so the difference does not show in the measurement; it shows only when interleaving occurs. Reading an unmeasured difference from the shape of the code is important for this reason. #### In a Real Application This pattern is met constantly in cart and booking screens. The user taps the "reserve" button twice, two screens look at the same product, or a refresh coming from the server races with a reservation the user has just made. In each case the operations themselves are correct; the problem is which one finishes first. The order in which network responses return does not have to match the order of the requests, and asynchronous code puts nothing on top of that order. So two questions are always asked together. What happens if the operation is repeated? What happens if two operations swap places? The answer to the first gives repeat sensitivity, and the answer to the second gives order sensitivity. Operations that do not change state, such as a read, are immune to both; this is why most screens share reads freely and gather writes in a single place. #### Common Mistakes - Assuming that writing `await` guarantees the order of the operation. `await` only says to call and wait; it does not determine the order of other calls. - Counting a read and a write as a single step. If there is a suspension point between them, the state may have changed in the meantime. - Forgetting error propagation. An `async` function marked `throws` requires `try` next to `await`; if `try` is omitted, the code does not compile. - Taking the order of the measurement as the real order. The four runs are two orders built by hand; they are not the order the runtime chose. #### The Bounding Measurement The rule compares whatever is in the state. A rule that looks only at the remaining stock does not see which requests were accepted. Below, six and four units are reserved; both fit and empty the shelf completely. ```swift // main.swift — the same two requests, two different state choices func runLedger(_ amounts: [Int]) async -> [Int] { let shelf = StockShelf(10) for amount in amounts { _ = await shelf.reserve(amount) } return await shelf.accepted } let leftSingle = await runReserve([6]) let leftRepeated = await runReserve([6, 6]) let leftForward = await runReserve([6, 4]) let leftBackward = await runReserve([4, 6]) let leftover = Trial(label: "remaining-stock", once: leftSingle, twice: leftRepeated, forward: leftForward, backward: leftBackward).report() let ledgerSingle = await runLedger([6]) let ledgerRepeated = await runLedger([6, 6]) let ledgerForward = await runLedger([6, 4]) let ledgerBackward = await runLedger([4, 6]) let ledger = Trial(label: "acceptance-ledger", once: ledgerSingle, twice: ledgerRepeated, forward: ledgerForward, backward: ledgerBackward).report() countSensitivities([leftover, ledger]) ``` ``` remaining-stock | once=4 twice=4 ab=0 ba=0 -> robust acceptance-ledger | once=[6] twice=[6] ab=[6, 4] ba=[4, 6] -> order-sensitive robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` The remaining stock is zero in both orders; the rule sees no difference and the operation comes out **robust**. The acceptance ledger, however, holds `[6, 4]` and `[4, 6]` in the two orders; the same two requests are this time **order-sensitive**. Both requests were the same; what changed is the state being compared. The count is one robust, one order-sensitive. **CC3.** This measurement counts the order of the acceptance ledger as part of the state. Whether the order carries meaning of its own, for example whether an invoice has a line order, was not measured. #### Reading Order and Expectation When reading the measurements in this lesson, it is enough to keep a single question in mind: if the same request runs twice, or two requests run in two orders, does the final state change? In the shelf example, requests made to different shelves do not affect each other, so order and repetition do not matter. Two requests coming to the same shelf change a shared state, and the result depends on the order of the calls. An asynchronous function does not remove this distinction; it only lengthens the interval in which two calls can get mixed up. Every `await` is a point where the function can be suspended and other work can come in between, and in code that touches shared state, knowing where these points are is the shortest way to predict in advance the sensitivity the measurement shows. #### Summary - An asynchronous function can wait at a suspension point; the caller calls it with `await`. - Two reservations that make no reservation on insufficient stock left different stock in two orders; the operation is order-sensitive. - A read operation that does not change state gave the same result in both orders and came out robust. - The rule sees only what is in the state: the remaining stock came out robust, the acceptance ledger order-sensitive. - Interleaving was not measured; the order is the order the lesson built. - This lesson's sensitivity: **order-sensitive**. #### Next Step If a function can wait while other work runs, there needs to be a way to start several pieces of work together. In the next lesson, it will be measured that the total stays robust while the results of a task group are collected, and that arranging the results in arrival order is order-sensitive. ### Tasks and Task Groups Source: https://academia.sh/en/courses/data-networking-and-concurrency/tasks-and-task-groups A measurement showing that when the results of jointly started work are collected as a set they are independent of arrival order, while collected as an array they depend on order. In the previous lesson, two reservation requests left different stock in two orders; the remaining stock came out the same in both orders, and only the operation that changed the state was order-sensitive. Now we ask the same question of code that starts several pieces of work together: whichever order the work finishes in, does the result we collect stay the same? The core file is the same in every lesson of the topic. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` This lesson measures two operations. The first is asynchronous, so its four runs are built by hand and given straight to `Trial`. The second only adds an item to an array in the state, so it is measured with `replay`. The model builds what the operation does to the state; it is not the network or the runtime itself. #### Three Task Forms There are three ways to start asynchronous work, and the difference is whose scope the work is tied to. **`async let`** starts child tasks whose number is known when the code is written: each `async let` spawns a child, and the result is taken when it is awaited. A **task group** is for children whose number is known only at run time; one child is added for each element of a list. An **unstructured task** (`Task { }`) is independent of the caller's scope; the task continues even if the caller finishes, and its cancellation is managed by hand. ```swift // async let: two child tasks of known number start together func loadDashboard() async throws -> (stock: Int, orders: Int) { async let stock = fetchStockCount() async let orders = fetchOrderCount() return try await (stock, orders) } func fetchStockCount() async throws -> Int { 12 } func fetchOrderCount() async throws -> Int { 7 } ``` The two children start at the same time; the `return` line waits for both. None of the children is left behind when the function finishes: when the scope ends, all of them have either finished or been cancelled. This is the guarantee of structured concurrency, and a task created with `Task { }` does not have it. A task left unattended can keep sending network requests even after the screen is closed. #### Task and Task Group A **task** (`Task`) is the unit that starts asynchronous work. In **structured concurrency** a task does not outlive its own scope: before the scope ends, all the child tasks inside it have finished. The most common tool for this is the **task group**: as many child tasks as you want are added inside the scope, and the results are collected in a loop. The children can run at the same time, so it is not known which finishes first. We do not bring this uncertainty into the measurement. Which child finishes first can change in every run, and building the output on it would make the measurement unstable. What is measured is whether the collected result depends on arrival order. ```swift // Groups.swift — the task group collects results in a set, arranges them on arrival func collectNames(_ names: [String]) async -> [String] { await withTaskGroup(of: String.self) { group in for name in names { group.addTask { name } } var collected = Set() for await name in group { collected.insert(name) } return collected.sorted() } } func record(_ name: String) -> ([String]) -> [String] { { list in list.contains(name) ? list : list + [name] } } ``` `collectNames` returns each name in a child task and receives them in arrival order with `for await`; but it puts the result into a `Set` and returns it sorted. A set keeps neither order nor repetition. `record`, on the other hand, is an operation that adds a name to the state: if the name is not in the list it appends it to the end, and if it is there it does not touch the list. #### Collecting in a Set and Collecting in an Array First the operation that collects into a set is measured in four runs: a single name, the same name twice, `alpha` then `beta`, `beta` then `alpha`. Then the same two names are collected into an array with `record`. ```swift // main.swift — the same two names, two ways of collecting let setOnce = await collectNames(["alpha"]) let setTwice = await collectNames(["alpha", "alpha"]) let setForward = await collectNames(["alpha", "beta"]) let setBackward = await collectNames(["beta", "alpha"]) let grouped = Trial(label: "group-set", once: setOnce, twice: setTwice, forward: setForward, backward: setBackward).report() let arrival = replay("arrival-order", from: [], record("alpha"), then: record("beta")).report() countSensitivities([grouped, arrival]) ``` ``` group-set | once=["alpha"] twice=["alpha"] ab=["alpha", "beta"] ba=["alpha", "beta"] -> robust arrival-order | once=["alpha"] twice=["alpha"] ab=["alpha", "beta"] ba=["beta", "alpha"] -> order-sensitive robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` In the set line, each of the four values is `["alpha"]` or `["alpha", "beta"]`: the repeated name melts away in the set, and the two orders give the same sorted list. The operation is **robust**. In the arrival-order line `once` and `twice` are the same, because `record` does not add an existing name again; but `ab` and `ba` differ. If `alpha` comes first the list is `["alpha", "beta"]`, and if `beta` comes first it is `["beta", "alpha"]`. The operation is **order-sensitive**. The count is one robust, one order-sensitive. **CC4.** This measurement counts the orders in which the children can finish as just two. In a real group, as the number of children grows, the number of possible orders grows too; not all of them were measured. **CC5.** The measurement assumes that all the child tasks return successfully. The case where a child throws an error or is cancelled is not in the measured model. #### Counting Instead of Collecting A set erases repetition; this is part of what makes the collection robust. If we write the numbers returned by the children not into a set but into a sum, repetition is no longer erased. The same measurement is made this time with a sum: one value, the same value twice, two different values in two orders. ```swift // main.swift — summing the group results func sumValues(_ values: [Int]) async -> Int { await withTaskGroup(of: Int.self) { group in for value in values { group.addTask { value } } var total = 0 for await value in group { total += value } return total } } let sumOnce = await sumValues([4]) let sumTwice = await sumValues([4, 4]) let sumForward = await sumValues([4, 5]) let sumBackward = await sumValues([5, 4]) let summed = Trial(label: "group-sum", once: sumOnce, twice: sumTwice, forward: sumForward, backward: sumBackward).report() countSensitivities([summed]) ``` ``` group-sum | once=4 twice=8 ab=9 ba=9 -> repeat-sensitive robust: 0 repeat-sensitive: 1 order-sensitive: 0 doubly-sensitive: 0 ``` In a sum, order does not matter: `ab` and `ba` are both nine. But `once` is four and `twice` is eight; adding the same value twice changes the sum. The operation is **repeat-sensitive**. In the previous set the same repetition melted away. So the two ways of collecting come from the same child tasks, and one is robust while the other is repeat-sensitive. The task group does not say which is correct; the meaning of the collection says it. A set is needed when counting unique users and a sum when adding order amounts, and the wrong choice silently produces a wrong result. **CC6.** The sum measurement used integer addition; integer addition is commutative and associative. The sum of floating-point numbers can differ in the last digit depending on order; this was not measured. #### Cancellation In task groups, **cancellation** is part of the structure. If a child throws an error and the group scope propagates it, the remaining children are cancelled; this can also be done by hand with `group.cancelAll()`. Cancellation does not kill a task, it tells the task that it is no longer wanted. The task itself checks for cancellation and exits cleanly. ```swift // Cancellation: when the group is cancelled, children can be cut short func fetchAll(_ ids: [Int]) async throws -> [Int] { try await withThrowingTaskGroup(of: Int.self) { group in for id in ids { group.addTask { try Task.checkCancellation() return id * 2 } } var results: [Int] = [] for try await value in group { results.append(value) if results.count == 2 { group.cancelAll() } } return results } } ``` `Task.checkCancellation()` in the fragment throws an error in a cancelled task. Cancellation can determine how many of the results are collected; which children get in at what position depends on timing. This is why the fragment is only read, not run. #### How Cancellation Works Cancellation is **cooperative**: the task carries a cancelled mark, but it does not stop by itself. The task's code checks this mark in two ways. `Task.checkCancellation()` throws an error and ends the function if the task has been cancelled. `Task.isCancelled` only returns a `Bool`; this way the task can leave a partial result and exit cleanly. The system's own asynchronous functions, such as a network request, already check for cancellation and throw an error when cancelled. Cancellation propagates downward. When a parent task is cancelled, its children are cancelled too; if a group scope exits by throwing an error, the children that have not yet finished are cancelled. It does not propagate upward: the cancellation of one child does not stop its siblings or its parent. If this asymmetry is not known, two kinds of mistakes appear. The first is not putting a cancellation check in a long child; the mark is carried and nobody looks at it. The second is mistaking cancellation for an error; a cancelled request should not be shown to the user as an error, because the user has already moved on to something else. #### Common Mistakes - Taking the finishing order of child tasks as the order of the results that arrive. The order is not the order in which the tasks were added. - Collecting the result as an array and forgetting to sort it. If order carries no meaning, a sorted set is a safe collection; if it does, the source of the order must be fixed separately. - Writing a long child that does not check for cancellation. Even if cancellation has been requested, the child continues until it finishes. - Trying to carry the task group outside its scope. The group lives only for its scope. #### When Order Is Needed If arrival order carries meaning, for example if the results are to be shown in a list in input order, the solution is to carry the index with the result. The children still finish in whatever order they like, but each one knows where it belongs. ```swift // If the order must be kept, the index is carried in the result func loadInOrder(_ ids: [Int]) async -> [Int] { await withTaskGroup(of: (Int, Int).self) { group in for (index, id) in ids.enumerated() { group.addTask { (index, id * 2) } } var slots = [Int?](repeating: nil, count: ids.count) for await (index, value) in group { slots[index] = value } return slots.compactMap { $0 } } } ``` Here the result no longer depends on arrival order but on input order. If the input list is fixed, the output is the same in every run. Order sensitivity does not disappear, its source changes: the order of the result is taken out of timing and carried into the input itself. The `arrival-order` line we measured classified arrival order; in the version written with an index, what is classified is now input order. This was not measured, it is only a conclusion drawn from the structure of the code. A common wrong setup is to append results to an array in the order they arrive without carrying an index, and to show this array on screen as if it were in input order. On the developer's machine the network is fast and steady, the children mostly finish in the order they were started, and the bug goes unnoticed. On a slow connection the third request returns before the first and the list swaps places before the user's eyes. The fix is to carry the position in the result as above; since the position is written once, the array comes out the same in every run. #### In a Real Application When a product page opens, the price, stock, and reviews are requested at the same time; the screen is drawn when all three arrive, or it shows each piece the moment it arrives. In the first case `async let` is enough. In the second, a separate task is needed for each piece, and the order in which the screen is updated is now a design decision. If the user leaves the page, all of these tasks must be cancelled; a task tied to a view (the `task` modifier in SwiftUI is an example) is cancelled when the view disappears, while an unattended `Task` is not. The choice of collection method follows from this too. Counters are collected with a sum, unique identifiers with a set, and the rows to be displayed with an array tied to input order. All three are built on the same task group; the difference is only in the line that collects the result, and the measurement makes this difference visible. #### The Bounding Measurement The same operation can fall into two different classes if it is compared with two different states. Take the arrival-order operation again: the same additions of `alpha` and `beta`, but this time we compare not the list as the state but only the number of elements in the list. ```swift // main.swift — the same operation, two state choices let listTrial = replay("list-view", from: [], record("alpha"), then: record("beta")) let listView = listTrial.report() let countTrial = Trial(label: "count-view", once: listTrial.once.count, twice: listTrial.twice.count, forward: listTrial.forward.count, backward: listTrial.backward.count) let countView = countTrial.report() countSensitivities([listView, countView]) ``` ``` list-view | once=["alpha"] twice=["alpha"] ab=["alpha", "beta"] ba=["beta", "alpha"] -> order-sensitive count-view | once=1 twice=1 ab=2 ba=2 -> robust robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` In the list view `ab` and `ba` differ and the operation is **order-sensitive**. In the count view both are two; the rule cannot see the difference and the operation comes out **robust**. The rule does not see the cause, it sees the inequality; the choice of the compared state changes the class. Before saying that a collection is robust, what was compared must be written down. The count is one robust, one order-sensitive. #### Which Form to Choose The choice comes down to three questions. If the number of children is known when the code is written, `async let` does the same job with the least noise. If the number is determined at run time, a task group is needed. If the work must not be tied to the lifetime of the caller, for example a load that will continue in the background after a user action, an unstructured task is used; but in that case cancellation and error handling are no longer the caller's responsibility but yours. The typical sign of a wrong choice is opening a `Task { }` for everything and writing the results to a shared variable. The scope guarantee is lost; which task will finish when, who will hear about it if it fails, and who will cancel it when the screen closes remain unclear; the three sensitivity questions of this lesson are reopened separately for each task. In short, the guarantee a task group gives is structural: no child leaks outside the group. Whether the result is meaningful is a decision made on the line that collects. #### Summary - A task group starts child tasks within a scope and waits for all of them to finish before the scope ends. - When the results are collected in a set and sorted, the operation is independent of arrival order and comes out robust. - When the same results are arranged in arrival order, the operation is order-sensitive. - The same operation looks robust if only the number of elements is compared; the choice of state changes the class. - Cancellation is part of the structure; if a child does not check for cancellation, it continues. - This lesson's sensitivity: **robust** (in a set) and **order-sensitive** (in an array). #### Next Step A collection hiding order does not mean that work sharing state is safe. In the next lesson we will see how actor isolation puts work that reaches the same data from several places into a line, and that the order between two requests remains measurable. ### Actors Source: https://academia.sh/en/courses/data-networking-and-concurrency/actors A measurement showing that actor isolation allows only one piece of work at a time, yet the order of two requests and the repetition of a counter still change the result. In the previous lesson, collecting the results of a task group in a set erased arrival order, while pouring them into an array wrote the order into the result. There the problem was the form of the collection. In this lesson the problem is the state itself: if several pieces of work reach the same data, what does the guarantee given by isolation cover, and what does it not cover? The core file is the same in every lesson of the topic. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` Actor operations are asynchronous; calls require `await`. For this reason each run is built by hand instead of with `replay` and given straight to `Trial`. The model builds what the operation does to the state; it is not the network or the runtime itself. #### Actor and Isolation An **actor** (`actor`) is a reference type that protects its own mutable state. The actor's state is not touched directly from outside; access is made by entering one of the actor's functions with `await`, and the actor runs only one piece of work at a time. This is called **actor isolation**. What isolation gains is the prevention of a **data race**: two pieces of work cannot read and write the same memory area at the same time. The difference between this and a **race condition** is important. A data race is an error at the memory level, and isolation removes it. A race condition is the result depending on the order of the work, and isolation does not solve this: the two pieces of work run one after the other, one at a time, safely; but which one ran first still changes the result. ```swift // Seats.swift — seat map and ticket counter actor SeatMap { private(set) var owner = "-" func claim(_ name: String) -> Bool { if owner == "-" { owner = name return true } return false } func currentOwner() -> String { owner } } actor TicketCounter { private(set) var sold = 0 func sell() { sold += 1 } func total() -> Int { sold } } func runClaims(_ names: [String]) async -> String { let map = SeatMap() for name in names { _ = await map.claim(name) } return await map.currentOwner() } func runSales(_ times: Int) async -> Int { let counter = TicketCounter() for _ in 0.. order-sensitive ticket-sale | once=1 twice=2 ab=2 ba=2 -> repeat-sensitive robust: 0 repeat-sensitive: 1 order-sensitive: 1 doubly-sensitive: 0 ``` In the seat line, the second claim by the same name does not change `alice`, because the second claim is rejected; `once` and `twice` are the same. But `ab` is `alice` and `ba` is `bob`: the first to arrive wins, and who arrives first is written into the result. The operation is **order-sensitive**. In the counter line `once=1` and `twice=2`: every call increases the counter, and repetition changes the result. Order does not affect the counter; since ticket sales are commutative, `ab` and `ba` are the same. The operation is **repeat-sensitive**. The count is one repeat-sensitive, one order-sensitive. **CC7.** This measurement models a single owner field for the seat. How several seats or a waiting list would affect the same rule was not measured. **CC8.** In the counter measurement, order means the order of two identical `sell` calls. The case where two operations of different kinds are used together was not measured. #### Why Actors Exist Before actors, the way to protect shared state was a lock: every piece of code that read or wrote the state first took a lock and released it when it finished. The method works, but the protection depends on the programmer's discipline. A single line that forgets to take the lock invalidates the whole protection, and the compiler does not notice. The bug also rarely shows itself: when two pieces of work hit that line at the same time, mostly only under load, sometimes only on one customer's device. The actor moves the protection into the language itself. The state is inside the actor and the only way to reach it is the actor's functions; the compiler rejects direct access from outside. Forgetting the lock becomes impossible, because the actor does the locking itself. In return there is a cost: every entry into an actor is an `await` point, and the caller waits in line if the actor is busy. This wait does not block the caller; the caller is suspended and the thread is given to other work. For this reason an actor is suited not to long and slow work but to short and frequent state changes. The typical wrong setup is trying to make the same protection with a class and skipping the protection. ```swift // Wrong: a mutable class, no protection final class UnsafeTicketCounter { var sold = 0 func sell() { sold += 1 } } ``` This class works correctly when used from a single flow. When two tasks call `sell` on the same instance at the same time, however, `sold += 1` consists of the steps read, increment, and write, so the two can interleave and one sale is lost. Since the class is not Sendable, the Swift 6 language mode should not let you share this example between two tasks; this compiler behaviour was not measured here. The fix is to make the class an `actor`; the `TicketCounter` above is exactly this, and its body does not change. #### Sendable Types The state inside an actor is isolated, but the values entering and leaving the actor cross the boundary. For a value to be sent safely across concurrent boundaries, its type must be **Sendable**. Value types and immutable values generally meet this condition on their own; mutable reference types do not, and the compiler reports this as an error. ```swift // Sendable type: an immutable value that can be sent between actors struct Reservation: Sendable { let seat: Int let owner: String } func describe(_ reservation: Reservation) -> String { "seat \(reservation.seat) -> \(reservation.owner)" } ``` `Reservation` carries only constant fields, so it is sent safely. Immutable values are copied or shared as they cross the boundary; nobody can change them at the same time. This fragment is not measured, because it shows what no operation does to the state; it only explains the type rule. #### Releasing the Seat Until now the seat was only won. In a real reservation the seat is also given back, and releasing, together with winning, brings a second form of order sensitivity. There is a rule here: only the current owner of the seat can release it; someone else's release request is silently rejected. ```swift // Release.swift — a releasable seat actor ReleasableSeat { private(set) var owner = "-" func claim(_ name: String) { if owner == "-" { owner = name } } func release(_ name: String) { if owner == name { owner = "-" } } } func runSeatOperations(_ operations: [(isRelease: Bool, name: String)]) async -> String { let seat = ReleasableSeat() for operation in operations { if operation.isRelease { await seat.release(operation.name) } else { await seat.claim(operation.name) } } return await seat.owner } ``` The measured pair is claiming and releasing for `alice`. Four runs: a single claim, two claims, claim then release, release then claim. ```swift // main.swift — claiming and releasing let releaseOnce = await runSeatOperations([(false, "alice")]) let releaseTwice = await runSeatOperations([(false, "alice"), (false, "alice")]) let releaseForward = await runSeatOperations([(false, "alice"), (true, "alice")]) let releaseBackward = await runSeatOperations([(true, "alice"), (false, "alice")]) let released = Trial(label: "release", once: releaseOnce, twice: releaseTwice, forward: releaseForward, backward: releaseBackward).report() countSensitivities([released]) ``` ``` release | once=alice twice=alice ab=- ba=alice -> order-sensitive robust: 0 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` When the same claim is made a second time the result does not change: `once` and `twice` are both `alice`. But claiming and then releasing empties the seat (`-` for `ab`), while releasing and then claiming gives the seat to `alice` (`ba`). A release that comes before the claim has no effect, because there is no ownership to release. The operation pair is **order-sensitive**. This is the model of a familiar bug on reservation screens: the user chooses a seat and immediately changes their mind; if the two requests swap places on the network, the seat appears to belong to the user even though they gave it up. **CC9.** In this measurement, release was built as a request made with the name of the current owner only. Another user's release request and automatic release on timeout were not measured. #### In a Real Application Think of a seat selection screen in an events app. The user taps a seat, the app sends a request to the server and shows the result on screen. On the client side an actor holds the state of the seats: which seat is free, which is selected, which is still waiting for a response from the server. The user acts quickly, selects a seat and releases it, and then selects another; that is, several requests are sent one after another in a short time. The actor keeps the state from being corrupted in the middle of these requests. Which request is processed first, however, is decided not by the actor but by the order in which the requests reach the actor. This distinction shows on screen as two separate bugs. With a data race the app would crash with memory corruption; the actor prevents this. With an order bug the app does not crash, it only shows the wrong seat as selected; the actor does not prevent this, because both requests are valid separately. For this reason, in flows that need order, one of two ways is chosen: the requests are sent from a single task, one after another with `await`, or each request carries a sequence number and the actor rejects a request with an old number. A common setup mistake is doing the check outside the actor and the action inside it. ```swift // Wrong: other work can come in between the check and the action func claimIfFree(_ map: SeatMap, _ name: String) async { if await map.currentOwner() == "-" { _ = await map.claim(name) } } ``` Another task can enter the actor between the two `await`s: both calls see the seat as free, and both call `claim`. Since `SeatMap.claim` checks and writes inside itself, the second call is rejected and the result stays correct in this example; but when the rule is moved outside the actor, for example when an action with a side effect such as "if free, send a payment request" is added, the payment can be sent twice. The fix is to combine the check and the action in a single function of the actor; `claim` already does this. This fragment was not measured, because the sequential run of this lesson cannot build an execution where two calls interleave. #### Common Mistakes - Thinking the actor is immune to race conditions. An actor prevents a data race; it does not fix the order of two pieces of work. - Reading the state inside an actor and writing it after an `await`. Another piece of work may have come in at the suspension point. - Trying to reach the actor's state directly from outside. Access is made with `await`; the compiler enforces this. - Giving an actor a reference type that is not Sendable. The type of every value crossing the boundary must be Sendable. #### The Bounding Measurement Reentrancy is the ability of another piece of work to enter the same actor when the actor is suspended at an `await` point. The seat map below writes the claim after a suspension point. In the sequential run the same calls run one after another and no work comes in between; the measurement does not see this. ```swift // main.swift — a suspending seat map, the same four runs actor SlowSeatMap { private(set) var owner = "-" func pause() async {} func claim(_ name: String) async -> Bool { if owner == "-" { await pause() owner = name return true } return false } } func runSlowClaims(_ names: [String]) async -> String { let map = SlowSeatMap() for name in names { _ = await map.claim(name) } return await map.owner } let slowOnce = await runSlowClaims(["alice"]) let slowTwice = await runSlowClaims(["alice", "alice"]) let slowForward = await runSlowClaims(["alice", "bob"]) let slowBackward = await runSlowClaims(["bob", "alice"]) let slow = Trial(label: "slow-seat", once: slowOnce, twice: slowTwice, forward: slowForward, backward: slowBackward).report() countSensitivities([slow]) ``` ``` slow-seat | once=alice twice=alice ab=alice ba=bob -> order-sensitive robust: 0 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` A suspension point was added and the line is the same output as `seat-claim`. The rule does not see that no other work came between the `await`s; it only looks at whether the states are equal at the end of the four runs. On top of that, the actor is free of data races and this does not make the result robust: the same line stays **order-sensitive**. The rule does not see the cause, it sees the inequality. The case where two requests suspend at the same time and both see the seat as free was not built here and was not measured; in that case the seat would have been won twice. The count is one order-sensitive. **CC10.** Another piece of work coming in at a suspension point was not built in any measurement in this lesson. Reentrancy is only explained here. As a rule, if the actor's function decides and writes in a single step, the state stays consistent; if you make the decision outside and have the actor only write, consistency returns to being your responsibility. This distinction looks small, but in areas such as reservations, stock, and balances it determines where the bug will appear. #### Summary - An actor isolates its state and allows only one piece of work at a time; this prevents a data race. - Isolation does not fix the order of two pieces of work: the seat map that accepts the first claim is order-sensitive. - A counter that increases on every call is repeat-sensitive and does not see the order. - Sendable types cross the boundary safely; mutable references cannot. - Adding a suspension point does not change the class in a sequential measurement; interleaving was not measured. - This lesson's sensitivity: **order-sensitive** (seat) and **repeat-sensitive** (counter). #### Next Step Protecting state inside actors is one thing, protecting the interface's state is another. In the next lesson we will measure code that is tied to the main thread, and a model in which search results are written to the screen in arrival order. ### Main Thread Isolation Source: https://academia.sh/en/courses/data-networking-and-concurrency/main-thread-isolation A measurement showing that the operation updating the list the interface shows depends on arrival order without a version number and becomes robust with one. In the previous lesson, actor isolation prevented a data race but did not fix the order of two requests. The list a screen shows is also shared state of this kind: new requests go out as the user types in the search box, and the responses are written to the screen in whatever order they arrive. In this lesson we measure this situation. The core file is the same in every lesson of the topic. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` In this lesson the operations only write a list to the state and are not asynchronous; so they are measured with `replay`. The model builds what the operation does to the state; it is not the network or the screen itself. Which thread runs which piece of work is not measured, because the measurement cannot see it. #### Main Thread and Main Actor The interface is updated only on the **main thread**. In concurrent code this rule is expressed with the **main actor** (`@MainActor`): the state of a type tied to the main actor is reached only from the main actor, and the compiler checks this. When a piece of work in the background wants to write its result to the screen, it moves to the main actor with `await`. This protects the consistency of the interface but does not solve the race. If two search requests are running in the background, the results of both eventually arrive at the main actor, and whichever arrives later is the one that stays on screen. The order of responses can differ from the order of requests. ```swift // Shown.swift — the list shown on screen and two ways of updating it struct ShownList: Equatable, CustomStringConvertible { var version: Int var items: [String] var description: String { "v\(version) \(items)" } func overwriting(items newItems: [String]) -> ShownList { ShownList(version: version, items: newItems) } func applying(version newVersion: Int, items newItems: [String]) -> ShownList { newVersion > version ? ShownList(version: newVersion, items: newItems) : self } } ``` `overwriting` writes the incoming list to the screen without looking at the version. `applying` writes only if the incoming version is greater than the shown version; if a response with an old version arrives, it ignores it. #### Unversioned and Versioned Updates There are two responses: the result of the old query (version 1, `old`) and the result of the new query (version 2, `new`). `a` writes the old response and `b` writes the new one. Unversioned and versioned writes are measured separately in the two orders. ```swift // main.swift — the same two responses, unversioned and versioned writes let start = ShownList(version: 0, items: []) let blind = replay("unversioned-write", from: start, { $0.overwriting(items: ["old"]) }, then: { $0.overwriting(items: ["new"]) }).report() let checked = replay("versioned-write", from: start, { $0.applying(version: 1, items: ["old"]) }, then: { $0.applying(version: 2, items: ["new"]) }).report() countSensitivities([blind, checked]) ``` ``` unversioned-write | once=v0 ["old"] twice=v0 ["old"] ab=v0 ["new"] ba=v0 ["old"] -> order-sensitive versioned-write | once=v1 ["old"] twice=v1 ["old"] ab=v2 ["new"] ba=v2 ["new"] -> robust robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` In the unversioned write, repeating changes nothing: `once` and `twice` are the same. But `ab` is `new` and `ba` is `old`: the last to arrive stays on screen. If the response of the old query arrives late, it overwrites the new result. The operation is **order-sensitive**. In the versioned write, the old response is ignored even if it arrives after the new one; both orders give `v2 ["new"]`, and repeating changes nothing either. The operation is **robust**. The count is one robust, one order-sensitive. **CC11.** The measurement assumes that responses are not lost, only that their order can change. The case where a response never arrives or is an error was not measured. **CC12.** The version numbers are assumed to be given as 1 and 2 in request order. Who produces the number is dealt with in the bounding measurement below. #### Why the Screen Is Updated in One Place The system that draws the screen works on a single flow: while deciding what will appear where in a frame, the data being shown must not change at that moment. If another piece of work added an element to the list while the list was being drawn, the drawing would produce either the old, the new, or a half-finished picture. For this reason every operation that touches the interface is gathered in a single place, the main thread, and runs in order. The main actor is the form of this rule checked by the compiler: touching a type marked `@MainActor` from elsewhere without `await` is a compile error. What the main actor gains is briefly this: the code that writes the screen's state always runs in the same place, one piece at a time. What it loses comes from the same rule: every long piece of work done on the main actor keeps the screen waiting. This is why the work is split. The network request, parsing, and sorting are done in the background; only the last step, writing the ready list to the screen state, happens on the main actor. The measurements in this lesson model that last step. #### Updating on the Main Actor The counterpart of the same rule in real code is the following. The fragment is not measured, because screen state tied to the main actor gives no meaningful output in the sandbox. ```swift import Foundation @MainActor final class SearchResults { private(set) var items: [String] = [] private var shownVersion = 0 func apply(version: Int, items newItems: [String]) { guard version > shownVersion else { return } shownVersion = version items = newItems } } ``` `apply` accepts only a newer version. The version number is produced in the code that starts the request: every time a search starts, a counter goes up by one and the number is tied to the request. When the response arrives, it carries its number. #### The Loading Indicator Search screens also have a spinning indicator: it is visible while the request is going out and disappears when the response arrives. Holding the indicator with a single `Bool` is the setup that comes to mind first, and it works correctly when there is one request. When two requests overlap, however, the first response turns the indicator off, and the screen looks finished while the second request is still running. The alternative is to hold the number of requests in flight: it goes up by one at the start and down by one at the response, and the indicator is visible as long as the number is greater than zero. The two setups are applied to the same pair, that is, to the start and finish operations separately. The measurement tests a single start with a single finish; it does not build two overlapping requests directly. ```swift // main.swift — loading indicator: flag and counter let flag = replay("flag", from: false, { _ in true }, then: { _ in false }).report() let counter = replay("counter", from: 0, { $0 + 1 }, then: { max($0 - 1, 0) }).report() countSensitivities([flag, counter]) ``` ``` flag | once=true twice=true ab=false ba=true -> order-sensitive counter | once=1 twice=2 ab=0 ba=1 -> doubly-sensitive robust: 0 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 1 ``` In the flag line, repeating changes nothing, because the second start finds the indicator already on. But `ab` and `ba` split: if the finish comes before the start, the indicator stays on. The flag is **order-sensitive**. The counter line came out **doubly sensitive**, and this is not a defect but the correct result of what a counter is: every start really means one more request, so two starts differ from one start. The correct reading of sensitivity is this: a class is not "good" or "bad", it shows what the operation claims. The counter counts two starts as two requests, and it has to; the error appears when the same start notification is repeated and applied twice. Repeated notifications are seen in network layers and event publishers; an indicator that works with a counter stays on forever in this case. **CC13.** The flag and counter measurement builds a single start and a single finish. The four-operation orderings of two overlapping requests (start, start, finish, finish) were not measured. #### In a Real Application On a search screen a request goes out each time the user presses a letter. The most common setup is to start an independent task at every change and write the result directly to the screen state. ```swift // Wrong: every letter opens a new task, the old task is not cancelled @MainActor final class SearchModel { private(set) var results: [String] = [] func queryChanged(_ text: String) { Task { let found = await search(text) results = found } } } ``` This code is the unversioned write itself. For a user who types fast, the requests `a`, `ab`, and `abc` go out one after another; on a slow network, if the `ab` response arrives after the `abc` response, the screen shows the `ab` results while it says `abc`. On the developer's machine the network is fast and regular and the bug is not visible; in production, on a mobile connection, it is. There are two ways to fix it, and most of the time both are used together. The first is to cancel the previous task: the task's handle is stored, the old one is cancelled when a new letter arrives, and the old task checks whether it has been cancelled before writing the result. The second is the version number, as this lesson measured: the number is given when the request starts, that is, on the main actor and in order, it returns with the response, and `apply` goes on discarding the older number. Cancellation stops unnecessary network work; the version discards a response that reached the screen despite cancellation. A task that does not check for cancellation can carry the mark and finish its work, so counting cancellation alone as enough is also a setup mistake. This fragment was not measured: the timing of a real `Task`, network latency, and at which point of the task the cancellation shows cannot be built in a sequential run. What is measured is the writing rule itself. #### Common Mistakes - Trying to change the screen state directly from background work. You need to move to the main actor with `await`. - Confusing the moment the result is written to the screen with the moment the request starts. The write happens when the response arrives. - Writing an old response on top of the new one. Nothing prevents this without a version or cancellation. - Making the main actor the workplace of a long computation. The screen stays unresponsive during that time. #### The Bounding Measurement For the version to work depends on who gives the number. If the code that starts the request gives the number, the old response carries the old number and is discarded. If the screen gives the number when the response arrives, every response looks newer than the previous one. ```swift // main.swift — if the version number is given on arrival let stampedStart = ShownList(version: 0, items: []) let stamped = replay("stamped-on-arrival", from: stampedStart, { $0.applying(version: $0.version + 1, items: ["old"]) }, then: { $0.applying(version: $0.version + 1, items: ["new"]) }).report() countSensitivities([stamped]) ``` ``` stamped-on-arrival | once=v1 ["old"] twice=v2 ["old"] ab=v2 ["new"] ba=v2 ["old"] -> doubly-sensitive robust: 0 repeat-sensitive: 0 order-sensitive: 0 doubly-sensitive: 1 ``` The same two responses, the same `applying` rule; only the number changed. Because every response got a number one higher on arrival, none was discarded. `once=v1` and `twice=v2` differ: the second application increased the version again and the operation became repeat-sensitive. `ab` and `ba` differ too. The operation is **doubly sensitive**. The rule does not see the cause, it sees the inequality; there was a version check, but the code that gave the number in the wrong place made it ineffective. The count is one doubly sensitive. **CC14.** This measurement is a model that increases the version with a single step on the arrival of a response. The case where the number is produced not from a counter but from a timestamp was not measured. #### When No Response Arrives The measurements assume that every response arrives in the end. On a real network a response may never arrive or may return with an error, and this is a separate decision for the screen. The most common setup mistake is swallowing the error and leaving the indicator on: the user looks at an indicator that spins forever. The error state must also carry a version, because a late error from an old request must not write an error message on top of the successful result of the new request. The rule is the same: every path that writes to the screen state asks, before writing, whether its own request is still current. The practical consequence of this is to pass the screen state through a single writing gate. Success, error, and cancellation come to the same function, and the function checks the version once. Code that writes three separate checks in three separate places will one day forget one of them; the forgotten place is usually the path seen most rarely, that is the error path, and nobody tries it during development. This is why our measurements also start with a simple model of two responses: first make the basic rule visible, then leave to the reader the question of which paths the rule will be applied to. The way to test the screen's resilience is to send the responses deliberately in reverse order and with a delay. A developer setting that slows down the network, or a fake server that changes the response order, reveals the unversioned write within minutes. This test does not replace the measurement; the measurement shows why the rule works, and the test shows that the app really applies the rule. #### Keeping the Main Actor Light The main actor is a single road, and the more work is done on it, the later the screen responds. A practical measure is that the time a piece of work spends on the main actor stays shorter than the duration of one frame; that duration depends on the refresh rate of the screen. Every piece of work that goes above this shows up as stutter during scrolling. Sorting a large list, decoding an image, or parsing a response exceeds this threshold quickly; these jobs are done in the background and only the ready result is carried to the main actor. There is a subtle point here: when the result is carried to the main actor, the version must be carried too. The background work takes the version number when it starts, returns to the main actor with this number when it finishes, and the main actor compares the number with the current number. Producing the number at the moment the background work finishes is the same mistake as in the last measurement of the lesson. So the source of the number is always the place where the work started. The number is taken once at the start, does not change during the work, and returns with the result; this way the code writing to the screen always knows which request's response it carries. Without this information no check would be reliable. #### Summary - Interface state is tied to the main thread; in concurrent code this is expressed with the main actor. - An unversioned write is sensitive to arrival order; an old response can overwrite the new one. - A version check discards the old response and makes the write robust. - Giving the number on the arrival of the response leaves the check ineffective and makes the operation doubly sensitive. - Which thread runs which piece of work was not measured. - This lesson's sensitivity: **order-sensitive** (unversioned) and **robust** (versioned). #### Next Step The main actor and actors are the building blocks of new code; but old code is still written with dispatch queues. In the next lesson we will measure adding work to a serial queue and blocks that assign values, and look at what the queue puts in order. ### Queue-Based Concurrency Source: https://academia.sh/en/courses/data-networking-and-concurrency/queue-based-concurrency A measurement showing that jobs added to a serial queue keep their order, but that appending and assigning a value fall into different classes under repetition and order. In the previous lesson, a version check kept a late, old response from overwriting the new result, while giving the number in the wrong place made the check ineffective. Before concurrent Swift was written, work was run with **dispatch queues**, and a great deal of code is still written that way. In this lesson we measure two kinds of jobs added to a queue. The core file is the same in every lesson of the topic. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` In this lesson the operations are added to a queue, the queue is drained, and the result is read. The four runs are each set up on a new queue and given directly to `Trial`. The model builds what the operation does to the state; it is not the network or the runtime itself. #### Serial and Concurrent Queues A **dispatch queue** runs the jobs added to it on a pool of threads. A **serial queue** runs jobs in the order they were added and one at a time: the next one does not start until one finishes. A **concurrent queue** can start jobs at the same time; only the start order is the order of addition, and the finish order is not defined. The result a serial queue produces is deterministic if the order of addition is known, and so it can be measured. On a concurrent queue the finish order depends on timing, so it does not enter the measurement. In this lesson only the serial queue is measured. ```swift // Queued.swift — append and assign operations running in order on a serial queue import Dispatch final class Cell: @unchecked Sendable { var lines: [String] = [] var current = "-" } func makeSerialQueue() -> DispatchQueue { .init(label: "serial") } func runAppends(_ labels: [String]) -> [String] { let cell = Cell() let queue = makeSerialQueue() for label in labels { queue.async { cell.lines.append(label) } } queue.sync {} return cell.lines } func runAssigns(_ labels: [String]) -> String { let cell = Cell() let queue = makeSerialQueue() for label in labels { queue.async { cell.current = label } } queue.sync {} return cell.current } ``` `Cell` is the shared state the queue works on; it carries `@unchecked Sendable` to tell the compiler "I am protecting this", so the guarantee comes from the queue, not from the compiler. `runAppends` adds each label to the queue as a job. `runAssigns` writes each label to a single field. At the end an empty `sync {}` block drains the queue: on a serial queue this block runs after all the jobs added, so by the time it returns, all of them have finished. #### Appending and Assigning a Value We measure two jobs, labeled `a` and `b`, with two separate operations: appending and assigning a value. ```swift // main.swift — appending to a queue and assigning a value let appendOnce = runAppends(["a"]) let appendTwice = runAppends(["a", "a"]) let appendForward = runAppends(["a", "b"]) let appendBackward = runAppends(["b", "a"]) let appending = Trial(label: "append", once: appendOnce, twice: appendTwice, forward: appendForward, backward: appendBackward).report() let assignOnce = runAssigns(["a"]) let assignTwice = runAssigns(["a", "a"]) let assignForward = runAssigns(["a", "b"]) let assignBackward = runAssigns(["b", "a"]) let assigning = Trial(label: "assignment", once: assignOnce, twice: assignTwice, forward: assignForward, backward: assignBackward).report() countSensitivities([appending, assigning]) ``` ``` append | once=["a"] twice=["a", "a"] ab=["a", "b"] ba=["b", "a"] -> doubly-sensitive assignment | once=a twice=a ab=b ba=a -> order-sensitive robust: 0 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 1 ``` In the append line, `once` and `twice` differ: the second append writes one more line to the list. `ab` and `ba` differ too: the list carries the order of appending. The operation is **doubly sensitive**. In the assignment line, `once` and `twice` are the same, because assigning the same value again changes nothing. But `ab` is `b` and `ba` is `a`: the last one assigned wins. The operation is **order-sensitive**. The serial queue kept the order; but because the order was kept, the result depended on the order. The count is one order-sensitive, one doubly sensitive. **CC15.** This measurement assumes that the jobs added run on a single queue, in the order they were written. The case where several queues reach the same `Cell` was not measured. **CC16.** Two append jobs with the same label count as the same job. The case where the two jobs can be told apart from each other and carry different labels exists only in the `ab` and `ba` lines. #### Eliminating Repeats Part of the double sensitivity comes from repetition: when the same label is appended twice, the list grows. In practice this is often unwanted; a list recording that a notification was sent should not contain the same notification twice. The remedy is to check whether the label is in the list before appending. What matters is that the check and the append are done in the same job of the queue; if the two are separate jobs, another job can slip in between them. ```swift // Unique.swift — a queue operation that appends only a new label import Dispatch func runUniqueAppends(_ labels: [String]) -> [String] { let cell = Cell() let queue = makeSerialQueue() for label in labels { queue.async { if !cell.lines.contains(label) { cell.lines.append(label) } } } queue.sync {} return cell.lines } ``` ```swift // main.swift — an append that eliminates repeats let uniqueOnce = runUniqueAppends(["a"]) let uniqueTwice = runUniqueAppends(["a", "a"]) let uniqueForward = runUniqueAppends(["a", "b"]) let uniqueBackward = runUniqueAppends(["b", "a"]) let unique = Trial(label: "unique-append", once: uniqueOnce, twice: uniqueTwice, forward: uniqueForward, backward: uniqueBackward).report() countSensitivities([unique]) ``` ``` unique-append | once=["a"] twice=["a"] ab=["a", "b"] ba=["b", "a"] -> order-sensitive robust: 0 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` The second append of the same label no longer changes the list; `once` and `twice` are equal. But `ab` and `ba` are still different, because the list carries the order of appending. The operation is **order-sensitive**. From the double sensitivity of the previous line, the repeat sensitivity has gone and the order sensitivity has stayed. This shows what eliminating repeats solves and what it does not: a repeat no longer leads to a copy, but the question of which label will come first is not answered either. **CC17.** Eliminating repeats relies on the exact equality of labels. Labels that mean the same but are written differently (for example, a difference in letter case) were not measured. #### Concurrent Queue and Barrier On a concurrent queue reads can run together, but a write must run alone. For this, the write job carries a **barrier** flag: a job with a barrier does not run at the same time as any job that is ahead of it or behind it in the queue. ```swift // Concurrent queue and barrier: reads together, write alone let shared = DispatchQueue(label: "shared", attributes: .concurrent) nonisolated(unsafe) var values: [String] = [] func read() -> Int { shared.sync { values.count } } func write(_ value: String) { shared.async(flags: .barrier) { values.append(value) } } ``` The barrier prevents a write from overlapping with reads. Which read will run before or after which write still depends on the order of addition and on timing; for this reason the fragment is not measured. #### In a Real Application In old code a queue is mostly used in place of a lock. A class has a shared dictionary; every read and write is sent to the same serial queue, and the dictionary is touched only from that queue. This setup has worked for decades in structures such as caches and session information. The queue keeps the order and runs only one job at a time; that is, it does exactly what the previous two lessons measured, in its form from before actors. The real trap of the queue is that the protection depends on discipline. Code that touches the dictionary from outside the queue with a single line makes the whole protection void, and the compiler does not say so. Another common mistake is writing to the queue and trying to read the result at the same moment. ```swift // Wrong: the write is in the queue, the read is outside the queue func record(_ label: String, in cell: Cell, on queue: DispatchQueue) -> Int { queue.async { cell.lines.append(label) } return cell.lines.count } ``` The `async` call leaves the job in the queue and returns immediately; the job may not have run yet. The number returned is sometimes the count from before the append and sometimes from after it, and which one it is depends on timing. On top of that, because the read is done outside the queue, it can run at the same time as the job writing to `cell.lines`; this is a data race. The fix is to give the read to the same queue as well and wait for the result with `sync`. But entering a queue with `sync` from the main thread keeps the screen waiting as long as that queue takes; for this reason, in new code the same job is written with an actor and the read is done with `await`. Another trap is entering the same queue with `sync` from inside the queue: the queue waits for its own running job to finish, and the job waits for the queue to empty. For this reason, helper functions that do not know which queue the caller is on should take a callback with `async` instead of `sync`. The re-reading of the measurements is this: a serial queue gives a guarantee of order, but this guarantee does not make the result independent of order. Where order matters, the queue may be enough; where it does not matter, what will make the result independent of order is the operation itself, that is, merging instead of assigning a value, a set instead of appending. The queue protects, the shape of the operation makes it robust. #### Common Mistakes - Taking a queue's keeping the order for the result being robust. The order is kept, but the result can still depend on the order. - Adding a job to the same queue from inside it with `sync`. The queue waits for itself and deadlocks. - Writing to a concurrent queue without a barrier. Reads overlap with the write. - Reading the result without draining the queue. The append jobs may not have finished yet. #### The Bounding Measurement This time we compare the same append operation with the sorted form of the appended lines and with the line count. The operation did not change; only the view of the state changed. ```swift // main.swift — the same append, three choices of state let rawTrial = Trial(label: "raw-list", once: runAppends(["a"]), twice: runAppends(["a", "a"]), forward: runAppends(["a", "b"]), backward: runAppends(["b", "a"])) let raw = rawTrial.report() let sortedTrial = Trial(label: "sorted-list", once: rawTrial.once.sorted(), twice: rawTrial.twice.sorted(), forward: rawTrial.forward.sorted(), backward: rawTrial.backward.sorted()) let sortedView = sortedTrial.report() let sizeTrial = Trial(label: "line-count", once: rawTrial.once.count, twice: rawTrial.twice.count, forward: rawTrial.forward.count, backward: rawTrial.backward.count) let sizeView = sizeTrial.report() countSensitivities([raw, sortedView, sizeView]) ``` ``` raw-list | once=["a"] twice=["a", "a"] ab=["a", "b"] ba=["b", "a"] -> doubly-sensitive sorted-list | once=["a"] twice=["a", "a"] ab=["a", "b"] ba=["a", "b"] -> repeat-sensitive line-count | once=1 twice=2 ab=2 ba=2 -> repeat-sensitive robust: 0 repeat-sensitive: 2 order-sensitive: 0 doubly-sensitive: 1 ``` On the raw list the operation is **doubly sensitive**. When the list is sorted, `ab` and `ba` become equal and only the repetition difference remains; the operation comes out **repeat-sensitive**. The same result holds for the line count. The rule does not see the cause, it sees the inequality; sorting the state hid the order difference, but could not hide the repetition difference, because repetition really does increase the line count. The count is two repeat-sensitive, one doubly sensitive. **CC18.** In the sorted view the elements are assumed to be comparable. For elements whose order has a meaning, for example lines with timestamps, what information sorting erases was not measured. #### Moving from Queue to Actor Turning an old queue-based class into an actor often looks mechanical: make the class an `actor`, remove the queue, add `await` to the calls. The real work is in the callers. Where the old code waited with `sync`, `await` is now needed, and this requires the caller itself to be in an asynchronous context. Callers that work with a callback are either wrapped in a `Task` or the actor call is wrapped in the callback; in both cases the finish time of the call is no longer the return time of the call, and the code must be rethought accordingly. The second difference is order. A serial queue ran jobs in the order they were added with `async`; an actor, on the other hand, does not guarantee to process calls in arrival order, because the order in which two callers reach the actor depends on the timing between them. The order guarantee the old code silently relied on can disappear during the move. For this reason, before the move, the places where order is really needed are determined. If order is needed, the calls are made one after another from a single task; if it is not needed, the move is trouble-free. The measurements of this lesson are a tool for finding which places need order: every operation that comes out order-sensitive is an operation that asks for attention in the move. Finally, testing. Testing queue-based code requires a wait call that drains the queue; in actor-based code this need disappears because each call returns its own result with `await`. Tests become simpler, but for operations with order sensitivity the test still needs to contain a scenario with two orders. #### Queue Labels and Debugging Naming queues is a small habit and makes a big difference in debugging. In a crash trace or a hang report the name of the queue is visible; from a queue named `serial` it is not possible to work out which subsystem is responsible, but a name like `reservations.state` points the way directly. For this reason the name `serial` in the measurements is only for the example; in real code the name of the queue should describe the state it protects. For the same reason, care is taken that a queue protects only one state. Gathering several unrelated states in a single queue produces a setup in which the slow job of one holds up the others. A separate queue for each state keeps independent problems independent; but an operation touching several queues no longer gives a single guarantee about order, and the first caveat in the measurement applies here: the case where several queues reach the same state was not measured. Choosing queue names in a way the next person reading the code will understand at first glance is the cheapest debugging investment; because in hang reports only this name is visible and it narrows down the place to look. The same holds for test outputs: if a failing test can say from its name which queue broke which state, finding the bug takes seconds instead of minutes. Not counting naming as an ornament to be added at the end of the code is a small decision to be made at first writing, and adding it afterwards is mostly forgotten. #### Summary - A serial queue runs jobs in the order they were added and one at a time; a concurrent queue can start jobs at the same time. - Appending to a queue is sensitive to both repetition and order. - Assigning a value in a queue is sensitive only to order; assigning the same value again is idempotent. - A barrier prevents a write from overlapping with reads on a concurrent queue. - Measuring the state in sorted form hides the order difference. - This lesson's sensitivity: **doubly sensitive** (append) and **order-sensitive** (assignment). #### Next Step Queues put jobs in a line; but a value flowing through time and reaching several subscribers is a different model. In the next lesson, on reactive streams, we will measure what subscribing to the same publisher twice and operators that drop consecutive values do. ### Reactive Streams Source: https://academia.sh/en/courses/data-networking-and-concurrency/reactive-streams A measurement showing that a repeated subscription to a publisher repeats the request, that a shared stream stays robust, and that the operator dropping consecutive repeats depends on the pair of values. In the previous lesson, appending to a queue came out sensitive to both repetition and order, and assigning a value came out sensitive only to order. In this lesson we look at a third form of concurrent code: reactive streams, in which values flow through time and many places listen to them. The core file is the same in every lesson of the topic. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` In this lesson the operations write only the information of a stream to the state and are not asynchronous; so they are measured with `replay`. The model builds what the operation does to the state; it is not the network or the framework that runs the stream itself. #### Publisher, Subscriber, and Operator In a **reactive stream**, a **publisher** produces values, a **subscriber** receives those values, and **operators** in between transform or filter the values. A `map` operator transforms every value, and a `removeDuplicates` operator drops consecutive repeats. A publisher can be cold: it starts its work when subscribed to, for example it sends the network request. If the publisher is shared, the work starts only at the first subscriber, and later subscribers get the same result. This distinction has a measurable consequence. Each time an unshared stream is subscribed to, the request goes out again. On a shared stream, the second subscriber does not repeat the request. ```swift // Feed.swift — subscribing to a stream and an operator that drops consecutive repeats struct Delivery: Equatable, CustomStringConvertible { var requests: Int var listeners: [String] var description: String { "requests=\(requests) listeners=\(listeners)" } } func subscribing(_ name: String) -> (Delivery) -> Delivery { { state in var next = state next.requests += 1 if !next.listeners.contains(name) { next.listeners = (next.listeners + [name]).sorted() } return next } } func subscribingShared(_ name: String) -> (Delivery) -> Delivery { { state in var next = state if next.listeners.isEmpty { next.requests += 1 } if !next.listeners.contains(name) { next.listeners = (next.listeners + [name]).sorted() } return next } } struct Output: Equatable, CustomStringConvertible { var emitted: [Int] var description: String { "\(emitted)" } } func emitting(_ value: Int) -> (Output) -> Output { { state in state.emitted.last == value ? state : Output(emitted: state.emitted + [value]) } } ``` `subscribing` builds the unshared subscription: each call raises the request count by one and adds the name to the listeners. `subscribingShared` builds the shared one: the request count rises only while there is no listener. The listeners are kept sorted, because the order of subscribers should not enter the measurement; this is a design choice and is mentioned in CC19. `emitting` builds the operator that drops consecutive repeats: if the new value equals the last value, it leaves the output untouched. #### Why Streams Exist If a value is computed once and returned, an asynchronous function is enough. Streams are for cases where the value arrives many times over time: every letter in a search box, location updates, events from a server. In these cases, instead of "get the value", you need to say "do this as values arrive", and a stream is the way of building this second sentence. There is a contract between publisher and subscriber: the publisher gives the values in order, the subscriber handles them in the order it receives them, and the operators in between pass each value on, transform it, or drop it. This contract leaves two things to the subscriber. The first is the lifetime of the subscription: values arrive as long as the subscription is held, and the stream ends when it is released. The second is the cost of the subscription: every subscription, if the publisher is cold, restarts the work behind it. The questions of the earlier lessons continue here unchanged; sensitivity to repetition and order has moved to the way values arrive. #### Subscription and Sharing First the same name subscribes twice. Then two different names subscribe, to both an unshared and a shared stream. ```swift // main.swift — unshared and shared subscription let fresh = Delivery(requests: 0, listeners: []) let cold = replay("cold-same-name", from: fresh, subscribing("alice"), then: subscribing("alice")).report() let sharedSame = replay("shared-same-name", from: fresh, subscribingShared("alice"), then: subscribingShared("alice")).report() let sharedPair = replay("shared-two-names", from: fresh, subscribingShared("alice"), then: subscribingShared("bob")).report() countSensitivities([cold, sharedSame, sharedPair]) ``` ``` cold-same-name | once=requests=1 listeners=["alice"] twice=requests=2 listeners=["alice"] ab=requests=2 listeners=["alice"] ba=requests=2 listeners=["alice"] -> repeat-sensitive shared-same-name | once=requests=1 listeners=["alice"] twice=requests=1 listeners=["alice"] ab=requests=1 listeners=["alice"] ba=requests=1 listeners=["alice"] -> robust shared-two-names | once=requests=1 listeners=["alice"] twice=requests=1 listeners=["alice"] ab=requests=1 listeners=["alice", "bob"] ba=requests=1 listeners=["alice", "bob"] -> robust robust: 2 repeat-sensitive: 1 order-sensitive: 0 doubly-sensitive: 0 ``` On the unshared stream the second subscription sends the request again: `once` is 1, `twice` is 2. The listener list stays the same, but the request count rises. The operation is **repeat-sensitive**. On the shared stream the request count rises only at the first subscriber; subscribing again under the same name or adding another name does not change the request. Because the listeners are kept sorted, the two orders give the same list, and both shared operations are **robust**. The count is two robust, one repeat-sensitive. **CC19.** The listener list was kept sorted; a stream in which the order of subscribers changes the result was not measured. Whether a shared stream restarts after the first subscriber leaves was not measured either. #### Dropping Consecutive Repeats The `removeDuplicates` operator does not pass a value on if it is the same as the last value. We measure the operator with two values: 3 and 5. ```swift // main.swift — the operator that drops consecutive repeats let fresh = Output(emitted: []) let distinct = replay("drop-repeat", from: fresh, emitting(3), then: emitting(5)).report() countSensitivities([distinct]) ``` ``` drop-repeat | once=[3] twice=[3] ab=[3, 5] ba=[5, 3] -> order-sensitive robust: 0 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` The second 3 is dropped, so `once` and `twice` are the same. But the order in which the values arrive is the order of the output: `ab` is `[3, 5]`, `ba` is `[5, 3]`. The operation is **order-sensitive**. The count is one order-sensitive. **CC20.** The measurement models only the operator that drops consecutive repeats. Time-dependent operators such as debounce were not measured, because their result depends on timing. #### Accumulating and Latest-Value Operators Two operators are used a great deal in streams. One is the operator that accumulates the incoming values and publishes the total at each step; counters, cart totals, and unread message counts are built this way. The other is the operator that keeps only the last value; the value of a field shown on screen, or the current state of a setting. Both are measured with the same two values (3 and 5) and the same four runs. The state here is a plain integer. ```swift // main.swift — an accumulating and a latest-value operator let running = replay("running-sum", from: 0, { $0 + 3 }, then: { $0 + 5 }).report() let latest = replay("latest-value", from: 0, { _ in 3 }, then: { _ in 5 }).report() countSensitivities([running, latest]) ``` ``` running-sum | once=3 twice=6 ab=8 ba=8 -> repeat-sensitive latest-value | once=3 twice=3 ab=5 ba=3 -> order-sensitive robust: 0 repeat-sensitive: 1 order-sensitive: 1 doubly-sensitive: 0 ``` In the running-sum line, `once` is 3 and `twice` is 6: if the same value arrives twice, the total rises twice as much. Order does not matter; 3 then 5 and 5 then 3 both give eight. The operation is **repeat-sensitive**. In the latest-value line the second arrival of the same value changes nothing, but because the last to arrive wins, `ab` is 5 and `ba` is 3. The operation is **order-sensitive**. The same stream can do two different jobs, and which operator it is built with determines the sensitivity. In a real application this difference looks like this. If a publisher delivers the same event twice while reconnecting in the network layer, an accumulating indicator (the unread count) rises twice and stays wrong; a field keeping the latest value writes the same value twice and nothing is noticed on screen. The danger of a repeated event is measured by the type of the operator. If you are using a repeat-sensitive operator such as a counter, you need to carry an identifier that tells events apart and drop the repeat before the operator. **CC21.** The running-sum and latest-value measurement uses an integer state. Overflow of the total and what the first value will be on an empty stream were not measured. #### A Real Stream A real stream comes from a framework, and the measured model does not replace it. The fragment below only shows what the model stands for. ```swift import Combine let subject = PassthroughSubject() let cancellable = subject .removeDuplicates() .sink { value in print(value) } subject.send(3) subject.send(3) subject.send(5) ``` Here `PassthroughSubject` is a publisher, `removeDuplicates` is the operator, and `sink` is the subscription. Of the two consecutive 3s, only one reaches the subscriber. If the subscription is not kept, the stream ends immediately; this is why `cancellable` is held in a variable. #### In a Real Application On a search screen the stream is typically built like this: the text in the box is a publisher; the operator that drops consecutive identical texts prevents a request when the user types the same text again; a debounce operator waits until the user stops typing; finally a subscriber sends the request and writes the result to the screen. Each link corresponds to a measurement in this lesson. Dropping repeats is order-sensitive, because the order in which the texts arrive matters. Subscription is repeat-sensitive, because every subscription can restart a request. Writing to the screen reopens the version question of the previous lesson. The most common setup mistake is rebuilding the subscription every time the screen appears. ```swift // Wrong: a new subscription each time the screen appears, the old one is not released @MainActor final class ListScreen { private var subscriptions: [AnyCancellable] = [] func screenAppeared(_ feed: AnyPublisher<[String], Never>) { feed .sink { items in self.show(items) } .store(in: &subscriptions) } func show(_ items: [String]) {} } ``` Each time the user returns to the screen, `screenAppeared` adds a new subscription and does not release the old ones. On an unshared publisher every subscription sends the request again; after returning three times, the same value is written to the screen three times and three requests go to the network. This is the on-screen counterpart of the measured `cold-same-name` line: subscribing a second time under the same name raised the request count. The fix has two forms. The subscription is built once, where the screen is created; or if it is to be built at each appearance, the old one is released first. Using a shared publisher also makes the request single, but in the case of two screens listening to the same stream with separate lifetimes, what will happen to the stream when the first subscriber leaves was not measured in this lesson. This fragment was not measured: the lifetime of a real subscription, the time of release, and whether the publisher is cold or shared depend on the behaviour of the framework. #### Common Mistakes - Not holding the subscription in a variable. When the subscription is freed, the stream is cancelled. - Subscribing several times to an unshared stream and thinking the request goes out once. - Thinking that dropping consecutive repeats is independent of order. The operator breaks consecutiveness, not order. - Running the subscriber that updates the screen outside the main actor. #### The Bounding Measurement The same operator falls into a different class depending on the pair of values. If the pair is 3 and 3, the two values are the same; if the pair is 3 and 5, they are different. ```swift // main.swift — the same operator, two pairs of values let fresh = Output(emitted: []) let equalPair = replay("pair-3-3", from: fresh, emitting(3), then: emitting(3)).report() let unequalPair = replay("pair-3-5", from: fresh, emitting(3), then: emitting(5)).report() countSensitivities([equalPair, unequalPair]) ``` ``` pair-3-3 | once=[3] twice=[3] ab=[3] ba=[3] -> robust pair-3-5 | once=[3] twice=[3] ab=[3, 5] ba=[5, 3] -> order-sensitive robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` When the same values arrive consecutively, the operator drops the second and all four states come out equal: the operation is **robust**. With different values the order is written into the output and the operation becomes **order-sensitive**. The rule does not see the cause, it sees the inequality; it classifies not the operator but the pair of values that comes with the operator. Before saying an operator is robust, it must be written down with which values it was tried. The count is one robust, one order-sensitive. #### Stream or Asynchronous Function A stream is not the answer to every problem. For a one-time request, an asynchronous function offers fewer concepts and a more readable error path. A stream earns its place where the value arrives again and again and many places listen to the same values. Distributing the changes of a setting to several screens is an example of this; fetching a profile once is not. The most common mistake in mixed use is wrapping a one-time result in a stream and then waiting for the first value. This brings a subscription lifetime problem and doubles the error path. There is a mistake in the opposite direction too: if values keep arriving and the code abandons them after the first value with a `for` loop, the real gain of the stream is lost. The common conclusion of this lesson's measurements is this: a stream carries time into the code, but the questions of repetition and order do not change. For every operator the same two questions are asked: if the same value arrives twice, does the result change; if two values swap places, does the result change. The answer differs from operator to operator, and asking these two questions before choosing an operator prevents most of the bugs that would appear later. One last note: testing streams requires a scheduler that controls time. Testing time-dependent operators such as debounce with a real clock is slow and brittle. In the test environment a mechanism that advances the scheduler by hand is used; this way a five-second wait becomes an instant step in the test. Advancing time by hand also makes race-like bugs reproducible: because the moment at which the response arrives is written inside the test, the bug appears in the same place on every run. Being able to reproduce a bug is half of fixing it; the other half is showing with the same scenario that the fix really works. The model of this lesson is timeless and deliberately leaves out the time-dependent aspects of the operators. #### Summary - In a reactive stream the publisher produces values, the subscriber receives them, and operators transform or filter. - Every subscription to an unshared stream repeats the request; the operation is repeat-sensitive. - On a shared stream the request goes out only at the first subscriber and the operation is robust. - The operator that drops consecutive repeats is order-sensitive on different values. - The same operator comes out robust on an equal pair of values; the class is determined by the pair of values, not the operator. - This lesson's sensitivity: **repeat-sensitive** (unshared subscription), **order-sensitive** (different values), and **robust** (shared stream). #### Next Step The results of concurrent code may not stay on screen; they are lost when the application closes. In the next topic, the first lesson, Simple Preference Storage, will measure a model in which two screens write the same preference and look at how persistent state reacts to repetition and order. ## Persistence Saving small settings, storing secret values in the keychain, directory types and backup rules, the relational local database, object persistence frameworks, cloud sync across devices, and the registration and permission flow of push notifications. ### Simple Preference Storage Source: https://academia.sh/en/courses/data-networking-and-concurrency/simple-preference-storage A measurement of how the order in which two screens write to the same key, the repetition of a launch counter, and a default value change the outcome when a small value is stored persistently. In the previous lesson, Reactive Streams, we saw how a value flows over time. When the stream ends, the value is gone. Small values that must survive the app being closed and reopened need another place: a theme choice, the last opened tab, whether a hint has already been shown. This topic covers that place; its name is **persistence**. The question of the course stays the same here: **if this operation is repeated or reordered, does the result change?** There are four answers, and their names are fixed throughout the course: **robust**, **repeat-sensitive**, **order-sensitive**, and **doubly sensitive**. The umbrella name is **sensitivity**, and the rule reads it not from a label but from the result the operation leaves in the state. The measurement runs on the core file. Every lesson gives it unchanged. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` `once` is the operation applied to the starting state one time, `twice` is two times; `ab` is `a` first and then `b`, and `ba` is the reverse order. If `once` and `twice` differ, the operation is repeat-sensitive; if `ab` and `ba` differ, it is order-sensitive. The operations measured in this lesson are applied to a pure Swift model of a preference store. The model builds what the operation does to the state; it is not the store or the disk itself. #### What the Preference Store Is For The platform's preference store keeps key-value pairs. The values are simple types such as strings, numbers, Booleans, dates, and arrays and dictionaries built from them. A `UserDefaults.standard` object is shared by the whole app: a value written by one screen is read immediately by another screen in the same process. This convenience is the real subject of the measurement, because two screens writing to a shared key can overwrite each other's value without noticing. The store is meant for small, non-secret values. Large data goes to the file system or a database; secret values such as passwords and tokens go to the keychain, which is the subject of the next lesson. A value written to the preference store sits there in readable form. ```swift import Foundation UserDefaults.standard.register(defaults: ["theme": "system", "launchCount": 0]) UserDefaults.standard.set("dark", forKey: "theme") let count = UserDefaults.standard.integer(forKey: "launchCount") UserDefaults.standard.set(count + 1, forKey: "launchCount") UserDefaults.standard.removeObject(forKey: "theme") ``` This fragment is SOURCE; it calls a class of the platform, so it does not run here. Three things are read from it: `set(_:forKey:)` writes the value, `integer(forKey:)` returns `0` when the key does not exist, and `removeObject(forKey:)` deletes the written value. The model below carries these names unchanged. #### The Model of the Preference Store The model keeps two separate dictionaries: the values written explicitly and the registered default values. A read returns the written value first, and the default if there is none. The state is also compared and printed by this **resolved** value. ```swift // PreferenceModel.swift — a pure Swift model of the preference store struct PreferenceBox: Equatable, CustomStringConvertible { private(set) var stored: [String: String] = [:] private(set) var registered: [String: String] = [:] var resolved: [String: String] { registered.merging(stored) { _, explicit in explicit } } var description: String { let pairs = resolved.keys.sorted().map { "\($0)=\(resolved[$0] ?? "-")" } return pairs.isEmpty ? "-" : pairs.joined(separator: ",") } static func == (lhs: PreferenceBox, rhs: PreferenceBox) -> Bool { lhs.resolved == rhs.resolved } mutating func register(defaults: [String: String]) { registered.merge(defaults) { _, new in new } } mutating func set(_ value: String, forKey key: String) { stored[key] = value } mutating func set(_ value: Int, forKey key: String) { stored[key] = String(value) } mutating func removeObject(forKey key: String) { stored[key] = nil } func string(forKey key: String) -> String? { resolved[key] } func integer(forKey key: String) -> Int { Int(resolved[key] ?? "") ?? 0 } } func registeredBox(_ defaults: [String: String]) -> PreferenceBox { var box = PreferenceBox() box.register(defaults: defaults) return box } ``` Because the order of dictionaries is undefined, `description` prints the keys sorted. The empty state appears as `-`. This is also a choice: the rule compares resolved values and does not look at whether a value inside the store was written or is a default. **PS1.** The model assumes that a write is seen immediately by later reads in the same process. When the value reaches the disk was not measured. **PS2.** The model assumes that two screens share a single store. The case where each screen uses a separate store object was not built. #### Measuring Three Operations Three operations are built. The first is two screens writing different values to the same `theme` key. The second is two screens writing to different keys. The third is incrementing a counter by one each time the app launches. ```swift // main.swift — two screens and a launch counter func writeValue(_ value: String, forKey key: String) -> (PreferenceBox) -> PreferenceBox { { box in var next = box next.set(value, forKey: key) return next } } func incrementLaunchCount(_ box: PreferenceBox) -> PreferenceBox { var next = box next.set(next.integer(forKey: "launchCount") + 1, forKey: "launchCount") return next } let sameKey = replay("two-screens-same-key", from: PreferenceBox(), writeValue("dark", forKey: "theme"), then: writeValue("light", forKey: "theme")).report() let otherKeys = replay("two-screens-different-keys", from: PreferenceBox(), writeValue("dark", forKey: "theme"), then: writeValue("en", forKey: "language")).report() let launches = replay("launch-counter", from: PreferenceBox(), incrementLaunchCount, then: incrementLaunchCount).report() countSensitivities([sameKey, otherKeys, launches]) ``` ``` two-screens-same-key | once=theme=dark twice=theme=dark ab=theme=light ba=theme=dark -> order-sensitive two-screens-different-keys | once=theme=dark twice=theme=dark ab=language=en,theme=dark ba=language=en,theme=dark -> robust launch-counter | once=launchCount=1 twice=launchCount=2 ab=launchCount=2 ba=launchCount=2 -> repeat-sensitive robust: 1 repeat-sensitive: 1 order-sensitive: 1 doubly-sensitive: 0 ``` In the first line, writing the same value twice did not change the result: `once` and `twice` are the same. But the order of two different values was decisive: `ab` is `light`, `ba` is `dark`. The last writer wins, and the write order decides the winner. That is why the class is **order-sensitive**. In the second line, the two screens write to different keys. The write order does not change the result, and repeating the same write does not change it either. The class is **robust**. This line shows that the problem is not in the writing itself but in two writes sharing the same key. In the third line, the counter became `1` when applied once and `2` when applied twice. The order is meaningless for this operation, because the two operations are identical; repetition, however, changes the result. The class is **repeat-sensitive**. Incrementing the counter requires a read followed by a write; the value written depends on the value read. An operation that writes a constant value to the same key does not change when repeated. That is the difference: `set` with a constant value is robust, and an operation that reads and increments is not. The count is one robust, one repeat-sensitive, one order-sensitive. #### Default Values If a key has never been written, a read returns `nil` or `0`. Rather than handling this case everywhere, the app registers default values when it starts: `register(defaults:)`. Registered defaults do not count as written values; they are the fallback that applies when no written value exists. The registration repeats at every launch, because defaults are not stored persistently. The default value is also the value read once a key is deleted. `removeObject(forKey:)` removes the written value and the read falls back to the default. How these two operations behave together changes with the default value, and that is the subject of this lesson's bounding measurement. #### Common Mistakes - Using the same key on two screens for different purposes. The last writer wins, and each silently overwrites the other's value. - Reading and incrementing a counter without accounting for repetition. If the button is pressed twice or the launch code runs twice, the counter goes up by two. - Writing a secret value to the preference store. The store provides no privacy boundary; secret values belong to the keychain. - Writing the default value separately at every read site. The value diverges in two places, and which one is valid becomes unclear. - Putting a large piece of data in the preference store. The store is designed for small values. - Duplicating the key name as a string. A typo opens a new key, and the old value keeps being read. #### Interpreting the Measured Result When two screens wrote to the same key, the write order decided the result; repeating the same value did no harm. When they wrote to different keys, the two screens did not affect each other. The launch counter, on the other hand, changed when repeated. The three lines show that the preference store does not have a single behavior: the same `set` call falls into a different class depending on who shares the key and on whether the value is constant or computed. In practice, two habits are enough. The first is that every key has a single owner: the screen that writes the value is known, and no other screen writes to the same key. The second is protecting counters and similar computed values against repetition: if an operation is considered to have run once, a separate record of that is kept. These two habits remove the source of the first and third lines above at the writing stage. The measurement shows that they are necessary, but it does not show when the two screens will actually write. **PS3.** This measurement builds the increment as a read followed by a write, without interruption. The case where two increments run at the same time was not built. #### The Bounding Measurement The rule measures from a single starting state. The same pair of operations can fall into a different class from a different start. Here the pair is this: `a` deletes the key, and `b` writes `dark` to the same key. The start is built in three ways, depending on what the default value is. ```swift // main.swift — the same operation pair from three different starts func removeValue(forKey key: String) -> (PreferenceBox) -> PreferenceBox { { box in var next = box next.removeObject(forKey: key) return next } } func writeDark(_ box: PreferenceBox) -> PreferenceBox { writeValue("dark", forKey: "theme")(box) } func writeValue(_ value: String, forKey key: String) -> (PreferenceBox) -> PreferenceBox { { box in var next = box next.set(value, forKey: key) return next } } let noDefault = replay("no-default", from: PreferenceBox(), removeValue(forKey: "theme"), then: writeDark).report() let sameDefault = replay("default-dark", from: registeredBox(["theme": "dark"]), removeValue(forKey: "theme"), then: writeDark).report() let otherDefault = replay("default-system", from: registeredBox(["theme": "system"]), removeValue(forKey: "theme"), then: writeDark).report() countSensitivities([noDefault, sameDefault, otherDefault]) ``` ``` no-default | once=- twice=- ab=theme=dark ba=- -> order-sensitive default-dark | once=theme=dark twice=theme=dark ab=theme=dark ba=theme=dark -> robust default-system | once=theme=system twice=theme=system ab=theme=dark ba=theme=system -> order-sensitive robust: 1 repeat-sensitive: 0 order-sensitive: 2 doubly-sensitive: 0 ``` The same two operations gave two classes from three starts. With no default and with the default `system`, the order is decisive: delete and then write `dark` gives `dark`, while write and then delete returns to the default or to empty. With the default `dark`, both orders give `dark`, because the value the deletion falls back to is already the value being written. The result comes out robust. This does not show that the operation is robust in every state; it shows only that the order is not visible from that start. Changing the default value changed the class without touching any operation code. This measurement has a limit too. The rule does not look at whether the value came from the store itself or from the default; it compares only the resolved value. A written `dark` and a default `dark` count as the same. #### Where It Shows Up in a Real App The measured order sensitivity is not an artificial laboratory case. In an app, the theme choice can be changed both by the settings screen and by a quick switch on the home screen. When two screens write to the same key, the remaining value depends on which one wrote later. If the user changes the theme from the home screen while the settings screen is open, the settings screen can write its own old copy back when it closes. The result is that the change the user just made silently disappears. The bug report has no trace of it: no call failed, and each write worked correctly on its own. The launch counter, on the other hand, is the typical example of repeat sensitivity. The code that increments the counter is written to run once each time the user opens the app. If the same code also runs when the screen reappears, when returning from the background, or on a retry, the counter ends up larger than the real number of launches. Decisions that depend on the counter, such as a welcome message or a request for a review, are triggered at the wrong time for the same reason. The `twice` column of the measurement shows exactly this situation: when the operation is repeated, the value has risen once more. The default value issue is even subtler. Reading a key that was never written looks the same in two different situations: the app was just installed, and the app was updated from an older version. The bounding measurement showed this: the same pair of operations can fall into a different class depending on the starting state. So whether a preference is safe cannot be told from the operation alone; the question of which start the operation ran from is asked as well. #### A Wrong Setup and Its Fix The most common wrong setup is each screen keeping the preference as a copy of its own. The screen reads the value when it opens, updates the copy as the user changes it, and writes the whole copy to the store when it closes. In this arrangement, when the lifetimes of two screens overlap, the final write overwrites the other screen's change. The fix has two steps. The first is to write only the key the user changed, at the moment the user changed it, instead of writing the whole copy when the screen closes. This way screens writing to different keys do not affect each other, which the different-keys line in the measurement showed as robust. The second, if the same key has to be changed from more than one place, is to route the change through a single place and have the screens listen to that place instead of the store. This does not solve the order problem, but it makes the order visible in one place; the way an order-sensitive operation becomes safe is, more often than not, that the order can no longer change. For the counter, the fix is different. Storing the information "has this launch been counted" instead of incrementing the counter protects the operation against repetition. A second call arriving in the same session sees that the flag is set and does nothing. In measurement terms, this means the operation stops being repeat-sensitive and becomes robust; but we did not measure that separately in this lesson, we are only stating the direction. #### Reading the SOURCE Fragment The unnamed fragment at the start of the lesson is not compiled, because a real preference store depends on the platform and on the app's lifecycle. The fragment is still worth reading. Its read and write calls are the exact counterparts of the two operations in the model we measured: where the model reads and writes a key, the real fragment reads and writes the same key. The difference is that in the real store a write does not go to the disk immediately, and it is not guaranteed that reads see a change coming from another process. This difference is recorded as PS1 and PS2, and it does not change the measured class; it only says how close the real behavior is to the model. The way the default value is supplied in the fragment also matters. When the key does not exist, the read call returns a value, and that value can conflict with a default defined separately elsewhere in the code. Defaults defined in two places are the real-life counterpart of the two different starting states in the measurement. Defining the default in a single place prevents the class shift shown by the bounding measurement from the start. #### Summary - The preference store keeps small, non-secret values as key-value pairs; secret values belong to the keychain. - When two screens write different values to the same key, the write order decides the result (`ab` is `light`, `ba` is `dark`); different keys do not affect each other. - An operation that reads and increments, like a launch counter, changes the result when repeated (`1`, then `2`); writing a constant value does not. - A default value can move the same delete-and-write pair from robust to order-sensitive; the result is measured from a single start. - The count of the three operations is one robust, one repeat-sensitive, one order-sensitive; the bounding measurement gives one robust and two order-sensitive. - This lesson's sensitivity: **order-sensitive**. #### Next Step The preference store is a place where values sit in the open. In the next lesson, The Keychain, we move to the store where secret values are kept, and we measure how adding the same item a second time changes the status code. ### The Keychain Source: https://academia.sh/en/courses/data-networking-and-concurrency/the-keychain A measurement of how adding the same item a second time changes the status code where secret values are stored, and how an update-or-add operation is order-sensitive. In the previous lesson we saw that the preference store keeps its values in the open. For secret values such as a password, a session token, or an application key, the platform has a separate store: the **keychain**. The keychain stores values encrypted, keeps some items even if the app is deleted and reinstalled, and controls access item by item. In return it charges a price: operations return a result code, and that code says whether an item already exists. The question of the course stays the same: **if this operation is repeated or reordered, does the result change?** In this lesson the code becomes part of the result as well. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` `once` is the operation applied to the starting state one time, `twice` is two times; `ab` is `a` first and then `b`, and `ba` is the reverse order. The model below is the pure Swift counterpart of the keychain: an item is nothing more than an account name and a value. The model is not the platform itself; it only builds the result the operation leaves in the state. #### Adding and the Status Code The platform call for adding an item to the keychain reads as follows. The fragment is SOURCE; it needs a real keychain, so it does not run here. ```swift import Security let query: [String: Any] = [ kSecClass as String: kSecClassGenericPassword, kSecAttrAccount as String: "token", kSecValueData as String: Data("a1".utf8) ] let first = SecItemAdd(query as CFDictionary, nil) // errSecSuccess let second = SecItemAdd(query as CFDictionary, nil) // errSecDuplicateItem ``` The second call comes with the same class and the same account name, so it does not create a new item; it returns `errSecDuplicateItem`. The existing value stays as it is. So the second add has no effect but it has a result, and that result is an error code. Changing an existing item takes a separate call: `SecItemUpdate`. If the item does not exist, that call returns `errSecItemNotFound` as well. Applications combine these two calls as **update-or-add**: first try the update, and add the item if it does not exist. #### The Model of the Keychain The model keeps a dictionary from account name to value, and the status code of the last operation. The status code enters the comparison; `includesStatus` can turn this off, and the bounding measurement makes use of that. ```swift // KeychainModel.swift — a pure Swift model of the keychain struct KeychainBox: Equatable, CustomStringConvertible { static let success = 0 static let duplicateItem = -25299 static let itemNotFound = -25300 private(set) var items: [String: String] private(set) var status: Int let includesStatus: Bool init(includesStatus: Bool = true, items: [String: String] = [:]) { self.includesStatus = includesStatus self.items = items self.status = KeychainBox.success } var description: String { let pairs = items.keys.sorted().map { "\($0)=\(items[$0] ?? "-")" } let text = pairs.isEmpty ? "-" : pairs.joined(separator: ",") return includesStatus ? "\(text) status=\(status)" : text } static func == (lhs: KeychainBox, rhs: KeychainBox) -> Bool { lhs.description == rhs.description } mutating func add(_ value: String, account: String) { if items[account] != nil { status = KeychainBox.duplicateItem } else { items[account] = value status = KeychainBox.success } } mutating func update(_ value: String, account: String) { if items[account] != nil { items[account] = value status = KeychainBox.success } else { status = KeychainBox.itemNotFound } } mutating func addOrUpdate(_ value: String, account: String) { update(value, account: account) if status == KeychainBox.itemNotFound { add(value, account: account) } } } func adding(_ value: String, account: String = "token") -> (KeychainBox) -> KeychainBox { { box in var next = box next.add(value, account: account) return next } } func upserting(_ value: String, account: String = "token") -> (KeychainBox) -> KeychainBox { { box in var next = box next.addOrUpdate(value, account: account) return next } } ``` The status code numbers are kept the same as the platform's constants: success `0`, duplicate item `-25299`, item not found `-25300`. The comparison and the printing both go through the same `description`; so what shows up in a classification and what counts in an equality do not drift apart. **PS4.** The model counts only the account name as the identity of an item. On the platform, identity is the combination of attributes such as class, service, and account; modeling it with the account name alone simplifies the measurement. **PS5.** The model produces status codes from the constants above. The platform's real return values, access control, and device lock were not measured. #### Measuring Three Operations Three operations are built. The first is adding the same value twice. The second is running the update-or-add operation with two different values. The third is adding two different values directly. ```swift // main.swift — add and update-or-add let sameValue = replay("add-same-value", from: KeychainBox(), adding("a1"), then: adding("a1")).report() let upsert = replay("upsert", from: KeychainBox(), upserting("a1"), then: upserting("b2")).report() let twoValues = replay("add-two-values", from: KeychainBox(), adding("a1"), then: adding("b2")).report() countSensitivities([sameValue, upsert, twoValues]) ``` ``` add-same-value | once=token=a1 status=0 twice=token=a1 status=-25299 ab=token=a1 status=-25299 ba=token=a1 status=-25299 -> repeat-sensitive upsert | once=token=a1 status=0 twice=token=a1 status=0 ab=token=b2 status=0 ba=token=a1 status=0 -> order-sensitive add-two-values | once=token=a1 status=0 twice=token=a1 status=-25299 ab=token=a1 status=-25299 ba=token=b2 status=-25299 -> doubly-sensitive robust: 0 repeat-sensitive: 1 order-sensitive: 1 doubly-sensitive: 1 ``` In the first line, the item is `a1` in both states; only the status code differs: `0` and `-25299`. The add had no effect, but the result changed. The order is meaningless, because the two operations are identical. The class is **repeat-sensitive**. This is a different kind of repetition from the counter in the previous lesson: it is not the value but the result the operation reports that changes. In the second line, when the update-or-add operation is repeated, the result does not change: the item and the code stay the same. Having no effect has solved repetition. But it does not solve order with two different values: `ab` is `b2`, `ba` is `a1`. The last writer wins, and the order decides the winner. The class is **order-sensitive**. The rule sees the inequality; it does not see the cause, that is, that the second call went through the update path. The third line combines the two effects. The second add fails, so both repetition and order diverge. The item stays with the value of whichever came first, and which came first depends on the order. The class is **doubly sensitive**. The count is one repeat-sensitive, one order-sensitive, one doubly sensitive; there is no robust. #### Why Update-or-Add A direct add assumes that the item does not exist beforehand. If the app adds the token at every launch, the call returns an error from the second launch on and the old value stays; the new value is not silently written. This is a common reason for continuing to work with an old token after the token has been renewed. Update-or-add removes this error: repetition no longer produces an error. Its price is the second line above. If two separate sources write different values to the same account, the order decides the winner. In practice, two decisions are enough. The result of the add is checked: `errSecDuplicateItem` is not a failure but information that the item already exists. If an account can be written by more than one source, which source gets the last word is written down in the design. #### Common Mistakes - Writing a secret value to the preference store. The keychain exists for this. - Not checking the result of an add. Even when an error is returned, the app assumes success and continues with the old value. - Adding directly at every launch. It works on the first launch and fails silently on the later ones. - Calling the update-or-add path from two separate places. The order decides the last writer and debugging becomes harder. - Thinking of delete and add as a single step. If the app is interrupted between the two calls, the item is lost. - Duplicating the account name as a string. A typo opens a new item. **PS6.** The model runs the calls without interruption and on a single thread. The case where two calls run at the same time was not built. #### The Bounding Measurement The first limit is whether the status code enters the compared state. The rule sees the inequality; if the status code is not part of the state, the second add is invisible. Next, we run the same three adds with `includesStatus: false`. On the third line, we measure the same pair of operations from a start where the item exists from the beginning. ```swift // main.swift — without the status code and from a seeded start let plain = KeychainBox(includesStatus: false) let withoutStatus = replay("no-status-same-value", from: plain, adding("a1"), then: adding("a1")).report() let withoutStatusTwo = replay("no-status-two-values", from: plain, adding("a1"), then: adding("b2")).report() let seeded = replay("status-seeded-start", from: KeychainBox(items: ["token": "a1"]), adding("a1"), then: adding("a1")).report() countSensitivities([withoutStatus, withoutStatusTwo, seeded]) ``` ``` no-status-same-value | once=token=a1 twice=token=a1 ab=token=a1 ba=token=a1 -> robust no-status-two-values | once=token=a1 twice=token=a1 ab=token=a1 ba=token=b2 -> order-sensitive status-seeded-start | once=token=a1 status=-25299 twice=token=a1 status=-25299 ab=token=a1 status=-25299 ba=token=a1 status=-25299 -> robust robust: 2 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` When the status code was taken out, the same pair of adds turned **robust**; with two different values, the doubly sensitive class dropped to order-sensitive. The operation did not change, only what we look at changed. Taking the code into the state makes repetition visible; leaving it out hides it. The second limit is the starting state. The same two adds were repeat-sensitive from an empty start. From a start where the item already exists, the first add fails too and the status is pinned at `-25299`: the class is robust. An operation that comes out robust is not robust in every state; it only does not show repetition from this start. #### Where It Shows Up in a Real App An add operation failing on the second call is one of the most common behaviors of the keychain. An app saves the session key at the first sign-in; when the user signs out and signs in again, the same code runs and the add is rejected because the item already exists. The developer mostly sees this only as an error code, and the user's new key is never stored: the old key stays in place, while the app thinks it has saved the new one. The `-25299` status in the measurement is the counterpart of this collision, and the `add-same-value` line shows that the status changes even when the same value is added a second time. This distinction matters, because even if the resulting value stays the same, the status code has changed. A caller that reads the code may count the second add as a failure and show an error screen. In the bounding lines, where we did not take the status code into account, the same pair of operations looks robust. The difference is what was included in the equality in the measurement; the rule sees the inequality, but we choose what counts as equal. Besides sign-out and sign-in, keychain calls are also repeated when the app reopens, when restoring from a backup, and when two separate extensions share the same item. Two places writing a shared item is the real counterpart of the two-value add line in the measurement: which value remains depends on which one ran first. #### A Wrong Setup and Its Fix A common wrong setup is deleting first and then adding. The logic goes like this: if the item exists, delete it, then add it; that way there is no collision. This arrangement leaves a gap between the two calls in which the item does not exist at all. If the app is interrupted in this gap, or if another place reads the item at exactly that moment, the user is left without a key. Besides, delete and add are two separate operations; each has its own status code, and both have to be checked. The fix is the "update, otherwise add" pattern we measured. It tries the update first and adds if the item does not exist. The update is robust against repetition; a second update with the same value changes nothing. The pattern does not solve order, though: if two different values arrive in two orders, the last one written remains. That is why the `upsert` line came out protected against repetition but order-sensitive. What the pattern provides is that the first collision does not turn into an error; it does not answer the question of which value will remain. A second common mistake is ignoring the status code. Code that does not check the result of the add call takes a failed write for a successful one. The fix is simple and tedious: read the status of every call and proceed only on the expected codes. On an unexpected code, the app should present the user with an understandable result. #### Reading the SOURCE Fragment The fragment at the start of the lesson shows that an item is defined by a query dictionary. The class, service, and account fields in the dictionary make up the identity of the item; the value is a separate field. The call returns a status code, and the code reports success or which collision occurred. Our model reduced the identity to the account name alone; this is recorded as PS4. In reality, the same account name can be two separate items under two different services, and two writes that collide in the model may never collide in reality. The access setting in the fragment also stayed outside the model. Decisions such as when an item can be read and whether it is reachable while the device is locked directly affect the status codes. That is why PS5 records the real return values and access control of the status codes as not measured. What is measured in this lesson is the rule itself: which pair of operations falls into which class from which start. #### From Measurement to Design The three lines of the measurement can be turned into a checklist for any code that writes to the keychain. The first question is what repetition does to the operation. When the add call runs a second time, the status code changes; this decides whether the caller will take the failure for an event. The second question is two different values arriving: if which one remains was not written as a rule, the result is left to chance. The third question is which start the operation ran from. That the status lines come out robust from the seeded start says that, in an app where the item already exists, the add call consistently gives the same error; this consistency is not a guarantee, only an observation. In practice this means: the code that writes the key in the sign-in flow must not assume whether the item exists before writing. The update-or-add pattern removes this assumption. In the sign-out flow, the code that deletes the item must not give an error when there is no item to delete; making the delete robust against repetition protects a user who presses the sign-out button twice. We did not measure deletion in this lesson, but the same question has to be asked of it, and the answer is found by measuring rather than assuming. Finally, remember that the value in the keychain is a secret. The outputs of this lesson contain example values; writing a real key to a log or showing it on screen is a separate mistake. The measurement code carries no secret, and your measurement code should not carry one either. #### Summary - Secret values belong to the keychain; operations return a status code, and that code is part of the result. - Adding the same item a second time does not change the item but turns the status from `0` to `-25299`; the class is repeat-sensitive. - Update-or-add solves repetition but does not solve order with two different values: `ab` is `b2`, `ba` is `a1`. - Adding two different values directly separates both repetition and order; the class is doubly sensitive. - When the status code is removed from the comparison, the same pair of adds looks robust; it also looks robust if the item exists at the start. - This lesson's sensitivity: **repeat-sensitive**. #### Next Step The keychain stores small secret values. In the next lesson, The File System, we move to where larger data is written to files, and we measure that appending to the end of a file is sensitive to both repetition and order. ### The File System Source: https://academia.sh/en/courses/data-networking-and-concurrency/the-file-system A measurement of the repetition and order sensitivity of appending, full replacement, and directory creation where larger data is written to files. The preference store was for small values, and the keychain for secret values. Larger data, such as a document, an export file, a log, or a downloaded image, is written to the **file system**. What the file system offers is not a key-value mapping but a path and a sequence of bytes. So its operations fall into two groups: those that change the content of a file and those that change the directory structure. The question of the course is the same for both: **if this operation is repeated or reordered, does the result change?** ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` `once` is the operation applied to the starting state one time, `twice` is two times; `ab` is `a` first and then `b`, and `ba` is the reverse order. The operations measured in this lesson are applied to a pure Swift model of a directory and file tree. The model writes nothing to disk and is not the platform itself. #### Which Data Goes to Which Directory The places where an app can write files are separated by purpose. The documents directory is for data the user produces and must not lose; it is included in the backup. The support directory is for data that the app does not show to the user but cannot regenerate. The cache directory is for data that can be regenerated, and the system can clear it when it needs space. The temporary directory is shorter-lived still. ```swift import Foundation let documents = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] let caches = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)[0] var exports = documents.appendingPathComponent("exports", isDirectory: true) try FileManager.default.createDirectory(at: exports, withIntermediateDirectories: true) var values = URLResourceValues() values.isExcludedFromBackup = true try exports.setResourceValues(values) let target = exports.appendingPathComponent("settings") try Data("v1".utf8).write(to: target, options: .atomic) ``` This fragment is SOURCE; it needs a real app directory, so it does not run here. Four details are read from it. The `createDirectory` call with `withIntermediateDirectories: true` does not fail if the directory already exists. The property corresponding to the `isExcludedFromBackupKey` key leaves a file or a directory out of the backup; it is used for data that can be downloaded again. The `.atomic` option writes the data to a temporary file first, then puts it in place of the target. The system chooses when the cache directory is cleared; this lesson does not measure that. #### The Model of the File Tree The model keeps a set of directories and, for each file, a list of lines. In the output the directories and files are printed sorted; the empty set appears as `-`. ```swift // FolderModel.swift — a pure Swift model of a directory and file tree struct FolderBox: Equatable, CustomStringConvertible { private(set) var directories: Set private(set) var files: [String: [String]] init(directories: Set = [], files: [String: [String]] = [:]) { self.directories = directories self.files = files } var description: String { let dirText = directories.isEmpty ? "-" : directories.sorted().joined(separator: ",") let fileText = files.isEmpty ? "-" : files.keys.sorted().map { "\($0)=\(files[$0]?.joined(separator: "+") ?? "")" }.joined(separator: ",") return "dirs:\(dirText) files:\(fileText)" } mutating func ensureDirectory(_ path: String) { directories.insert(path) } mutating func append(_ line: String, to path: String) { files[path, default: []].append(line) } mutating func appendIfPresent(_ line: String, to path: String) { if files[path] != nil { files[path]?.append(line) } } mutating func writeAtomically(_ text: String, to path: String) { files[path] = [text] } } func ensuring(_ path: String) -> (FolderBox) -> FolderBox { { box in var next = box next.ensureDirectory(path) return next } } func appending(_ line: String, to path: String) -> (FolderBox) -> FolderBox { { box in var next = box next.append(line, to: path) return next } } func appendingIfPresent(_ line: String, to path: String) -> (FolderBox) -> FolderBox { { box in var next = box next.appendIfPresent(line, to: path) return next } } func writing(_ text: String, to path: String) -> (FolderBox) -> FolderBox { { box in var next = box next.writeAtomically(text, to: path) return next } } ``` The model separates two ways of writing. `append` adds a line to the end of the file; the existing content is kept. `writeAtomically` replaces the content of the file completely with the new one; a half-written intermediate state is never visible. On the real platform, an atomic write is done with a temporary file and a replacement. In the model, only the result of this operation is built. **PS7.** The model assumes that a write is either seen completely or not seen at all. The app being interrupted during a write and leaving a half file was not built. **PS8.** The model keeps file names as plain path strings. The directory does not have to exist before the file; in a real write, the call fails if the parent directory does not exist. #### Measuring Four Operations Four operations are built. The first is appending the same line twice. The second is appending two different lines. The third is writing two different contents atomically. The fourth is creating two different directories. ```swift // main.swift — appending, atomic write, directory creation let sameLine = replay("append-same-line", from: FolderBox(), appending("a1", to: "log"), then: appending("a1", to: "log")).report() let twoLines = replay("append-two-lines", from: FolderBox(), appending("a1", to: "log"), then: appending("b2", to: "log")).report() let atomic = replay("atomic-write", from: FolderBox(), writing("v1", to: "settings"), then: writing("v2", to: "settings")).report() let folders = replay("create-directory", from: FolderBox(), ensuring("cache"), then: ensuring("docs")).report() countSensitivities([sameLine, twoLines, atomic, folders]) ``` ``` append-same-line | once=dirs:- files:log=a1 twice=dirs:- files:log=a1+a1 ab=dirs:- files:log=a1+a1 ba=dirs:- files:log=a1+a1 -> repeat-sensitive append-two-lines | once=dirs:- files:log=a1 twice=dirs:- files:log=a1+a1 ab=dirs:- files:log=a1+b2 ba=dirs:- files:log=b2+a1 -> doubly-sensitive atomic-write | once=dirs:- files:settings=v1 twice=dirs:- files:settings=v1 ab=dirs:- files:settings=v2 ba=dirs:- files:settings=v1 -> order-sensitive create-directory | once=dirs:cache files:- twice=dirs:cache files:- ab=dirs:cache,docs files:- ba=dirs:cache,docs files:- -> robust robust: 1 repeat-sensitive: 1 order-sensitive: 1 doubly-sensitive: 1 ``` Appending the same line twice grows the file: `a1` and `a1+a1`. Since the two operations are identical, the order is meaningless. The class is **repeat-sensitive**. If a retry logic runs an append call twice, the line is doubled in the log. Appending two different lines separates both repetition and order. The file becomes either `a1+b2` or `b2+a1`, and in both cases appending twice grows the file. The class is **doubly sensitive**. That appending is this sensitive is not surprising: the content of the file is the ordered sum of everything appended up to that moment. An atomic write solves repetition: writing the same content a second time does not change the file. But it does not solve order with two different contents: `ab` is `v2`, `ba` is `v1`. The last writer wins. Atomicity prevents a half file; it does not decide which write is the last. The class is **order-sensitive**. Directory creation is the only robust one of the four lines. With `withIntermediateDirectories: true`, the operation does nothing if the directory already exists; whichever order two different directories are created in, the same set results. The class is **robust**. The count is one of each class. #### Choice Guide The measurement reduces the decision about writing files to a small rule. If the data is written once and never touched again, an atomic write is chosen; the risk of a half file and of duplication disappears, but if there are two writers, the order decides the winner. If the data is a log that grows over time, appending is chosen, but a key or a counter that prevents duplication is needed. Directory creation can always be considered robust; the app can create the directories it needs at startup without asking. #### Common Mistakes - Putting regenerable data in the documents directory. It bloats the backup. - Putting user data in the cache directory. The system can delete it to free space. - Modifying a large file in place without an atomic write. An interruption leaves a half file. - Calling an append inside a retry. The line is duplicated. - Writing to the same file from two different places and assuming that the winner is known. - Checking whether a directory exists before creating it. Checking is unnecessary; creation is already robust. #### The Bounding Measurement Some operations depend on the file existing beforehand. Here, let the operation be an append that adds a line only if the file exists: a call that cannot open the file silently does nothing. The same pair of operations is measured from two starts: where the file does not exist and where it does. ```swift // main.swift — the same operation pair from two different starts let missing = replay("append-if-present-empty", from: FolderBox(), appendingIfPresent("x", to: "log"), then: appendingIfPresent("x", to: "log")).report() let present = replay("append-if-present-seeded", from: FolderBox(files: ["log": ["a1"]]), appendingIfPresent("x", to: "log"), then: appendingIfPresent("x", to: "log")).report() countSensitivities([missing, present]) ``` ``` append-if-present-empty | once=dirs:- files:- twice=dirs:- files:- ab=dirs:- files:- ba=dirs:- files:- -> robust append-if-present-seeded | once=dirs:- files:log=a1+x twice=dirs:- files:log=a1+x+x ab=dirs:- files:log=a1+x+x ba=dirs:- files:log=a1+x+x -> repeat-sensitive robust: 1 repeat-sensitive: 1 order-sensitive: 0 doubly-sensitive: 0 ``` The same two operations came out robust from the empty start and repeat-sensitive from the seeded start. When the file does not exist, the operation does nothing and so cannot break anything; when the file exists, every call adds a line. The first line, which measured robust, does not say that the operation is robust, only that it had no effect from that start. The rule sees the inequality; it does not generalize the starting state. What stays outside the measurement is stated openly. The directory types, the property of excluding from backup, and the time at which the cache directory is cleared remained as SOURCE and prose. They depend on the timing and the policy of the system; they cannot be built in this model as the result of an operation. #### Where It Shows Up in a Real App Appending to a file is the typical use of a log file, and the `append-same-line` line of the measurement showed that it is unprotected against repetition. The app writes an event to the log; if the code that produces the event runs a second time in a retry or in a lifecycle transition, the same line is in the file twice. An analysis that reads the log later counts one event twice. If the file is always in the append form, the trace of the duplication cannot be erased, because the two lines cannot be told apart. Appending two different lines in two orders, on the other hand, shows double sensitivity: in the `append-two-lines` line, both repetition and order changed the result. In a log, this means the order of events slips. The order may look unimportant, but think of how a log is most often read as a narrative: if the "connected" line comes after the "disconnected" line, the reader reaches the wrong conclusion. Atomic write is the operation of this lesson that was expected to look robust but still came out order-sensitive. Writing the whole file under a temporary name and then moving it over the real name prevents a half file; but if two writes race, the last one wins. This is the measured proof that it solves the half-file problem but does not solve the order problem. Directory creation, on the other hand, is robust for both order and repetition, because an operation that checks existence does not create something a second time that is already there. #### A Wrong Setup and Its Fix A common wrong setup is writing the configuration directly to the real file. If the app is interrupted during the write, the file is left half-written and cannot be read on the next launch. The user sees that the app no longer opens. The fix is the temporary file pattern: the new content is first written under a separate name, verified as complete, and then put in place of the real file in a single step. This ensures that the file holds either the old or the new content. PS7 records that the model assumes this guarantee. The second mistake is storing accumulating data like a log and overwritten data like a setting in the same way. Each line of the accumulating one is an independent event, and repetition creates duplication. Each write of the overwritten one overwrites the previous value, and repetition is harmless. The right setup is to keep the two in separate files and under separate rules. If you want to protect the accumulating one against duplication, you give each event an identity and do not write the same identity a second time; this still does not remove order sensitivity, it only solves repetition. The third mistake is assuming that the directory exists. Code that writes a file gets an error if the parent directory does not exist. That directory creation came out robust in the measurement says that you can safely set up the order "create the directory first, then write the file". #### Reading the SOURCE Fragment The unnamed fragment at the start of the lesson queries the directories in the app's own area. The documents directory is for the user's data, the cache directory is for regenerable data, and the temporary directory is for short-lived data. The system treats these directories differently: files in the cache directory can be deleted when space runs low, while files in the documents directory go into the backup. This is the real lesson of the fragment: where a file is written decides how long the file will live. Because the model keeps directory names as plain path strings, it does not build this distinction; PS8 records that. The measurements therefore tell only the results of the operations, that is, the classes of appending, overwriting, and creating. Which directory is deletable is outside the model and has to be considered separately in the app design. #### From Measurement to Design The common lesson of the four measurements is that file operations are evaluated not by themselves but by what they are used for. Appending is the right operation for a log, but it is unprotected against duplication. Overwriting a file is the right operation for a setting, but it lets the last writer win. So the design question becomes not "which operation should I use" but "which flaw can I accept for this data". A line that is occasionally duplicated in a log is most often acceptable; a lost change in a settings file most often is not. For data whose flaw cannot be accepted, there are two ways. The first is to route every write through a single place; that way the order is decided in one place. The second is to add an identity that protects the write against repetition: each line carries an event identity, and the same identity is not written a second time. An identity turns a doubly sensitive operation into only an order-sensitive one; the order problem can be solved only by a single writer. The file system brings another flaw as well: everything we did not measure. A write fails when the disk is full, the user closes the app during a write, the system clears the cache directory. None of these situations is in the model. The measurement shows the flaw in the logic of the operation, not the flaws of the environment. Thinking about both is the responsibility of any code that writes to a file. #### Summary - The directory is chosen by the type of data: user data to documents, regenerable data to the cache; excluding from backup is possible for data that can be downloaded again. - Appending the same line twice grows the file from `a1` to `a1+a1`; the class is repeat-sensitive. - Appending two different lines separates repetition and order together (`a1+b2` and `b2+a1`); the class is doubly sensitive. - An atomic write solves repetition but does not solve order: `ab` is `v2`, `ba` is `v1`. - Directory creation is robust; an append that depends on file existence is robust from the empty start and repeat-sensitive from the seeded start. - This lesson's sensitivity: **doubly sensitive**. #### Next Step Files keep data as a plain sequence of bytes. In the next lesson, Local Databases, we move to where data is stored by rows and primary keys, and we measure whether update-or-add comes out robust this time. ### Local Databases Source: https://academia.sh/en/courses/data-networking-and-concurrency/local-databases A measurement of how an upsert with a primary key is robust, how inserting with an auto-increment key is sensitive, and how the read order changes the result. Files keep data as a sequence of bytes; finding a record inside, updating a field, or merging two records is the app's job. If the data can be divided into rows and fields, and if it is searched and updated, a **local database** is a better tool. The most common option on a device is an embedded relational database; it lives in a single file and offers a query language. The question of this lesson is the same as in the earlier lessons: **if this operation is repeated or reordered, does the result change?** The difference is this: the database manages identity itself, and where that identity comes from decides the result. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` `once` is the operation applied to the starting state one time, `twice` is two times; `ab` is `a` first and then `b`, and `ba` is the reverse order. The model below is the pure Swift counterpart of a table. The model is not a query engine; it only builds the result the operation leaves in the table. #### Primary Key and Insertion Every row has a **primary key**, and a table cannot hold two rows with the same key. When a row is inserted, the key comes in one of two ways. Either the app supplies the key itself; in that case a second insertion that comes with the same key either fails or, with the upsert syntax, replaces the existing row. Or the database generates the key automatically; in that case every insertion opens a new row. ```swift import SQLite3 var database: OpaquePointer? sqlite3_open("notes.sqlite", &database) sqlite3_exec(database, "CREATE TABLE IF NOT EXISTS notes (id TEXT PRIMARY KEY, body TEXT)", nil, nil, nil) sqlite3_exec(database, "INSERT INTO notes (id, body) VALUES ('k1', 'a1') ON CONFLICT(id) DO UPDATE SET body = excluded.body", nil, nil, nil) sqlite3_exec(database, "SELECT id, body FROM notes ORDER BY id", nil, nil, nil) sqlite3_close(database) ``` This fragment is SOURCE; it needs a database file, so it does not run here. The `ON CONFLICT(id) DO UPDATE` clause builds the upsert behavior: if the key does not exist it inserts the row, and if it exists it replaces the `body` field with the new one. `ORDER BY id` sorts the read by key. Without this clause, the order in which the rows come back is left to the query engine. #### The Model of the Table The model keeps the insertion order of the keys and a mapping from key to value. It carries two extra pieces of information: the next auto-increment key, and whether the read is sorted by key. ```swift // TableModel.swift — a pure Swift model of a table struct TableBox: Equatable, CustomStringConvertible { private(set) var keys: [String] = [] private(set) var values: [String: String] = [:] private(set) var nextId = 1 let sortedRead: Bool init(sortedRead: Bool = true) { self.sortedRead = sortedRead } var description: String { let order = sortedRead ? keys.sorted() : keys return order.isEmpty ? "-" : order.map { "\($0)=\(values[$0] ?? "-")" }.joined(separator: ",") } static func == (lhs: TableBox, rhs: TableBox) -> Bool { lhs.description == rhs.description } mutating func upsert(_ value: String, key: String) { if values[key] == nil { keys.append(key) } values[key] = value } mutating func insert(_ value: String) { let key = String(nextId) nextId += 1 keys.append(key) values[key] = value } } func upserting(_ value: String, key: String) -> (TableBox) -> TableBox { { table in var next = table next.upsert(value, key: key) return next } } func inserting(_ value: String) -> (TableBox) -> TableBox { { table in var next = table next.insert(value) return next } } ``` An upsert on an existing key leaves the row in place and changes only the value; its place in the insertion order is kept. The auto-increment key starts from `1` and goes up by one with every insertion. **PS9.** The model assumes that rows come back in insertion order when no ordering is specified. A real query engine does not guarantee this order; the bounding measurement uses this assumption openly. **PS10.** The model builds a single table and a single primary key column. Keys with multiple columns and relations were not built. #### Measuring Four Operations Four operations are built. The first is applying an upsert to two different keys. The second is writing two different values to the same key. The third is inserting the same value twice with an auto-increment key. The fourth is inserting two different values the same way. ```swift // main.swift — primary key and auto-increment key let distinctKeys = replay("upsert-distinct-keys", from: TableBox(), upserting("a1", key: "k1"), then: upserting("b2", key: "k2")).report() let sameKey = replay("upsert-same-key", from: TableBox(), upserting("a1", key: "k1"), then: upserting("b2", key: "k1")).report() let autoSame = replay("auto-same-value", from: TableBox(), inserting("a1"), then: inserting("a1")).report() let autoTwo = replay("auto-two-values", from: TableBox(), inserting("a1"), then: inserting("b2")).report() countSensitivities([distinctKeys, sameKey, autoSame, autoTwo]) ``` ``` upsert-distinct-keys | once=k1=a1 twice=k1=a1 ab=k1=a1,k2=b2 ba=k1=a1,k2=b2 -> robust upsert-same-key | once=k1=a1 twice=k1=a1 ab=k1=b2 ba=k1=a1 -> order-sensitive auto-same-value | once=1=a1 twice=1=a1,2=a1 ab=1=a1,2=a1 ba=1=a1,2=a1 -> repeat-sensitive auto-two-values | once=1=a1 twice=1=a1,2=a1 ab=1=a1,2=b2 ba=1=b2,2=a1 -> doubly-sensitive robust: 1 repeat-sensitive: 1 order-sensitive: 1 doubly-sensitive: 1 ``` In the first line, two different keys give the same table whichever order they are written in, and repeating the same write does not change the table. Because the read is sorted by key, the insertion order is not visible either. The class is **robust**. This is the only robust line measured in this lesson, and the reason is two things being present together: the app supplies the key and the read is sorted. In the second line, two different values are written to the same key. Repetition is harmless; but `ab` is `b2`, `ba` is `a1`. The last writer wins. The primary key prevents duplicate rows; it does not say who wins in a collision. The class is **order-sensitive**. In the third line, the database generates the key. Inserting the same value twice leaves two rows: `1=a1` and `1=a1,2=a1`. Since the two operations are identical, the order is meaningless. The class is **repeat-sensitive**. An insert call that is retried is duplicated, because the database counts it as two separate records. In the fourth line, when two different values are inserted in two different orders, the keys are distributed to the values differently as well: `1=a1,2=b2` and `1=b2,2=a1`. The class is **doubly sensitive**. The count is one of each class. #### Who Supplies the Key The measurement gives a practical rule. If the records have a natural identity, for example an identity given by the server or a digest derived from the content, the app should supply the key and the insertion should be written as an upsert. That makes a retry safe. If there is no natural identity, an auto-increment key is used; but a retry is duplicated, and something that prevents it, for example a separate unique column, is added to the design. #### Common Mistakes - Relying on an auto-increment key in an insert call that is retried. The row is duplicated. - Reading without specifying an ordering and assuming that the insertion order is preserved. - Writing a two-step path that reads first and then inserts, instead of an upsert. Another write can slip in between the two steps. - Updating the same row from two sources and assuming that the winner is known. The last writer wins. - Opening a table for every small value. The preference store is enough for that. - Writing many rows one by one, in separate transactions. If they are all written in a single transaction, consistency and speed are gained. **PS11.** The model assumes that every call runs without interruption and from a single connection. The case where two connections write at the same time was not built. #### The Bounding Measurement The robustness of the first line rested on the read order. Next, we measure the same two operations in a table that reads in insertion order, and put it side by side with the sorted one for comparison. ```swift // main.swift — read order let sorted = replay("key-order-read", from: TableBox(sortedRead: true), upserting("a1", key: "k1"), then: upserting("b2", key: "k2")).report() let insertion = replay("insertion-order-read", from: TableBox(sortedRead: false), upserting("a1", key: "k1"), then: upserting("b2", key: "k2")).report() countSensitivities([sorted, insertion]) ``` ``` key-order-read | once=k1=a1 twice=k1=a1 ab=k1=a1,k2=b2 ba=k1=a1,k2=b2 -> robust insertion-order-read | once=k1=a1 twice=k1=a1 ab=k1=a1,k2=b2 ba=k2=b2,k1=a1 -> order-sensitive robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` The operation and the data are the same; only the read changed. Reading sorted by key is robust, and reading in insertion order is order-sensitive: `k1,k2` and `k2,k1`. The content of the table is the same two rows in both cases; the difference is how the rows are listed. This is a mistake seen everywhere a set is read like an ordered sequence: the result affects not the set but the listing. The limit has two sides. The first is where PS9 is used: reading in insertion order is not guaranteed in a real query engine, and here it is only an option of the model. The second is what the rule does not see: the rule says that the two states are not equal; it does not say that the difference comes from the read order. A person supplies the cause. #### Where It Shows Up in a Real App An automatically generated key is the default choice in most tables, and the `auto-same-value` and `auto-two-values` lines of the measurement show its price. When the code that saves a record runs again, the database gives a new identity each time, and the same record sits as two rows. Pressing the submit button of an order form twice, retrying the request after a network error, or the app restarting a half-finished job produces this duplication. The list screen shows two identical orders; the user sees this as a bug, while the app sees two separate valid records. When the caller supplies the key, the table refuses the duplication, but a different problem appears: if two operations write two different values to the same key, the value that remains depends on which one ran last. The `upsert-same-key` line shows this. It tells us that order sensitivity continues even in a place where tables are counted as robust. Operations writing to separate keys do not affect each other, and the `upsert-distinct-keys` line comes out robust. So in a database, the label "robust" depends not on the table but on which keys the operation touches. The same table is robust when working with non-colliding keys and order-sensitive when writing to the same key. That this lesson's sensitivity label is robust rests on the most protected form of operation in tables, that is, the keyed update. #### A Wrong Setup and Its Fix The most common wrong setup is to query first whether the record exists and then insert. First it is checked "does this record exist", and if not, it is inserted. If two operations ask this question at the same time, both get the answer "no" and both insert. The gap between the check and the insertion is where repetition leaks in. The fix is to have the table itself do the check instead of leaving it as a separate step. The key is supplied by the caller, as a stable value derived from the record, and the table refuses a second insertion of the same key or updates the existing row. That way "does it exist, otherwise insert" becomes a single operation and there is no gap left to slip into. In the language of the measurement: the operation stops being repeat-sensitive. The second mistake is trusting the order of a query result. For a query with no ordering specified, the order in which it returns rows is not guaranteed. Even if the order of a screen coincides with the insertion order today, it can change when an index is added to the table or when the data grows. The insertion-order line in the bounding measurement used exactly this, and PS9 recorded that this assumption is an option of the model. The fix is to specify the ordering explicitly in every query where order matters; key order is the form of reading that came out robust in the measurement. #### Reading the SOURCE Fragment The unnamed fragment at the start of the lesson is a sequence of connection calls that defines a table, inserts a row into it, and then queries it. A real connection opens a file, prepares the query as a statement, binds the values to the statement, and runs it step by step. The single call in our model is, in reality, the sum of these four steps; PS10 records this simplification and the table being limited to a single primary key column. The values in the fragment are written directly into the query text for brevity; if text coming from the user is joined into the query this way, it can change the query itself. In real code the values are bound to the prepared statement, and the user's text is never joined into the query text. This is an important habit from a security point of view and has nothing to do with the measurements of this lesson; it is still noted so that its place in real code is known. The assumption that the connection works without interruption in every call is PS11: the case where two connections write at the same time was not measured, and in real databases this case enters separate topics such as locks and transaction isolation. #### From Measurement to Design The result of this lesson says why tables are a safe default and how far that safety extends. A keyed update is robust, but only when writing to separate keys. If two different values are written to the same key, the order is still decisive. What the table adds is that the collision becomes visible and the rule is written in the table: does the last writer win, or the first, or is the second write refused. Expressing this rule in the schema instead of hiding it in code makes every write path obey the same rule. Three questions are enough for design. Does the record have a natural identity? If it does, deriving the key from it makes the table refuse repetition. If it does not, a generated identity always carries a duplication risk, and this risk is managed on the operation side, for example with a key generated once for each request. Can two operations write to the same record? If they can, the rule must be written down. Does the read order matter? If it does, the ordering must be specified explicitly. None of these three questions depends on the choice of database. However powerful the query language is, how the operation behaves against repetition and order depends on how the record is defined. This is what the measurement teaches: what is robust is not the table, but the operation in which the table is used correctly. A small extra warning: the measurement looks at a single table. In a real database, two tables are connected by a relation, and an operation has to write to both. In that case a write left half done can leave inconsistent data; this topic belongs to transaction isolation and is outside the scope of this lesson's measurement. #### Summary - If the data is divided into rows and fields and searched, a local database is chosen instead of a file. - An upsert with a primary key is robust with two different keys and with a sorted read. - Writing two different values to the same key is order-sensitive: `ab` is `b2`, `ba` is `a1`. - Inserting with an auto-increment key grows the table from `1=a1` to `1=a1,2=a1`; with two different values it is doubly sensitive. - The same two operations are robust if the read is sorted by key and order-sensitive if it is done in insertion order. - This lesson's sensitivity: **robust**. #### Next Step Instead of using the database directly with queries, there are frameworks that work with objects. In the next lesson, Object Persistence Frameworks, we measure that creating and saving an object in a context is repeat-sensitive, and how a unique constraint changes that. ### Object Persistence Frameworks Source: https://academia.sh/en/courses/data-networking-and-concurrency/object-persistence-frameworks A measurement of how creating and saving objects in a context is repeat-sensitive, how saving with a unique constraint is order-sensitive, and how the constraint changes the class. In the previous lesson we approached the database at the level of rows and primary keys. Most apps think not in rows but in **objects** of their own types: a note, an order, a person. An **object persistence framework** tracks these objects inside a **context**, accumulates the changes, and writes them to the store underneath when saved. The counterpart of a row is an object; the counterpart of a primary key is most often a unique constraint. Our question does not change: **if this operation is repeated or reordered, does the result change?** ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` `once` is the operation applied to the starting state one time, `twice` is two times; `ab` is `a` first and then `b`, and `ba` is the reverse order. The model is the pure Swift counterpart of a context. It is not the framework itself; it only builds the result of creating and saving an object. #### Context, Object, and Save A context is the manager of a set of objects living in memory. The app creates a new object, adds it to the context, and saves. Until it is saved, the change exists only in memory. There are two ways of writing it: the older framework uses a persistent container and a managed object context; the newer framework uses classes marked with `@Model` and a model context. ```swift import SwiftData @Model final class Note { @Attribute(.unique) var id: String var body: String init(id: String, body: String) { self.id = id self.body = body } } let container = try ModelContainer(for: Note.self) let context = ModelContext(container) context.insert(Note(id: "k1", body: "a1")) try context.save() ``` This fragment is SOURCE; it needs a model container and a disk, so it does not run here. `@Attribute(.unique)` makes an attribute unique: a second object that arrives with the same `id` does not open a new record, it updates the existing one. If the attribute is left without a constraint, every insertion becomes a new record. In the older framework the same purpose is served by a uniqueness constraint and a merge policy. The measurement of this lesson compares the two cases. #### The Model of the Context The model keeps a list of records. Each record is a key and a value. If the constraint is on, a record that arrives with the same key takes the place of the existing record; if it is off, it is added as a new record. For an object created without a key, the context supplies its own key. ```swift // ContextModel.swift — a pure Swift model of a context struct ContextBox: Equatable, CustomStringConvertible { struct Entry { let key: String var value: String } private(set) var entries: [Entry] = [] private(set) var nextId = 1 let enforcesUnique: Bool init(enforcesUnique: Bool = true) { self.enforcesUnique = enforcesUnique } var description: String { entries.isEmpty ? "-" : entries.map { "\($0.key)=\($0.value)" }.joined(separator: ",") } static func == (lhs: ContextBox, rhs: ContextBox) -> Bool { lhs.description == rhs.description } mutating func create(_ value: String) { entries.append(Entry(key: "obj\(nextId)", value: value)) nextId += 1 } mutating func save(_ value: String, key: String) { if enforcesUnique, let index = entries.firstIndex(where: { $0.key == key }) { entries[index].value = value } else { entries.append(Entry(key: key, value: value)) } } } func creating(_ value: String) -> (ContextBox) -> ContextBox { { context in var next = context next.create(value) return next } } func saving(_ value: String, key: String) -> (ContextBox) -> ContextBox { { context in var next = context next.save(value, key: key) return next } } ``` **PS12.** The model assumes that when the same unique key arrives, the last writer wins: the value of the existing record is replaced with the new one. The framework's real merge policy and its settings were not measured. **PS13.** The model builds every operation as adding and saving immediately. Unsaved changes, undo, and multiple contexts were not built. #### Measuring Four Operations Four operations are built. The first is creating a new object with the same value. The second is creating two objects with two different values. The third is saving the same value twice to a unique key. The fourth is saving two different values to the same key. ```swift // main.swift — object creation and unique save let createSame = replay("create-same-value", from: ContextBox(), creating("a1"), then: creating("a1")).report() let createTwo = replay("create-two-values", from: ContextBox(), creating("a1"), then: creating("b2")).report() let uniqueSame = replay("constrained-same-value", from: ContextBox(), saving("a1", key: "k1"), then: saving("a1", key: "k1")).report() let uniqueTwo = replay("constrained-two-values", from: ContextBox(), saving("a1", key: "k1"), then: saving("b2", key: "k1")).report() countSensitivities([createSame, createTwo, uniqueSame, uniqueTwo]) ``` ``` create-same-value | once=obj1=a1 twice=obj1=a1,obj2=a1 ab=obj1=a1,obj2=a1 ba=obj1=a1,obj2=a1 -> repeat-sensitive create-two-values | once=obj1=a1 twice=obj1=a1,obj2=a1 ab=obj1=a1,obj2=b2 ba=obj1=b2,obj2=a1 -> doubly-sensitive constrained-same-value | once=k1=a1 twice=k1=a1 ab=k1=a1 ba=k1=a1 -> robust constrained-two-values | once=k1=a1 twice=k1=a1 ab=k1=b2 ba=k1=a1 -> order-sensitive robust: 1 repeat-sensitive: 1 order-sensitive: 1 doubly-sensitive: 1 ``` When creating a new object with the same value is repeated, it leaves two objects: `obj1=a1` and `obj1=a1,obj2=a1`. The order is meaningless. The class is **repeat-sensitive**. Creation code that runs a second time because a screen appears again or because of a retry shows up to the user as a duplicate record. Creation with two different values separates both repetition and order: the identities are distributed to the values differently. The class is **doubly sensitive**. Saving the same value twice to a unique key changes nothing. The constraint makes repetition robust. The class is **robust**. When two different values are saved to the same key, the order decides the result: `ab` is `b2`, `ba` is `a1`. The constraint prevents a duplicate record, but which value wins in a collision is left to the order. The class is **order-sensitive**. The count is one of each class. #### Schema Migration The object type of an app changes between versions: an attribute is added, a name changes, two types merge. On the user's device, meanwhile, the data written by the old version sits there. **Schema migration** is the step that carries data written with the old schema to the new schema. Simple changes, for example adding an attribute, can be done by the framework automatically; changes such as renaming or merging two fields need an explicit mapping. Schema migration was not built in the model of this lesson. This is because its result depends on the old data in the store and on the order of versions; it cannot be read from the in-memory state of a context. Here it stays as SOURCE and prose. The practical rule is simple: no version that changes the schema is released without being tried with data written by the old version. #### Common Mistakes - Tying creation code to the appear event of a screen. A new object is born every time the screen appears. - Not making a natural identity unique when the record has one. A retry is duplicated. - Assuming that a uniqueness constraint resolves a collision. In a collision, the order decides the winner. - Forgetting to save. The change is in memory and is lost when the app closes. - Sharing a context across threads. The framework does not support this; each thread uses its own context. - Changing the schema and releasing it without trying it with old data. #### The Bounding Measurement To isolate the contribution of the constraint, we measure the same pair of records with the constraint off. If the constraint is off, a record that arrives with the same key does not take the place of the existing record; it becomes a new record. ```swift // main.swift — constraint on and off let openSame = replay("unconstrained-same-value", from: ContextBox(enforcesUnique: false), saving("a1", key: "k1"), then: saving("a1", key: "k1")).report() let openTwo = replay("unconstrained-two-values", from: ContextBox(enforcesUnique: false), saving("a1", key: "k1"), then: saving("b2", key: "k1")).report() let closedSame = replay("constrained-same-value-again", from: ContextBox(), saving("a1", key: "k1"), then: saving("a1", key: "k1")).report() countSensitivities([openSame, openTwo, closedSame]) ``` ``` unconstrained-same-value | once=k1=a1 twice=k1=a1,k1=a1 ab=k1=a1,k1=a1 ba=k1=a1,k1=a1 -> repeat-sensitive unconstrained-two-values | once=k1=a1 twice=k1=a1,k1=a1 ab=k1=a1,k1=b2 ba=k1=b2,k1=a1 -> doubly-sensitive constrained-same-value-again | once=k1=a1 twice=k1=a1 ab=k1=a1 ba=k1=a1 -> robust robust: 1 repeat-sensitive: 1 order-sensitive: 0 doubly-sensitive: 1 ``` The same pair of records came out repeat-sensitive with the constraint off and robust with it on. The pair with two values was doubly sensitive without the constraint and order-sensitive with it. The constraint solved repetition and brought two rows down to one row; it did not solve order. The limit of this measurement is PS12. The model assumes that the last writer wins in a collision. In the real framework the merge policy can be set, and some policies keep the existing record; none was measured in this lesson. If the policy changes, `ab` and `ba` can swap places, but the conclusion that the two will not be equal does not change. #### Where It Shows Up in a Real App Object persistence frameworks turn an object graph into rows on your behalf; this convenience is also the reason repetition becomes invisible. When the user creates a note, the app makes a new object and saves the context. If the same code path runs again in a retry or a screen refresh, the framework produces a new object each time. This is the duplication we saw in the `create-same-value` line: two calls, two objects. The same note stands twice on the list screen, and no error was given. A uniqueness constraint prevents this duplication. Once an attribute is marked unique, a second object with the same value cannot be added; the framework either merges the new object with the existing one or rejects the save. In the `constrained-same-value` line, two calls give a single object and the class is robust. But the constraint does not resolve two different values arriving with the same unique key: in the `constrained-two-values` line the remaining value depends on the order. The constraint solves repetition, not order. In a real app this shows up when records coming from the server are matched with the local copy. The server gives a stable identity for each record; if the local record is made unique with that identity, processing the same response twice produces one record. Without the identity, the local object is created again and two copies are born. #### A Wrong Setup and Its Fix The common wrong setup is to check uniqueness in code: "is there an object with this value, otherwise create it." The gap between the check and the creation is where two operations can both get the answer "no." Besides, the check looks only at the objects in the context; an object in another context or one not yet saved is missed. The fix is to make uniqueness a part of the schema. The constraint sits in the storage layer and is applied on every write; it does not matter which path the code came from. The merge policy of the framework is also decisive here. The policy says which value stays in a collision. Our model assumed that the last writer wins, and this is recorded as PS12. If another policy keeps the existing record, `ab` and `ba` swap places; but the fact that the two orders do not give the same result does not change. The second mistake is forgetting to save. Code that creates the object and does not save the context assumes the change is permanent. When the app closes, the object is lost. Our model adds and saves every operation immediately; PS13 records this. In real code, who makes the save call must be determined explicitly everywhere work is done. #### Reading the SOURCE Fragment The unnamed fragment at the start of the lesson defines an object type, opens a context, creates the object, and saves. The object type specifies the attributes that will be persistent, and the framework turns them into a table. What the fragment really shows is that the call you make with "save" is in fact a transaction boundary: the changes made up to this call stay in memory, and with the call they go to disk. The model reduced this to a single step and did not build unsaved changes, undo, and multiple contexts. These are important situations in a real app; but the question we measured, that is, repeating the same operation and changing its order, stays meaningful with this simplification too. The version change described in the schema migration section is a separate topic, and in this lesson it is only described, not measured. #### From Measurement to Design The measurement of the object framework separates what the constraint does from what it does not do. Without the constraint, repeated creation creates duplication; with the constraint, the same calls give a single object. But the constraint resolves only equality. Which of two different values that arrive with the same key stays depends on the merge policy of the framework. So the design requires making two decisions separately: which attribute determines identity, and which value stays in a collision. The choice of identity is a data model decision. For records coming from the server, the identity given by the server is the natural candidate. For records the user creates, the app produces an identity and writes this identity into the record itself at the moment of creation, once. Producing the identity again on every call makes the constraint meaningless, because every call carries a different identity and no collision ever arises. The policy decision is a product decision. Letting the last writer win is simple and carries a risk of data loss; keeping the existing record is safe but throws away the new information; merging field by field gives the best result and needs the most work. The measurement does not say which is right. The measurement only shows which class the chosen policy leads to. There is also schema migration. The data type changes between versions, and the data written by the old version must be carried to the new schema. Simple additions can be done automatically by the framework; renaming and merging want an explicit mapping. A migration is an operation that runs once in each version. If it is repeated or cut halfway, the data must remain either in the old or in the new form; a half-finished migration is the most expensive form of the repeat sensitivity we measured. A situation outside the measurement must also be noted: related objects. If there is a relation between a note and its tags, what does deleting the note do to the tags, and what does deleting a tag do to the note? The framework asks for the answer to this question as a delete rule: break the relation, delete the connected objects too, or refuse the deletion. If no rule is given, the default behavior takes effect, and most often orphan objects remain. The model of this lesson does not build relations; but in a real schema this decision is as important as the uniqueness constraint and should be written with the same care. The connection of the delete rule to the measurement is direct. If a delete operation is repeated, the second deletion targets an object that no longer exists; whether the framework gives an error in this case decides whether the deletion is robust against repetition. Not assuming this without measuring it is the general habit of this lesson. #### Summary - An object persistence framework tracks objects in a context and writes them to the store when saved. - When creating a new object with the same value is repeated, it leaves two objects (`obj1=a1` then `obj1=a1,obj2=a1`); the class is repeat-sensitive. - A save with a unique constraint is robust with the same value and order-sensitive with a different value for the same key (`ab` is `b2`, `ba` is `a1`). - If the constraint is removed, the same pair of records becomes repeat-sensitive; the pair with two values becomes doubly sensitive. - Schema migration and the merge policy were not measured; they stayed as SOURCE and prose. - This lesson's sensitivity: **repeat-sensitive**. #### Next Step Until now all the data stayed on a single device. In the next lesson, Cloud Sync, we will measure the case where two devices write the same record, and see how the values of the devices arriving in a different order changes the result. ### Cloud Sync Source: https://academia.sh/en/courses/data-networking-and-concurrency/cloud-sync A measurement of how the arrival order of values decides the result when two devices write the same record, how a per-field set union is robust, and how the server rule changes the class. The first five lessons of this topic dealt with the data on a single device. If the user has a phone, a tablet, and a computer, the same record stands in several places. **Cloud sync** brings these copies closer to each other through a server: each device sends the changes it makes, the server merges them, and the other devices pull the result. No new question arrives; the existing question is asked in a new place: **if this operation is repeated or reordered, does the result change?** The difference is this: the two devices do not wait for each other, so the order in which the writes reach the server may not be in the app's hands. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` `once` is the operation applied to the starting state one time, `twice` is two times; `ab` is `a` first and then `b`, and `ba` is the reverse order. The model is the pure Swift counterpart of a single record on the server. It does not build the network, the delay, or the real delivery order; it only builds the result a write leaves in the record. #### Conflict and Rule If two devices write different values to the same field, a **conflict** arises on the server. A rule is needed: which value stays? The most common rule is that the last writer wins; the value that reaches the server later takes the place of the earlier one. Another rule is to put a stamp on every write and keep the one with the highest stamp. A third way is to merge the value itself: if two devices added different tags to a set of tags, the result is the union of the two. ```swift import CloudKit let recordID = CKRecord.ID(recordName: "k1") let record = CKRecord(recordType: "Note", recordID: recordID) record["theme"] = "dark" let database = CKContainer.default().privateCloudDatabase database.save(record) { saved, error in if let error = error as? CKError, error.code == .serverRecordChanged { // the record on the server has changed in the meantime; the app applies its own rule } } ``` This fragment is SOURCE; it needs a cloud account and a network, so it does not run here. The `serverRecordChanged` error reports that the server's version of the record that is being saved has changed in the meantime. Resolving the conflict is left to the app: the version on the server returns with the error, and the app decides which value stays. The measurement of this lesson compares three forms of this decision. The error itself and the real delivery order are not measured. #### The Model of the Record The model keeps the scalar fields and the set fields in a single record. Each scalar value carries a stamp. The server's rule is a setting: an incoming write either always wins, or wins only if its stamp is not smaller than the current stamp. ```swift // SyncModel.swift — a pure Swift model of a record on the server struct SyncBox: Equatable, CustomStringConvertible { enum Rule { case lastArrival case highestStamp } struct Stamped { let value: String let stamp: Int } private(set) var scalars: [String: Stamped] = [:] private(set) var sets: [String: Set] = [:] let rule: Rule init(rule: Rule = .lastArrival) { self.rule = rule } var description: String { let scalarText = scalars.keys.sorted().map { "\($0)=\(scalars[$0]?.value ?? "-")@\(scalars[$0]?.stamp ?? 0)" } let setText = sets.keys.sorted().map { "\($0)={\((sets[$0] ?? []).sorted().joined(separator: ","))}" } let all = scalarText + setText return all.isEmpty ? "-" : all.joined(separator: " ") } static func == (lhs: SyncBox, rhs: SyncBox) -> Bool { lhs.description == rhs.description } mutating func write(_ value: String, stamp: Int, field: String) { if rule == .highestStamp, let current = scalars[field], current.stamp > stamp { return } scalars[field] = Stamped(value: value, stamp: stamp) } mutating func merge(_ items: [String], field: String) { sets[field, default: []].formUnion(items) } } func writing(_ value: String, stamp: Int, field: String) -> (SyncBox) -> SyncBox { { record in var next = record next.write(value, stamp: stamp, field: field) return next } } func merging(_ items: [String], field: String) -> (SyncBox) -> SyncBox { { record in var next = record next.merge(items, field: field) return next } } ``` The model builds the two devices as two separate operations; `ab` is the case where the first device's write reaches the server first, and `ba` is the case where the second one's does. **PS14.** The model assumes that the server applies the rule consistently: the rule is a setting and works on every write exactly. The real delivery order, the delay, and `serverRecordChanged` were not measured. **PS15.** The stamps are numbers given to the model; no clock is read or stamped. The drift of real device clocks was not built. **PS16.** The model assumes that every write reaches the server. Lost, delayed, and repeated delivery were not built. #### Measuring Two Operations Two operations are built. The first is two devices writing different values to the same field. The second is two devices adding different tags to the same set of tags. ```swift // main.swift — two devices write the same record let twoDevices = replay("two-devices-same-field", from: SyncBox(), writing("dark", stamp: 1, field: "theme"), then: writing("light", stamp: 2, field: "theme")).report() let tags = replay("set-union", from: SyncBox(), merging(["a1"], field: "tags"), then: merging(["b2"], field: "tags")).report() countSensitivities([twoDevices, tags]) ``` ``` two-devices-same-field | once=theme=dark@1 twice=theme=dark@1 ab=theme=light@2 ba=theme=dark@1 -> order-sensitive set-union | once=tags={a1} twice=tags={a1} ab=tags={a1,b2} ba=tags={a1,b2} -> robust robust: 1 repeat-sensitive: 0 order-sensitive: 1 doubly-sensitive: 0 ``` In the first line, repeating the same write changes nothing: `once` and `twice` are the same. But when the values of the two devices reach the server in a different order, the result changes: `ab` is `light`, `ba` is `dark`. The last writer wins, and the arrival order decides the winner, that is, the network and not the app. The class is **order-sensitive**. This is the end of the chain that has been building up since the beginning of the topic: the preference of two screens, the repeated keychain item, the append to a file, the row with a primary key, the object created twice, and now the value written by two devices. At every step a local problem showed up again in another place in the next one. In the second line, two devices add different tags to the same set. Repetition does not change the set, and the order does not change the result either: both orders give `{a1,b2}`. A set union makes the order irrelevant, because no value takes the place of the other. The class is **robust**. This is why a per-field merge is the most durable way of resolving conflicts; its cost is that every field must have a structure that can be merged. A theme has no union. The count is one robust, one order-sensitive. #### What to Merge The measurement returns to the app's data model as a question: can this field be merged? Tags, read marks, and the list of attached files merge. A theme, a name, a date do not merge; a winner has to be chosen for these. Using a stamp when choosing the winner reduces the dependence on order but does not remove it. The bounding measurement below shows this. #### Common Mistakes - Assuming that the last writer will win and not noticing the data loss. The losing value is silently deleted. - Writing a field that can be merged as a scalar. One of the tags added by the two devices is lost. - Trusting the device clock and taking the stamp from there. Clocks drift. - Ignoring the conflict error. The record stays unsaved and the user does not notice. - Running sync immediately on every change. The calls are repeated and the order becomes even less controlled. - Binding a delete made on one device to the same rule as a write on another. A delete is a separate operation and is thought about separately. #### The Bounding Measurement The order sensitivity of the first line rested on the server's rule that lets the latest arrival win. We measure the same two writes with a rule that keeps the one with the larger stamp, and we also try once more with equal stamps. ```swift // main.swift — server rule let arrival = replay("rule-arrival-order", from: SyncBox(rule: .lastArrival), writing("dark", stamp: 1, field: "theme"), then: writing("light", stamp: 2, field: "theme")).report() let stamped = replay("rule-stamp", from: SyncBox(rule: .highestStamp), writing("dark", stamp: 1, field: "theme"), then: writing("light", stamp: 2, field: "theme")).report() let tie = replay("rule-stamp-tie", from: SyncBox(rule: .highestStamp), writing("dark", stamp: 1, field: "theme"), then: writing("light", stamp: 1, field: "theme")).report() countSensitivities([arrival, stamped, tie]) ``` ``` rule-arrival-order | once=theme=dark@1 twice=theme=dark@1 ab=theme=light@2 ba=theme=dark@1 -> order-sensitive rule-stamp | once=theme=dark@1 twice=theme=dark@1 ab=theme=light@2 ba=theme=light@2 -> robust rule-stamp-tie | once=theme=dark@1 twice=theme=dark@1 ab=theme=light@1 ba=theme=dark@1 -> order-sensitive robust: 1 repeat-sensitive: 0 order-sensitive: 2 doubly-sensitive: 0 ``` The same two writes fell into two separate classes depending on the rule. It is order-sensitive under the arrival order rule; robust under the highest stamp rule, because the write with the larger stamp wins whichever order it arrives in. When the stamps were equal, the order came back: `ab` is `light`, `ba` is `dark`. The stamp did not remove the order; it only narrowed the case in which the order comes into play. The limit is PS14. The model assumes that the server applies the rule correctly and consistently. In a real service, where and how the rule is applied, when the error is returned, and in what order the writes arrive are outside this model. What is measured is the result of the rule; not that the rule really works. #### Where It Shows Up in a Real App Think of a notes app. The user changes the title of a note on the phone, and at the same time turns the title of the same note into something else on the tablet. Both devices are offline and connect a few minutes later. Whichever reaches the server first, the second to arrive takes the place of the first; the title that remains is decided by the delay of the network, not by the user. The `two-devices-same-field` line in the measurement is the model of this situation. Tags are different. The phone adds a tag, the tablet adds another tag. The field is kept as a set, and the server records the union of the two sets. Which device reached first no longer matters: in the `set-union` line both orders gave the same result. This difference is the first question to ask when building the data model of an app: can this field be merged, or must a winner be chosen? Deletion brings a separate problem. One device deletes the record, the other updates the same record. Will the server count the record as deleted or as updated? The model of this lesson does not build deletion; but in a real sync design this decision is as important as the scalar conflict rule and must be made explicitly. #### A Wrong Setup and Its Fix The most common wrong setup is for every device to stamp all the records with its local clock and let the newest stamp win. Device clocks drift; the user can change the clock by hand, and when the clock of one device is wrongly ahead, all its writes crush the correct writes made later. As we showed in the bounding measurement, the stamp rule reduces the dependence on order but the order returns when the stamps are equal. When real clocks are involved, this equality is not rare. The fix has two layers. The first is to take the stamp not from the device clock but from an increasing version number given by the server; that way the stamps are comparable and the drift has no effect. The second is to move every field that can be, to a structure that can be merged. A set of tags, a set of read marks, and a list of attached files are of this class. No conflict rule is needed for merged fields. The second mistake is ignoring the conflict error. If the app stays silent when the server reports that the record has changed in the meantime, the user's change has never been saved, and the user does not notice. The correct setup is to take the server version that returns with the error, apply the app's own rule, and send the result again. PS15 records that the model takes the stamps as numbers given from outside; real clock drift is not in the model. #### Reading the SOURCE Fragment The unnamed fragment at the start of the lesson creates a record, writes a value to a field of it, and saves it to a private database. The completion of the save call reports the result of the operation as an error or a saved record. One of the error types says that the server's version of the record that is being saved has changed in the meantime. This error is not an exception but a normal outcome: in an app with several devices, a conflict happens sooner or later. Our model does not build this error or the real delivery order; PS14 assumes that the server applies the rule consistently, and PS16 that every write reaches the server. On a real network, writes are lost, delayed, and repeated. A repeated write being applied a second time on the server brings the repeat sensitivity question back onto the agenda, and it will be taken up in the next lesson, in the context of notifications arriving twice. #### From Measurement to Design What the measurement says in a sync design is that a decision has to be made field by field. For each field one of three options is chosen: turning it into a structure that can be merged, choosing a winner with a stamp, or asking the user about the conflict. Tags, marks, and lists fall into the first group; they came out robust in the measurement. Fields such as a name, a theme, and a date are in the second group and at best narrow the dependence on order. The third group is rare, but for some data it is the only honest option; the user choosing between two different texts is better than silently deleting one. The rule must always be written down. "The last writer wins" is a rule, and a correct one, as long as it was chosen consciously. What is dangerous is that the rule is never thought about and the server's default becomes the product's rule. The chain of this topic closes here. In the simple preference, the writing of two screens depended on the order; in the keychain, the insertion on repetition; in the file, on both repetition and order; in the table, on the key; in the object framework, on the identity; and in sync, on the nature of the field. In every layer the same two questions were asked: what happens if it is repeated and if its order changes. The answers changed, the question did not. Each layer of persistence does not bring a new flaw; it houses the same two flaws in a new place. One last note: the question of how a sync success is shown to the user is also a part of the design. The change shows up locally at once and goes to the server later. If the user does not know that the value has not been sent yet and closes the device, the change stays in a waiting queue. The pending operations have to be stored and sent again when the connection returns; sending again is also repetition. So the pending writes of sync must be robust against repetition. #### Summary - Cloud sync lets several devices write the same record; a rule decides which value stays in a conflict. - The different values written by two devices to the same field depend on the arrival order: `ab` is `light`, `ba` is `dark`; the class is order-sensitive. - A per-field set union is robust: both orders give `{a1,b2}`. - The highest stamp rule makes the same pair of writes robust, but the order returns when the stamps are equal. - The server rule, the real delivery order, and the conflict error were not measured; they stayed as PS14. - This lesson's sensitivity: **order-sensitive**. #### Next Step Sync described how a value travels between devices. In the next lesson, Push Notifications, we move on to the path on which the server informs the app on its own: next to a device token, a permission, and a badge, we will measure whether the same notification arriving twice changes the result. ### Push Notifications Source: https://academia.sh/en/courses/data-networking-and-concurrency/push-notifications A measurement of how raising the badge locally on a notification from the server is repeat-sensitive and how an absolute badge value in the payload is order-sensitive, and the wrap-up table of the nineteen lessons of the course. In the previous lesson, two devices writing the same record took the order out of the app's hands. In a push notification, the order and the count move one step further away: the server prepares the message, the platform's push notification service carries it, and the app sees it only when it is delivered. This service **does not guarantee** delivery; a notification can be delayed, can never arrive, or can arrive twice. The question of the course is the same at this end too: **if this operation is repeated or reordered, does the result change?** This lesson is our last one; after the measurement, the nineteen lessons of the course are gathered in a single table. ```swift // Replay.swift — classifies an operation's sensitivity to repetition and order struct Trial { let label: String let once: State let twice: State let forward: State let backward: State var sensitivity: String { switch (once != twice, forward != backward) { case (false, false): return "robust" case (true, false): return "repeat-sensitive" case (false, true): return "order-sensitive" case (true, true): return "doubly-sensitive" } } func report() -> String { print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)") return sensitivity } } func replay(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial { Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start))) } func countSensitivities(_ results: [String]) { for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] { print("\(name): \(results.filter { $0 == name }.count)") } } ``` `once` is the operation applied to the starting state one time, `twice` is two times; `ab` is `a` first and then `b`, and `ba` is the reverse order. The model below builds the state of the app that changes with a notification: a badge count, a device token, and the handled notification identifiers. #### Permission and Registration Flow The app takes two steps to be able to receive notifications. First it asks the user for permission, then it asks the platform for a device token. The token is the address of this app on this device and is passed to the server; the server sends the notification to this address. The fragment is SOURCE, because it needs a real permission screen and a network connection. ```swift import UIKit import UserNotifications let center = UNUserNotificationCenter.current() let granted = try await center.requestAuthorization(options: [.alert, .badge, .sound]) if granted { await MainActor.run { UIApplication.shared.registerForRemoteNotifications() } } func application(_ application: UIApplication, didRegisterForRemoteNotificationsWithDeviceToken deviceToken: Data) { let hex = deviceToken.map { String(format: "%02x", $0) }.joined() print(hex) } ``` If the user refuses the permission, the registration call is not made and no notification arrives; after a refusal the permission can be turned on from settings, so the app reads the permission status again at every launch. The token is not fixed: the platform can renew it, so the app renews the registration call at every launch and reports the incoming value to the server again. The result of this repetition is one of the three operations this lesson measures. #### The Model of the Notification The model keeps three states. The badge is the number on the app's icon. The token is the last saved device address. The handled set is the notification identifiers the app has processed before, and it fills only in the operation that filters by identifier. When printing, the handled set is not shown if it is empty. ```swift // PushModel.swift — a pure Swift model of the badge, token, and handled notifications struct PushState: Equatable, CustomStringConvertible { private(set) var badgeCount: Int private(set) var token: String private(set) var handled: Set init(badgeCount: Int, token: String = "-", handled: Set = []) { self.badgeCount = badgeCount self.token = token self.handled = handled } var description: String { let base = "badge=\(badgeCount) token=\(token)" return handled.isEmpty ? base : base + " handled=" + handled.sorted().joined(separator: ",") } func incrementing() -> PushState { PushState(badgeCount: badgeCount + 1, token: token, handled: handled) } func setting(badgeCount value: Int) -> PushState { PushState(badgeCount: value, token: token, handled: handled) } func registering(token value: String) -> PushState { PushState(badgeCount: badgeCount, token: value, handled: handled) } func incrementing(onceFor id: String) -> PushState { guard !handled.contains(id) else { return self } return PushState(badgeCount: badgeCount + 1, token: token, handled: handled.union([id])) } } func incrementingBadge() -> (PushState) -> PushState { { $0.incrementing() } } func settingBadge(_ value: Int) -> (PushState) -> PushState { { $0.setting(badgeCount: value) } } func registeringToken(_ value: String) -> (PushState) -> PushState { { $0.registering(token: value) } } func incrementingOnce(id: String) -> (PushState) -> PushState { { $0.incrementing(onceFor: id) } } ``` **PS21.** The model builds each notification as an operation applied exactly once. How often the platform can deliver the same notification zero, one, or several times was not measured; what is measured is what happens if the delivery is twice. **PS22.** The absolute badge value is computed on the server and carried in the payload; the model assumes the value is computed correctly. How the server finds this number was not measured. **PS23.** The notification identifier is assumed to be unique and fixed. In a real service, how the identifier is produced and whether it is kept in a retry were not measured. #### Measuring Three Operations Three operations are built. The first is raising the badge locally at every delivery. The second is carrying the absolute badge value in the payload: the server writes the last count into the `badge` field inside `aps`, and the app uses it as it is. The third is registering the same token again. The badge starts at three. ```swift // main.swift — badge and token operations let start = PushState(badgeCount: 3) let local = replay("badge-local-increment", from: start, incrementingBadge(), then: incrementingBadge()).report() let absolute = replay("badge-absolute-value", from: start, settingBadge(4), then: settingBadge(7)).report() let register = replay("token-reregister", from: start, registeringToken("t1"), then: registeringToken("t1")).report() countSensitivities([local, absolute, register]) ``` ``` badge-local-increment | once=badge=4 token=- twice=badge=5 token=- ab=badge=5 token=- ba=badge=5 token=- -> repeat-sensitive badge-absolute-value | once=badge=4 token=- twice=badge=4 token=- ab=badge=7 token=- ba=badge=4 token=- -> order-sensitive token-reregister | once=badge=3 token=t1 twice=badge=3 token=t1 ab=badge=3 token=t1 ba=badge=3 token=t1 -> robust robust: 1 repeat-sensitive: 1 order-sensitive: 1 doubly-sensitive: 0 ``` The first line is the same as the `badge-increment` operation in the Network Client lesson: one delivery raises the badge to four, two deliveries to five. If the notification is delivered twice, the user sees an unread message that does not really exist. The operation is **repeat-sensitive**. In the second line the payload carries the number, so the second delivery of the same notification does not change the result: `once` and `twice` are the same. But two different notifications carry different numbers, and the badge stays at the value of the one that comes later: `ab` is seven, `ba` is four. The absolute value solved repetition, not order; the operation is **order-sensitive**. It has the same structure as the last writer wins rule of the previous lesson. In the third line, registering the same value again does not change the state; the order is meaningless too, because the two operations are the same. The operation is **robust**. This is why renewing the token is safe: the server acts as if it received the same address twice. The count is one robust, one repeat-sensitive, one order-sensitive. #### Delivery Is Not Guaranteed The measurement builds the case where a notification is delivered twice as two applications of the operation. In reality, how many times and in what order the delivery will happen is in the platform's hands. A notification may never arrive: the device is off, it cannot reach the network, the user has withdrawn the permission, or the platform has dropped the notification. When two notifications are sent, the second can arrive first. So a notification cannot be the app's single source of truth. Two conclusions follow. The first is that the information a notification carries must be readable again from the server when the app opens; a notification is a hint, not a record. The second is that the local change a notification makes must either be protected against repetition or not be made at all. Raising the badge locally requires the first way, and the absolute value is a form of the second way. This choice is a design decision and not a rule, and the rule does not make it for us. #### The Bounding Measurement The rule sees the inequality; it does not see how many times the delivery happened. We measure the same badge operation again, this time with a notification identifier: the app processes an identifier only once. We also try the case where two notifications carry the same absolute value. ```swift // main.swift — filtering by identifier and equal absolute value let start = PushState(badgeCount: 3) let sameNotification = replay("identified-same-notification", from: start, incrementingOnce(id: "n1"), then: incrementingOnce(id: "n1")).report() let twoNotifications = replay("identified-two-notifications", from: start, incrementingOnce(id: "n1"), then: incrementingOnce(id: "n2")).report() let equalAbsolute = replay("absolute-equal-value", from: start, settingBadge(4), then: settingBadge(4)).report() countSensitivities([sameNotification, twoNotifications, equalAbsolute]) ``` ``` identified-same-notification | once=badge=4 token=- handled=n1 twice=badge=4 token=- handled=n1 ab=badge=4 token=- handled=n1 ba=badge=4 token=- handled=n1 -> robust identified-two-notifications | once=badge=4 token=- handled=n1 twice=badge=4 token=- handled=n1 ab=badge=5 token=- handled=n1,n2 ba=badge=5 token=- handled=n1,n2 -> robust absolute-equal-value | once=badge=4 token=- twice=badge=4 token=- ab=badge=4 token=- ba=badge=4 token=- -> robust robust: 3 repeat-sensitive: 0 order-sensitive: 0 doubly-sensitive: 0 ``` All three operations came out robust. Filtering by identifier ignored the second delivery of the same notification; with two different notifications, the total of the increase is independent of the order, because the handled set is kept as a set and printed in sorted order. The equal absolute value also hid the difference in order: if two notifications carry the same number, nobody can see which one came first. Here the rule fails to see two things. The first is whether the notification was really delivered twice: the measurement does not show whether the filtering works, it only shows that the result will stay the same if the delivery is twice. The second is whether the equal value is a coincidence or a design. The measured class is the result of a setup, not of the nature of the operation. #### Choosing a Storage The six lessons of the persistence topic support a choice: which storage the data goes into. This choice is not reduced to a number; sensitivity is not the only criterion of the choice. The shape of the data determines the criteria. - Small, single-valued, and non-secret settings go into the preference store. - Secret values such as passwords, tokens, and keys go into the keychain. - Documents, media, and caches that the app produces itself go into the file system; which directory is backed up is thought about separately. - Records that are queried, related, and numerous go into the local database or the object persistence framework. - A record that must be the same on several devices goes into cloud sync, and the conflict rule is written from the start. In this list a notification is not a storage; it is an event that changes state. Every change a notification brings is written into one of the places above and inherits the class of that place. #### Summary - The platform's push notification service does not guarantee delivery; a notification can be delayed, can never arrive, or can arrive twice. - Raising the badge locally at every delivery is repeat-sensitive: one delivery raises the badge to four, two deliveries to five. - Carrying an absolute badge value in the payload solves repetition but not order: `ab` gives seven, `ba` gives four; the class is order-sensitive. - Registering the same token again is robust; filtering by identifier also makes the badge increase robust. - The real delivery count, the order, and the production of the identifier were not measured; they stayed as PS21–PS23. - This lesson's sensitivity: **repeat-sensitive**. #### Course Wrap-Up The nineteen lessons of the course moved with the same question: if the operation is repeated or reordered, does the result change? The table gives, in a single row, the operation each lesson measured, the sensitivity written in the lesson's Summary section, and the bounding measurement. | Lesson | Operation | Sensitivity | Bounding measurement | |---|---|---|---| | The Network Client | cache write, keyed send, retry, badge increment, preference write, journal append | all four (1/2/2/1) | starting state: robust from zero, repeat-sensitive from one | | JSON Encoding and Decoding | appending the decoded record to a list, writing by identifier, encode-decode | order-sensitive | equality of raw text and decoded value | | Errors and Retries | order without and with a key, spending a budget | repeat-sensitive | a lost response and a request that did not arrive cannot be told apart | | Authentication | last writer wins, larger stamp wins | order-sensitive | the second use of the refresh token is rejected | | Image and Data Caching | same key, two keys, write-read | robust | empty and full cache | | Background Transfers | enqueueing without checking, recording by identifier | repeat-sensitive | the system decides the order of completion and enqueueing | | Asynchronous Functions | reserving stock, reading stock, separate shelves | order-sensitive | remaining stock and the acceptance ledger | | Tasks and Task Groups | group set, arrival order, group total | robust (in the set), order-sensitive (in the array) | list and count view | | Actors | seat request, ticket sale, release | order-sensitive (seat), repeat-sensitive (counter) | suspension point not measured | | Main Thread Isolation | unversioned and versioned write, flag, counter | order-sensitive (unversioned), robust (versioned) | doubly sensitive if the number is given on the arrival of the response | | Queue-Based Concurrency | append to the end, assigning a value, unique insertion | doubly sensitive (append), order-sensitive (assignment) | sorted list and row count | | Reactive Streams | subscription, dropping repeats, accumulation, latest value | repeat-sensitive (unshared), order-sensitive (different value), robust (shared) | equal and different value pair | | Simple Preference Storage | two screens same key, separate key, launch counter | order-sensitive | three default values | | The Keychain | add, upsert | repeat-sensitive | robust if the status code is not part of the comparison | | The File System | append to the end, atomic write, creating a directory | doubly sensitive | robust when the file is absent, repeat-sensitive when present | | Local Databases | upsert, auto-increment key | robust | read in key order is robust, read in insertion order is order-sensitive | | Object Persistence Frameworks | object creation, unique attribute | repeat-sensitive | repeat-sensitive when the constraint is off | | Cloud Sync | writes of two devices, set union | order-sensitive | arrival order or stamp rule | | Push Notifications | raising the badge locally, absolute badge in the payload, re-registering the token | repeat-sensitive | filtering by identifier is robust; delivery count and order not measured | The eighteen lessons other than the opening one, this lesson included, are distributed as follows: robust 3, repeat-sensitive 6, order-sensitive 7, doubly sensitive 2. In the lessons with mixed classes, the class written first in the Summary was counted. The core of the opening lesson is a separate count: six operations were distributed to the four classes as 1, 2, 2, and 1. The two distributions are not the same thing. The core counts six operations; the distribution above counts the sensitivities of the eighteen lessons other than the opening one. Three observations come out of the table. The same operation falls into a different class depending on the starting state, the compared state, and the rule; no class is a fixed property of an operation. A measure that solves repetition, idempotence or filtering by identifier, does not solve order; a measure that solves order, a stamp or a version, must solve repetition from somewhere else. And a measurement is always inside a model: the platform's real delivery, timing, and policy are recorded as not measured in the bounding measurement column. The next course, M30/K06, deals with architecture, testing, and publishing: how an iOS app is divided into layers, how it is tested, and how it is taken to the store.