JSON Encoding and Decoding

In the previous lesson we measured the three parts of a network client and the four sensitivities of six operations. What comes back from the network, however, is a byte sequence. The app turns this sequence into a typed record with encoding and decoding. The conversion itself looks safe: if the type does not fit, decoding raises an error, and if it fits, the record is ready. The trouble starts after decoding. The decoded record is written to a store, and if two responses carry two versions of the same record, which one is written last determines the value in the store.

The core is the same. As in every lesson, the first measured Swift block is Replay.swift.

// Replay.swift — classifies an operation's sensitivity to repetition and order
struct Trial<State: Equatable & CustomStringConvertible> {
    let label: String
    let once: State
    let twice: State
    let forward: State
    let backward: State

    var sensitivity: String {
        switch (once != twice, forward != backward) {
        case (false, false): return "robust"
        case (true, false): return "repeat-sensitive"
        case (false, true): return "order-sensitive"
        case (true, true): return "doubly-sensitive"
        }
    }

    func report() -> String {
        print("\(label) | once=\(once) twice=\(twice) ab=\(forward) ba=\(backward) -> \(sensitivity)")
        return sensitivity
    }
}

func replay<State>(_ label: String, from start: State, _ a: (State) -> State, then b: (State) -> State) -> Trial<State> {
    Trial(label: label, once: a(start), twice: a(a(start)), forward: b(a(start)), backward: a(b(start)))
}

func countSensitivities(_ results: [String]) {
    for name in ["robust", "repeat-sensitive", "order-sensitive", "doubly-sensitive"] {
        print("\(name): \(results.filter { $0 == name }.count)")
    }
}

Three operations will be measured in this lesson: appending the record to a list, writing the record to the store by its identifier, and encoding and decoding the record. Each works on a record type and a small shelf.

Record and Shelf

Record is both Codable and carries a short description so that it can be read in the measurement output. Shelf holds the state of the records; description writes the records sorted by identifier, because dictionary order is not known.

// Records.swift — record, shelf, and three write forms
import Foundation

struct Record: Codable, Equatable, CustomStringConvertible {
    let id: Int
    let name: String
    let version: Int
    var description: String { "\(id):\(name):v\(version)" }
}

struct Shelf: Equatable, CustomStringConvertible {
    var records: [Record] = []
    var description: String { records.sorted { $0.id < $1.id }.map(\.description).joined(separator: ",") }
}

func decodeRecord(_ text: String) -> Record {
    try! JSONDecoder().decode(Record.self, from: Data(text.utf8))
}

func append(_ record: Record) -> (Shelf) -> Shelf {
    { shelf in Shelf(records: shelf.records + [record]) }
}

func upsert(_ record: Record) -> (Shelf) -> Shelf {
    { shelf in
        var next = shelf.records
        if let index = next.firstIndex(where: { $0.id == record.id }) {
            next[index] = record
        } else {
            next.append(record)
        }
        return Shelf(records: next)
    }
}

func roundTrip(_ record: Record) -> Record {
    let encoder = JSONEncoder()
    encoder.outputFormatting = [.sortedKeys]
    let data = try! encoder.encode(record)
    return try! JSONDecoder().decode(Record.self, from: data)
}

func rename(_ name: String) -> (Record) -> Record {
    { record in Record(id: record.id, name: name, version: record.version) }
}

We test the three operations with the text of two responses. first and second are two versions carrying the same identifier.

// main.swift — two versions, three write forms
let first = decodeRecord(#"{"id":7,"name":"notebook","version":1}"#)
let second = decodeRecord(#"{"id":7,"name":"notebook-new","version":2}"#)

let listed = replay("append-to-list", from: Shelf(), append(first), then: append(second)).report()
let stored = replay("write-by-id", from: Shelf(), upsert(first), then: upsert(second)).report()
let travel = replay("round-trip", from: first, roundTrip, then: rename("notebook-new")).report()

countSensitivities([listed, stored, travel])

append-to-list | once=7:notebook:v1 twice=7:notebook:v1,7:notebook:v1 ab=7:notebook:v1,7:notebook-new:v2 ba=7:notebook-new:v2,7:notebook:v1 -> doubly-sensitive
write-by-id | once=7:notebook:v1 twice=7:notebook:v1 ab=7:notebook-new:v2 ba=7:notebook:v1 -> order-sensitive
round-trip | once=7:notebook:v1 twice=7:notebook:v1 ab=7:notebook-new:v1 ba=7:notebook-new:v1 -> robust
robust: 1
repeat-sensitive: 0
order-sensitive: 1
doubly-sensitive: 1

append-to-list is doubly sensitive. Appending the same response twice duplicates the record, and the order of the two versions changes the order of the list. write-by-id is robust against repetition: once and twice are the same, because a record with the same identifier is replaced by the new one, not appended. But it stays order-sensitive: ab and ba differ, because the last version written wins. round-trip is robust; when the record is encoded and decoded the same record comes back, and even if the order is changed by renaming, the result is the same.

NW4. The measurement assumes the identifier of the two versions is the same. Records with different identifiers are written to separate rows, and the order becomes invisible only in the sorted display.

NW5. rename changes only the name and does not touch the version number. An operation that also changes the version must be measured separately.

The Response from the Network

In a real app these two texts are the bodies of two separate responses. The client receives the response as a byte sequence and decodes it.

import Foundation

func loadRecord(session: URLSession, url: URL) async throws -> Record {
    let (data, _) = try await session.data(from: url)
    let decoder = JSONDecoder()
    decoder.keyDecodingStrategy = .convertFromSnakeCase
    return try decoder.decode(Record.self, from: data)
}

This snippet stays SOURCE; it reaches out to the network. The model above builds what the write operation does to the state; it is not the network itself. convertFromSnakeCase maps the server’s keys such as created_at to field names such as createdAt. Decoding can throw an error; for this reason the call is made with try, and the caller either propagates or catches the error.

The client cannot choose in which order the two responses will arrive. If two requests leave at the same time, the one sent second can come back first. So the order sensitivity in the write-by-id line is not a theory but a normal behavior of the network.

Why Encoding Is Safe

The round-trip line came out robust because encoding and decoding are written to be each other’s inverse. Record carries all three fields with fixed types; the encoder writes each field to a text under its name, and the decoder reads it back under the same names. There is no decision in between: which field goes where comes from the type, not from the order of operations or the number of times they run. So encoding and decoding the same record twice gives the same record, and whether the round trip or the renaming comes first does not affect the result.

This safety has two conditions. The first is that the encoded value itself must be lossless. If a Double field carries a very long decimal fraction, its last digits can change when it is written to text and read back; the integer and string fields of Record do not carry this danger. The second is that the names the decoder expects must be the same as the names the encoder writes. If the server writes created_at and the client expects createdAt, decoding throws an error on the first line. The measurement treats both as met; so its coming out robust does not say the conversion is flawless, it says the measured record is one for which the conversion is easy.

Where It Shows Up in a Real App

A notes app receives every record from the server as JSON. The user opens a note, the app decodes the note and updates the list on the screen. The same note arrives once more when the user refreshes the screen. The code that appends to the list takes this second response for a new note, and the same note appears as two rows on the screen. The append-to-list line in the measurement is exactly this: in the twice column the same record stands twice.

The user sees this as a visual flaw, and the developer mostly starts looking for something in the network layer. Yet the network layer has worked correctly; both responses are valid and were decoded correctly. The error is in the way the decoded record is written to the store. Decoding being robust and writing being sensitive is the dividing line of this lesson: the place to look for the problem is not the two sides of the conversion but the operation performed with the conversion’s output.

A second case is order. The user edits a note twice in quick succession, and each edit sends a separate save request. The server processes the two requests, and then the app reads the note again with two responses. The arrival order of the responses does not have to match the sending order. If the old version comes later, the old text stays on the screen and the user thinks the text they wrote has been lost. The difference between the ab and ba columns of the write-by-id line is exactly this.

The Wrong Setup and Its Fix

The common wrong setup is to write every response to the store as soon as it arrives.

import Foundation

// Wrong: every incoming response overwrites the record, without looking at its version
func receive(_ record: Record, into shelf: Shelf) -> Shelf {
    upsert(record)(shelf)
}

This snippet stays SOURCE. Writing by identifier solves repetition, but a late-arriving old response overwrites the new version. The fix is to accept only the larger version.

import Foundation

// Fix: only a newer version enters the store
func receiveNewer(_ record: Record, into shelf: Shelf) -> Shelf {
    if let current = shelf.records.first(where: { $0.id == record.id }), current.version >= record.version {
        return shelf
    }
    return upsert(record)(shelf)
}

This also stays SOURCE. For a record with the same identifier, the function compares the version in the store with the one in the response. If the store has an equal or larger version, the incoming response is ignored. This removes the problem of the two responses’ order: whichever order they arrive in, the one with the larger version stays in the store. Repetition is solved too, because the second arrival of the same version is ignored. This function is not in the measurement; the measured write-by-id is still order-sensitive, and the effect of the fix is described only in this explanation.

The fix has one condition: the server must provide the version number. A client’s own counter cannot be compared between two clients. If the server provides no version, the last modification time does a similar job, but clock skew makes this path fragile too.

Key and Date Strategies

The naming habit of the server and the client is often different. The server separates words with underscores, Swift code with capital letters. Giving the decoder a key strategy closes this gap in one place.

import Foundation

let decoder = JSONDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase
decoder.dateDecodingStrategy = .iso8601

This snippet stays SOURCE. The date strategy tells the decoder which format the date text sent by the server is in. If the format is chosen wrong, decoding throws a DecodingError and the whole response cannot be read. Setting the strategy up in a single place in the network layer, instead of rebuilding it on every call, prevents two screens from reading the same field differently.

Missing and extra fields are also a matter decided in the same place. If the response has a field that is not in the type, the decoder ignores it; this makes it possible for the server to add a new field without breaking the client. A required field that is in the type but missing from the response gives a keyNotFound error. If you know that the field can really be missing at times, you define it as optional in the type. Swallowing the error without doing this leaves the record silently half complete.

What the Measurement Shows and Does Not Show

The three classes of the three lines separate where the conversion is safe from where it is sensitive. What is safe is the transformation between text and record; what is sensitive is the way the record joins a collection. This distinction saves time when looking for a bug. In an app that gets a decoding error, the problem is in the shape of the response: a field is missing or its type is different. In an app that sees records multiplying or reverting to an old value, decoding has worked correctly and the problem is in the write rule.

There are also things the measurement does not show. If the record had a nested object or an array, decoding could fail in more places. If it had a date field, the choice of format would change the result. Large numbers, optional fields, and empty values in the response produce separate behaviors. The record measured in this lesson was deliberately kept small, so that the sensitivity question could be seen without getting mixed up with the details of the conversion. In a real app a decoding error is also an error class, and it falls within the classification described in the third lesson: retrying a response that cannot be decoded mostly gives the same result.

Facing a Decoding Error

If a decoding error is left silent, it becomes one of the most expensive errors. When the server changes the name or type of a field, the whole response becomes unreadable and the app shows an empty screen. The code that catches the error should log in which field and on which path the DecodingError occurred; saying only “could not be decoded” stretches finding the problem into hours. The user, on the other hand, is shown not the details but an understandable message.

Whether the error is retried is also a decision. Requesting a response that cannot be decoded from the same server again mostly gives the same result; the error is in the shape of the response, not in the network. For this reason a decoding error falls among the non-retryable errors in the third lesson’s classification. One exception is when the response has been cut off halfway; then the body is incomplete and requesting it again can fix the result. To tell the two apart, the status code and the length information of the response are read together.

Common Mistakes

• Decoding the response and appending it straight to the list. The same record multiplies when it arrives with two responses.
• Treating two responses as “the same” by comparing raw text. Two texts with a different key order can carry the same record.
• Swallowing a decoding error. When the server changes a field, the app silently works with the old value.
• Assuming the record has no version number and thinking the last response to arrive is the newest.

Interpreting the Measured Result

The three operations had three classes: doubly sensitive, order-sensitive, and robust. Encoding and decoding were robust; the problem was not in the conversion itself but in how the converted record was written to the store. Writing by identifier solved repetition; the order problem remained. To solve order, a version number must be carried in the record and only the larger version written. This rule was not measured; a similar storage rule will be measured in the fourth lesson.

The Bounding Measurement

The rule does not say whether it looks at the raw text or the decoded value; which one is compared determines the result. Below, two texts carrying the same record with a different key order are compared in two ways.

// main.swift — the same record, two texts
import Foundation

let textA = #"{"id":7,"name":"notebook","version":1}"#
let textB = #"{"version":1,"name":"notebook","id":7}"#
print("raw text equal: \(textA == textB)")
print("decoded record equal: \(decodeRecord(textA) == decodeRecord(textB))")

raw text equal: false
decoded record equal: true

The raw texts are different, the decoded records are the same. A client that compares two responses as raw text takes a repeated response for a new one; a client that compares the decoded value ignores it. The rule sees the inequality, it does not see what was compared.

NW6. This measurement shows the difference of key order. A difference in whitespace or number format was not measured.

Summary

• Encoding and decoding turn a typed record into a byte sequence and back; the round trip came out robust in the measurement.
• Appending the decoded record to a list is sensitive to both repetition and order.
• Writing by identifier solves repetition, not the arrival order of two versions.
• The arrival order of two responses is a behavior of the network; the client cannot choose it.
• Raw text equality and decoded value equality do not give the same decision.
• This lesson’s sensitivity: order-sensitive.

Next Step

A response may never arrive, or may be cut off halfway. In the next lesson, Errors and Retries will measure the classification of an error and why an order retried after a timeout can create a second record.