The Store Review Process

Before a build is sent to the store, teams mostly go through a readiness list: are the screenshots in place, does the privacy link work, have the permission descriptions been written. The list is drawn from problems experienced in past submissions and grows with every new problem. The store’s review decision is made outside this list; the list is the team’s own way of protecting itself.

The measure of this course is called separation: when the same check is applied to the correct version and to the faulty version, do the two answers differ. There are four categories: separating, where the correct version passes and the faulty version fails; blind, where both pass; noisy, where both fail; and inverted, where the correct version fails and the faulty version passes. In the output they appear as separating, blind, noisy, and inverted. This lesson measures with the core file.

// Separation.swift — classifies whether a check tells the correct version from the faulty one
struct Contrast<Output: CustomStringConvertible> {
    let label: String
    let correct: Output
    let faulty: Output
    let passesCorrect: Bool
    let passesFaulty: Bool

    var separation: String {
        switch (passesCorrect, passesFaulty) {
        case (true, false): return "separating"
        case (true, true): return "blind"
        case (false, false): return "noisy"
        case (false, true): return "inverted"
        }
    }

    func describe() -> String {
        let onCorrect = passesCorrect ? "pass" : "fail"
        let onFaulty = passesFaulty ? "pass" : "fail"
        print("\(label) | correct=\(correct)/\(onCorrect) faulty=\(faulty)/\(onFaulty) -> \(separation)")
        return separation
    }
}

func contrast<Output>(_ label: String, correct: Output, faulty: Output, check: (Output) -> Bool) -> Contrast<Output> {
    Contrast(label: label, correct: correct, faulty: faulty, passesCorrect: check(correct), passesFaulty: check(faulty))
}

func countSeparations(_ results: [String]) {
    for name in ["separating", "blind", "noisy", "inverted"] {
        print("\(name): \(results.filter { $0 == name }.count)")
    }
}

This lesson’s fault is in a field of the metadata: the description text to be shown to the user for the location permission is empty. The application may ask for the location, but the permission screen does not tell the user why it is being asked. The build passes, the signing passes, the tests pass; this field is not something any build step checks. In the store’s review, such a gap may cause the submission to be turned back; the lesson encodes this possibility only as an item of the readiness list.

The Readiness List Model

The model holds a submission with three fields: the location permission description, the screenshot sizes, and the presence of the privacy link. The readiness list consists of three items. purpose-filled checks that the description is not empty, and counts a text made only of spaces as empty. privacy-link-present checks that the link is defined. old-size-screenshot-present asks for a size named old to be present in the screenshot set; this item is left over from an old submission of the team.

readiness applies each item to the submission and writes item:pass or item:fail. Submission summarizes itself in one line, so the output shows which field carries what.

// Review.swift — the submission's fields and the team's readiness list
struct Submission: CustomStringConvertible {
    let locationPurpose: String
    let screenshotSizes: [String]
    let privacyLink: Bool

    var purposeState: String {
        if locationPurpose.isEmpty { return "empty" }
        if locationPurpose.allSatisfy({ $0.isWhitespace }) { return "blank" }
        return "filled"
    }

    var description: String {
        "purpose=\(purposeState) sizes=\(screenshotSizes.joined(separator: ",")) privacy=\(privacyLink ? "present" : "missing")"
    }
}

let readinessItems = ["privacy-link-present", "old-size-screenshot-present", "purpose-filled"]

func itemPasses(_ item: String, _ submission: Submission) -> Bool {
    switch item {
    case "purpose-filled": return submission.purposeState == "filled"
    case "purpose-not-empty": return !submission.locationPurpose.isEmpty
    case "privacy-link-present": return submission.privacyLink
    case "old-size-screenshot-present": return submission.screenshotSizes.contains("old")
    default: return false
    }
}

func readiness(_ submission: Submission, items: [String]) -> [String] {
    items.map { "\($0):\(itemPasses($0, submission) ? "pass" : "fail")" }
}

func listPasses(_ submission: Submission, items: [String]) -> Bool {
    readiness(submission, items: items).allSatisfy { $0.hasSuffix(":pass") }
}

let correctSubmission = Submission(locationPurpose: "Your location is used to show nearby stores", screenshotSizes: ["large", "medium"], privacyLink: true)
let emptySubmission = Submission(locationPurpose: "", screenshotSizes: ["large", "medium"], privacyLink: true)
let spaceSubmission = Submission(locationPurpose: "   ", screenshotSizes: ["large", "medium"], privacyLink: true)

The two submissions differ only in the description field. The screenshots are in the large and medium sizes; the old size was removed from the team’s visual layout and is absent from both submissions. The item, however, stayed on the list.

DS13. The readiness list in this measurement is the team’s own list. The items are not a rule published by the store, and the grounds on which the store turns a submission back were not measured.

Four Checks

Four checks are applied to the same two submissions: the description item, the old-size item, the privacy item, and the whole list.

// main.swift — three items and the whole list
let purpose = contrast("purpose-filled", correct: correctSubmission, faulty: emptySubmission) { itemPasses("purpose-filled", $0) }.describe()
let oldSize = contrast("old-size-screenshot-present", correct: correctSubmission, faulty: emptySubmission) { itemPasses("old-size-screenshot-present", $0) }.describe()
let privacy = contrast("privacy-link-present", correct: correctSubmission, faulty: emptySubmission) { itemPasses("privacy-link-present", $0) }.describe()
let wholeList = contrast("whole-list-passed", correct: correctSubmission, faulty: emptySubmission) { listPasses($0, items: readinessItems) }.describe()

countSeparations([purpose, oldSize, privacy, wholeList])

purpose-filled | correct=purpose=filled sizes=large,medium privacy=present/pass faulty=purpose=empty sizes=large,medium privacy=present/fail -> separating
old-size-screenshot-present | correct=purpose=filled sizes=large,medium privacy=present/fail faulty=purpose=empty sizes=large,medium privacy=present/fail -> noisy
privacy-link-present | correct=purpose=filled sizes=large,medium privacy=present/pass faulty=purpose=empty sizes=large,medium privacy=present/pass -> blind
whole-list-passed | correct=purpose=filled sizes=large,medium privacy=present/fail faulty=purpose=empty sizes=large,medium privacy=present/fail -> noisy
separating: 1
blind: 1
noisy: 2
inverted: 0

Four lines show three different behaviors of the list. The description item passes the correct submission and fails the submission with the empty description: separating. The privacy link item passes on both submissions; since the link exists in both versions, it does not see the fault and is therefore blind. Being blind is not a defect; the item was written for another problem and has nothing to say about this fault.

The real lesson is in the old-size item. The item fails on both submissions, because there is no old size in the screenshot set. Even the correct submission cannot pass the list. The faulty submission fails too, but the reason for failing has nothing to do with the fault. Since the two answers are the same, the item counts as noisy: it looks red and carries no separation.

The Cost of Noise

This cost shows in the whole-list line. whole-list-passed fails even on the correct submission, and the only reason is the old-size item. The person reading the list meets two situations. In the first, they see the red, look at the cause, understand that they got stuck on an old item, and move on. In the second, they no longer look at all, because they already know about the red. Teams most often fall into the second path, because a red that separates nothing becomes invisible after a while.

This habit of ignoring continues when a real fault turns red on the list. The description item may catch the fault correctly; but since the list is red anyway, the caught fault is lost in the crowd. An item being separating on its own does not mean the list works. What carries the separation is the behavior of the whole list, and with the noisy item it is noisy.

DS14. This measurement does not know why the item stayed outdated. That the screenshot layout changed and the old size is no longer produced is the lesson’s assumption. In a real team’s list there may also be a case where such an item is still needed.

How an Old Item Is Cleaned Up

Two things can be done with a noisy item: delete it or update it. Deleting is right if the concern the item covered no longer exists. If the screenshot layout changed and the old size is not asked for in any submission, the item does not belong on the list. Updating is done when the concern continues but its wording has aged: the item counts the sizes that are asked for today.

The question to ask when deciding is the measurement’s question: when I put a faulty submission and a correct submission together, does this item separate the two answers? If it does not and gives the same answer on every submission, then either the answer is always pass and the item is blind, or the answer is always fail and the item is noisy. Each may have its place. A blind item may stand like a guard watching for a fault experienced only once; a noisy item does not even do that job.

A measurement is also taken before deleting. Taking the item off the list and checking the same two submissions again shows that the whole list now passes the correct submission and fails the faulty one. This is the proof that the item really consisted of noise. Without the proof, deletion is a dry guess.

A Red Item and the Decision to Submit

Between the list coming out red and the submission being stopped there is a decision, and the list does not make that decision. The team reads the red item and stops the submission, or decides the item is old and submits. The first path is safe but slow; the second is fast but breaks the list’s own word. A noisy item recreates exactly this dilemma with every submission.

A separating item has no such dilemma. Red means something in the submission is wrong, and green means that thing is in place. The decision becomes easier accordingly. This is the practical return of measuring the list: looking at the red and fixing it instead of looking at the red and arguing.

There is also the question of who reads. The readiness list is mostly read by the person who makes the submission; they do not remember when the items were written. Someone who does not know why an item is on the list takes the old-size item seriously and may start preparing a visual for nothing. A noisy item thus costs not only time but also effort.

Empty Versus Only Spaces

The description item can be written in two forms. purpose-filled also checks that the text is not made only of spaces. purpose-not-empty checks only that the text is not of zero length. The two seem written for the same purpose, but they behave differently against a description made of three spaces. A description that shows three spaces on the user’s screen is no different from an empty description; but the second item cannot see this.

The next measurement applies the same two checks to the submission with three spaces. It also puts the old-size item next to them, so that it can be seen that the noisy item stays the same when the input changes.

The Bounding Measurement

The same items are applied to the submission that carries a description made only of spaces instead of an empty description. On the screen the two show the same blank.

// main.swift — the same items, empty and blank descriptions
let emptyByLength = contrast("not-empty-empty-description", correct: correctSubmission, faulty: emptySubmission) { itemPasses("purpose-not-empty", $0) }.describe()
let blankByLength = contrast("not-empty-blank-description", correct: correctSubmission, faulty: spaceSubmission) { itemPasses("purpose-not-empty", $0) }.describe()
let blankByContent = contrast("filled-blank-description", correct: correctSubmission, faulty: spaceSubmission) { itemPasses("purpose-filled", $0) }.describe()
let oldSizeOnBlank = contrast("old-size-blank-description", correct: correctSubmission, faulty: spaceSubmission) { itemPasses("old-size-screenshot-present", $0) }.describe()

countSeparations([emptyByLength, blankByLength, blankByContent, oldSizeOnBlank])

not-empty-empty-description | correct=purpose=filled sizes=large,medium privacy=present/pass faulty=purpose=empty sizes=large,medium privacy=present/fail -> separating
not-empty-blank-description | correct=purpose=filled sizes=large,medium privacy=present/pass faulty=purpose=blank sizes=large,medium privacy=present/pass -> blind
filled-blank-description | correct=purpose=filled sizes=large,medium privacy=present/pass faulty=purpose=blank sizes=large,medium privacy=present/fail -> separating
old-size-blank-description | correct=purpose=filled sizes=large,medium privacy=present/fail faulty=purpose=blank sizes=large,medium privacy=present/fail -> noisy
separating: 2
blind: 1
noisy: 1
inverted: 0

The item that looks at length separates on the empty description and is blind on the description of spaces. Two appearances of the same fault make the same item give different answers. The item that looks at content separates on both. The old-size item stays red whatever the input is, and in this measurement too it is noisy.

The limits are clear. This lesson did not measure the store’s review decision or its duration; and that is not what should be measured, because that decision is an output rather than an input in the team’s hands. The lesson also did not measure on which grounds the store turns submissions back. The reasons for rejection are countless: permission descriptions, privacy texts, the content of screenshots, the application behaving differently from what it declares, and many more. Out of this countlessness, what enters a list is what the team has experienced in the past.

DS15. Every item in the readiness list is the team’s list, not the store’s rule. Passing the list does not mean the review will pass; failing the list does not say the submission will be turned back. The lesson measures only this: does an item on the list separate a fault the team knows from a submission without that fault.

Keeping the List Honest

A readiness list decays in two directions. The first is an item staying on the list after the source of the fault is gone; the old-size item is this. The second is a new fault passing without entering the list. When a submission is turned back, the work to do is to turn the reason for the rejection into an item and to test that item with this lesson’s measurement: apply the item to the faulty submission, apply it to the correct submission, and see that the two answers differ.

This test is done before the item is added, because every added item brings a maintenance load. An item that is not separating brings load but no separation. A blind item will not catch the next fault; a noisy item accustoms the reader of the list to red. An inverted item is the worst: it fails the correct submission and passes the faulty one, and pushes the team in the wrong direction.

Maintenance of the list works if it is tied to one person or one step. After every submission the list is reviewed with a single question: which item stayed red and only red in this submission? That item is either updated or deleted. This habit turns the list into a tool that watches for today’s faults and not the past’s.

Summary

• The readiness list before the store’s review comes from the team’s own history of faults; it is not a rule published by the store.
• The item checking whether the permission description is filled is separating, because it tells the submission with an empty description from the correct submission.
• An outdated item stays red on both versions and makes the whole list noisy.
• The two appearances of the same fault, empty and made only of spaces, give a different answer to an item that looks only at length; an item that looks at content separates both.
• Before adding an item to the list or removing it, it is measured on a faulty and a correct submission.
• This lesson’s separation: noisy.

Next Step

The readiness list checks the content of the submission; the page the user sees in the store is a separate job. In the next lesson, The Store Listing, we will measure whether the screenshots and the subtitle separate the fault when compared with a recorded reference, and a case where an equality check can work backwards.