---
title: 'Staged Rollout and Rollback'
source: 'https://academia.sh/en/courses/cross-platform-production/staged-rollout-and-rollback'
course: 'Cross-Platform Production Practice'
language: en
updated: '2026-10-21T05:02:51+00:00'
license: 'CC BY-SA 4.0'
objectives:
    - 'Defines a staged rollout, a bucket, and halting a rollout in a mobile release flow'
    - 'Compares rollback with rolling forward and explains why an installed version cannot be withdrawn'
    - "Builds the condition of each rollout bucket and measures the feature's operability per bucket"
    - 'Shows by measurement that the number of devices a rollout has reached does not determine operability'
---

# Staged Rollout and Rollback

A staged rollout opens a new version to only a portion of the users; if a problem appears, the rollout is halted or the feature is turned off. This lesson measures how well a feature works in each bucket the rollout has reached.

In the previous lesson we measured that if a client asking for the new contract is published before the server is ready, some of the users see the feature degraded. This lesson describes how to choose that portion deliberately and under control. A new version is not opened to all users at once; it is opened first to a small group, and if no problem appears, the group grows.

The question of the course stays the same: how well does this feature work under this condition? In this lesson the condition is which bucket of the rollout the device is in.

## Staged Rollout

In a staged rollout, a version is published in the store but offered to only a part of the users. The store selects the part itself. On Android, the team decides which stage the rollout moves to; on iOS, the stages advance by themselves on a fixed schedule, and the team can only pause the rollout or open it to everyone. As the stage widens, the number of users who can receive the version grows; in the last stage everyone receives it.

The value of this arrangement is that it limits the effect of a defect. If the new version carries an unexpected defect, the defect first shows up in a small group. Crash reports and user feedback are watched; if things are going well, the rollout moves to the next stage. Monitoring is the subject of a separate lesson and is covered at the end of this course.

A staged rollout is not a single mechanism. The store's own staged rollout feature distributes the version to devices. A flag inside the application, on the other hand, turns the feature on and off on the devices that received the version. The two can be used together: the version reaches a portion of devices at the store level, and the feature is opened by a flag on only a portion of those.

M12/K02 described how a change reaches production. In a staged rollout, the new version reaches only some buckets; operability classifies not the number of devices reached but the operability in each bucket. How many devices the version reached is a question for the rollout plan. How well the feature works in the bucket it reached is the question for operability.

## Bucket and Halting a Rollout

A bucket is a group into which devices are split by a stable rule. The rule derives a number from the device identifier and divides that number by the bucket count; the remainder is the device's bucket. The same device always falls into the same bucket. This stability matters: when the rollout widens, a device in the first bucket does not leave its bucket; new buckets are added to the list of open buckets.

A configuration file can hold the stages of a rollout. The fragment below shows only the shape; it is not measured.

```yaml
# Platform code: staged rollout stages
rollout:
  - stage: first
    buckets: [2]
  - stage: wider
    buckets: [2, 3]
  - stage: full
    buckets: [0, 1, 2, 3]
```

To halt a rollout means not to move to the next stage. In a halted rollout, the devices that already received the version keep using it. On Android, new devices do not receive the version; on iOS, halting stops only the automatic update, and a user who downloads the version from the store on their own can still get it. Halting largely freezes the number of newly affected users, but it does not heal those already affected.

## Rollback and Rolling Forward

When halting is not enough, two paths remain.

Rollback returns the user to the previous behavior. On mobile, an installed version cannot be withdrawn. The store does not downgrade the application on the user's device; at most it offers the old version again, and that comes as an update with a new version number. For this reason, rollback on mobile mostly means rolling back the feature, not the version.

The tool for rolling back a feature is the remote kill flag: the feature ships behind a flag, and if a problem appears the flag is pulled and the application returns to the old behavior. If the flag is read remotely, no new version is needed for this; it is the only way to cut the effect without waiting for store review. The two fragments below show how the flag is read.

```kotlin
// Platform code: reading a remote kill flag
val enabled = config.getBoolean("new_feed_enabled")
if (enabled) showNewFeed() else showLegacyFeed()
```

```swift
// Platform code: reading a remote kill flag
let enabled = config.bool(forKey: "new_feed_enabled")
if enabled { showNewFeed() } else { showLegacyFeed() }
```

Rolling forward means publishing a new version that fixes the defect. It is not as fast as rollback, because the new version goes through store review and then waits for the user to install it. But for a feature that was not placed behind a flag, it is the only path.

The choice between the two depends on whether the feature is behind a flag. For a feature behind a flag, rollback comes first, followed by an unhurried roll forward. For a feature without a flag there is only rolling forward, and the affected users live with the problem until they install the fixed version.

## Measuring the Condition of the Rollout

For the measurement, take the new news feed. `NewFeed` needs the network and the `flag:new-feed` capability; the flag has no fallback. `LegacyFeed` is the old feed and needs only the network. The rollout supplies the flag in open buckets and does not supply it in closed buckets. The core is familiar from the first lesson and is given again in the same form in every lesson; the lesson does not touch it.

```javascript
// operability.mjs — classifies how fully a feature works under a condition
export const LEVELS = ["full", "degraded", "deferred", "disabled"];
const FALLBACKS = ["degraded", "deferred"];

export function feature(name, needs, fallback = {}) {
  return { name, needs, fallback };
}

export function condition(name, has) {
  return { name, has };
}

function judge(item, cond) {
  const missing = item.needs.filter((need) => !cond.has.includes(need)).sort();
  let level = "full";
  for (const need of missing) {
    const given = FALLBACKS.includes(item.fallback[need]) ? item.fallback[need] : "disabled";
    if (LEVELS.indexOf(given) > LEVELS.indexOf(level)) level = given;
  }
  return { level, feature: item.name, condition: cond.name, missing };
}

export function assess(features, conditions) {
  const records = [];
  for (const cond of conditions) {
    for (const item of features) records.push(judge(item, cond));
  }
  return records;
}

export function printLevels(records) {
  for (const r of records) {
    const missing = r.missing.length === 0 ? "-" : r.missing.join(",");
    console.log(`${r.level}: ${r.feature} @ ${r.condition} missing=${missing}`);
  }
}

export function countLevels(records) {
  for (const name of LEVELS) {
    console.log(`${name}: ${records.filter((r) => r.level === name).length}`);
  }
  console.log(`affected: ${records.filter((r) => r.level !== "full").length}`);
}
```

After the core comes the lesson's helper file. `bucketOf(deviceId, bucketCount)` adds up the character codes of the device identifier and divides the sum by the bucket count; the remainder is the bucket. `rolloutConditions(fleet, openBuckets, bucketCount)` produces a condition from each device: the network is present on every device, and the flag capability is added only for devices whose bucket is open.

```javascript
// rollout.mjs — splits a device into its bucket, builds conditions from open buckets
import { feature, condition } from "./operability.mjs";

export function bucketOf(deviceId, bucketCount) {
  let sum = 0;
  for (const ch of deviceId) sum += ch.charCodeAt(0);
  return sum % bucketCount;
}

export function rolloutConditions(fleet, openBuckets, bucketCount) {
  return fleet.map((deviceId) => {
    const open = openBuckets.includes(bucketOf(deviceId, bucketCount));
    return condition(deviceId, open ? ["network", "flag:new-feed"] : ["network"]);
  });
}

export const rolloutFeatures = [
  feature("NewFeed", ["network", "flag:new-feed"]),
  feature("LegacyFeed", ["network"]),
];
```

**RO5.** The model builds the rollout with a fixed list of devices and four buckets. In a real rollout the number of devices changes constantly and the bucket count is much larger; moreover, the store's staged rollout and the application's flag are two separate mechanisms, and each may use its own bucketing rule. The model shows only one rule, and it builds this operability's rule; the model is not the framework itself.

The first measurement shows the first stage of the rollout: only bucket `2` is open. The buckets of the four devices are printed first, then the eight records follow.

```javascript
// main.mjs
import { assess, printLevels } from "./operability.mjs";
import { bucketOf, rolloutConditions, rolloutFeatures } from "./rollout.mjs";

const fleet = ["device-a", "device-b", "device-c", "device-d"];
for (const deviceId of fleet) {
  console.log(`bucket: ${deviceId} ${bucketOf(deviceId, 4)}`);
}
printLevels(assess(rolloutFeatures, rolloutConditions(fleet, [2], 4)));
```

```text
bucket: device-a 2
bucket: device-b 3
bucket: device-c 0
bucket: device-d 1
full: NewFeed @ device-a missing=-
full: LegacyFeed @ device-a missing=-
disabled: NewFeed @ device-b missing=flag:new-feed
full: LegacyFeed @ device-b missing=-
disabled: NewFeed @ device-c missing=flag:new-feed
full: LegacyFeed @ device-c missing=-
disabled: NewFeed @ device-d missing=flag:new-feed
full: LegacyFeed @ device-d missing=-
```

The four devices fall into four separate buckets. Only `device-a` is in the open bucket, and there `NewFeed` is full. On the other three devices the flag is absent; `NewFeed` is disabled, because the flag has no defined fallback. `LegacyFeed` is full on all four devices. The user of a disabled device is not left without a feed; they see the old feed. The new feed being disabled and the old feed being full are two separate records.

## Widening and Rollback

When the rollout widens, a new bucket is added to the list of open buckets. When it is rolled back, the list empties. The rule works the same way in both cases; the only thing that changes is the capability in the condition.

The second measurement puts two stages one after the other. First buckets `2` and `3` are opened; then the flag is pulled and no open bucket remains. Only the `NewFeed` records are printed.

```javascript
// main.mjs
import { assess, printLevels } from "./operability.mjs";
import { rolloutConditions, rolloutFeatures } from "./rollout.mjs";

const fleet = ["device-a", "device-b", "device-c", "device-d"];
const stages = [
  { name: "widening", open: [2, 3] },
  { name: "rollback", open: [] },
];
for (const stage of stages) {
  console.log(`stage: ${stage.name} open-buckets=${stage.open.length === 0 ? "-" : stage.open.join(",")}`);
  const records = assess(rolloutFeatures, rolloutConditions(fleet, stage.open, 4));
  printLevels(records.filter((r) => r.feature === "NewFeed"));
}
```

```text
stage: widening open-buckets=2,3
full: NewFeed @ device-a missing=-
full: NewFeed @ device-b missing=-
disabled: NewFeed @ device-c missing=flag:new-feed
disabled: NewFeed @ device-d missing=flag:new-feed
stage: rollback open-buckets=-
disabled: NewFeed @ device-a missing=flag:new-feed
disabled: NewFeed @ device-b missing=flag:new-feed
disabled: NewFeed @ device-c missing=flag:new-feed
disabled: NewFeed @ device-d missing=flag:new-feed
```

In the widening stage, `device-a` and `device-b` are in open buckets and `NewFeed` is full there; `device-c` and `device-d` are still disabled. After the rollback, all four of the four devices return to disabled. At the level of a record, pulling the flag says this: the capability the new feed needs no longer exists on any device.

Rollback is not an operability level. Rollback is work the team does; operability classifies what the condition is at the end of that work. The rule does not see who pulled the flag, when they pulled it, or why it was pulled; it sees only that the `flag:new-feed` capability is absent after it was pulled.

**RO6.** The model assumes that pulling the flag takes effect on every device instantly. In reality devices read the configuration at different times, and a device that has been offline can keep using the old value for a while longer. Rollback therefore happens not at a single moment but as a distribution. The rule does not show this distribution; it requires a separate condition to be built from the value each device read.

## The Bounding Measurement

The focus record of this lesson is `NewFeed @ device-a`, the first device in the open bucket in the first stage of the rollout. The bounding measurement shows two things together. First, the focus record is full in the open bucket, and three of the eight records are affected. Second, the number of devices the rollout reached does not determine the focus record: when the rollout widens by one more bucket, the record of `device-a` does not change.

```javascript
// main.mjs
import { assess, printLevels, countLevels } from "./operability.mjs";
import { rolloutConditions, rolloutFeatures } from "./rollout.mjs";

const fleet = ["device-a", "device-b", "device-c", "device-d"];
const records = assess(rolloutFeatures, rolloutConditions(fleet, [2], 4));
printLevels(records.filter((r) => r.feature === "NewFeed" && r.condition === "device-a"));
countLevels(records);
const reached = records.filter((r) => r.feature === "NewFeed" && r.level === "full").length;
console.log(`reached devices: ${reached} total: ${fleet.length}`);

const wider = assess(rolloutFeatures, rolloutConditions(fleet, [2, 3], 4));
const pick = (list) => list.find((r) => r.feature === "NewFeed" && r.condition === "device-a");
const same = JSON.stringify(pick(records)) === JSON.stringify(pick(wider));
console.log(`focus record unchanged when the share changes: ${same ? "yes" : "no"}`);
```

```text
full: NewFeed @ device-a missing=-
full: 5
degraded: 0
deferred: 0
disabled: 3
affected: 3
reached devices: 1 total: 4
focus record unchanged when the share changes: yes
```

The first line carries the lesson's operability: on the device in the open bucket, the new feed is full. The count lines say that of eight records, five are full and three are disabled; affected is three. The number of devices reached is one in four.

The last two lines show the limit of the model. The number of devices reached is a number for the rollout plan; it does not say how well the feature works on each device. That the new feed is full on a device that was reached does not mean that something is broken on the devices that were not reached; there the new feed was never offered, and the old feed works fully. Even when the rollout widens and the number reached rises to two, the record of `device-a` stays the same. The share changes which devices fall under which record, not a record itself.

This distinction is used in rollout decisions. The question to ask before widening is not only whether the number of devices reached is large enough, but also whether the feature works fully in the buckets reached. Both are examined separately.

## Summary

- This lesson's operability: full. `NewFeed @ device-a` supplies the flag capability because the device's bucket is open, and the new feed works fully; on the three devices in closed buckets the same feature stayed disabled.
- A staged rollout opens the version to buckets; halting stops the rollout from widening (on iOS, only the automatic update) but does not withdraw an installed version.
- Rollback on mobile mostly means rolling back the feature: a remote kill flag cuts the effect without waiting for store review, and for a feature without a flag only rolling forward remains.
- Of eight records, five were full and three were disabled; affected was three. When the flag was pulled, the new feed returned to disabled on all four devices.
- When the rollout widened by one bucket, the focus record did not change; operability classifies the condition in each bucket, not the number of devices reached.

## Next Step

In this lesson the new version did not reach some users, and the old version lived without trouble. This is not always the case: some versions can no longer be supported. The next lesson, `Forced Update Strategy`, covers the minimum version, soft and forced update prompts, and the end of support for an old version. It then measures how well the payment feature works on a client below the minimum version the server requires.