Skip to content
academia.sh

Lesson 09 / 13

Availability in Numbers

Converting a service availability percentage into an outage budget: converting percentages into per-year, per-month, per-week, and per-day duration, splitting the budget between planned work and failure, raising the target making planned work impossible, and computing the effect of multiplication in a series chain and of the complement in replication on composite service availability.

Contents

The previous three lessons measured under the assumption that the system works: load, capacity, speedup, and cost. When a unit goes down, different questions apply. How much of the system goes down, for how long, and is that duration acceptable? The third question is answered with a percentage, and a percentage alone says nothing.

This lesson converts the percentage into a duration, treats that duration as a spendable budget, and computes what combining components does to the budget. Every number belongs to the computed-value class; its inputs are explicitly named assumptions.

Two Terms and Two Definitions

Service availability is the ratio of the time the system can respond to requests to the total time. The term should not be confused with interface accessibility, which is an interface’s usability with assistive technologies and the subject of the Accessible Component Patterns course. In this lesson, the term always means service availability, never accessibility.

The ratio has two bases, and the same outage looks different under each. The time-based measurement divides uptime by total time; outage minutes are counted. The request-based measurement divides successful requests by total requests; the number of affected users is counted. The two diverge because traffic is not evenly distributed: since the Back-of-the-Envelope Estimation lesson’s peak factor is 3, five minutes of downtime falling in the peak hour affects three times as many requests as five minutes falling at average load. This lesson builds the budget on the time base, because a budget is a quantity set aside in advance, allocated without knowing the traffic.

Converting a Percentage into a Duration

The percentage format misleads the comparison: the 0.09-point gap between 99.9% and 99.99% looks small. The same gap stops being invisible once it is converted into minutes.

// service/budget.mjs — converting a service availability percentage into a per-period outage budget
const PERIOD = [["year", 365 * 24 * 60], ["month", 30 * 24 * 60], ["week", 7 * 24 * 60], ["day", 24 * 60]];
const PERCENT = [99, 99.5, 99.9, 99.95, 99.99, 99.999];

const format = (min) => (min >= 60 ? `${(min / 60).toFixed(1)} hr` : min >= 1 ? `${min.toFixed(1)} min` : `${(min * 60).toFixed(1)} sec`);

console.log(`${"percent".padEnd(9)}${PERIOD.map(([a]) => a.padStart(12)).join("")}`);
for (const p of PERCENT) {
  const fraction = 1 - p / 100;
  console.log(`${`${p}%`.padEnd(9)}${PERIOD.map(([, min]) => format(fraction * min).padStart(12)).join("")}`);
}

const first = 1 - 99.9 / 100;
const second = 1 - 99.99 / 100;
console.log(`\n99.9% -> 99.99%: monthly budget ${format(first * 43200)} instead of ${format(second * 43200)}, ` +
  `${(first / second).toFixed(0)}x narrower`);
console.log(`adding one nine divides the outage share by 10 each time; a 0.09-point gap ` +
  `means ${format((first - second) * 43200)} monthly`);
percent          year       month        week         day
99%           87.6 hr      7.2 hr      1.7 hr    14.4 min
99.5%         43.8 hr      3.6 hr    50.4 min     7.2 min
99.9%          8.8 hr    43.2 min    10.1 min     1.4 min
99.95%         4.4 hr    21.6 min     5.0 min    43.2 sec
99.99%       52.6 min     4.3 min     1.0 min     8.6 sec
99.999%       5.3 min    25.9 sec     6.0 sec     0.9 sec

99.9% -> 99.99%: monthly budget 43.2 min instead of 4.3 min, 10x narrower
adding one nine divides the outage share by 10 each time; a 0.09-point gap means 38.9 min monthly

A month is taken as thirty days; since months are not equal in length, the budget’s period must be defined up front, or the same percentage gives a different minute count in February.

The table’s first result is a difference of scale. In spoken discussion, “ninety-nine percent available” sounds strong; its counterpart is 87.6 hours a year, an outage exceeding three and a half days. Ninety-nine point nine nine nine percent is 5.3 minutes in the same year — a single restart spends the entire budget.

The second result is a measure of the period choice. 99.9% gives 43.2 minutes in a month, 1.4 minutes in a day. The two numbers describe the same target but permit different behavior: the monthly budget accepts a single 43-minute outage, the daily budget does not. A target’s period is as decisive as the target itself.

The Budget Gets Spent

The real use of an outage budget is turning an outage from something that just happens into a resource that gets spent. Part of the budget is spent deliberately: processes shut down and restart at every release, the service stops during planned maintenance. What remains is the share set aside for failure.

The computation below takes four assumptions. Target 99.9%; its rationale is that the tracking query carries a commercial commitment but is not a payment flow. 20 deployments a month and 15 seconds of unresponsiveness per deployment; the rationale is the old process shutting down and the new one starting at every release. 10 minutes of planned maintenance a month. And a recovery time of 10 minutes; the same order of magnitude as the recovery time target in the Relational Database Administration course.

// service/spend.mjs — splitting the monthly outage budget into planned work and failure
const MONTH = 30 * 24 * 60;        // minutes
const TARGET = 99.9;               // assumption: service availability target
const DEPLOYS = 20;                // assumption: monthly deployment count
const DEPLOY_SEC = 15;             // assumption: unresponsiveness per deployment (seconds)
const MAINT_MIN = 10;              // assumption: monthly planned maintenance (minutes)
const RECOVERY_MIN = 10;           // assumption: recovery time of a failure (minutes)

function budget(target, deploys) {
  const total = (1 - target / 100) * MONTH;
  const planned = (deploys * DEPLOY_SEC) / 60 + MAINT_MIN;
  return { total, planned, failure: total - planned, count: (total - planned) / RECOVERY_MIN };
}

console.log("target      deploys   monthly budget   planned   left for failure   failures absorbed");
for (const [target, deploys] of [[TARGET, DEPLOYS], [TARGET, DEPLOYS * 2], [99.99, DEPLOYS]]) {
  const b = budget(target, deploys);
  console.log(`${`${target}%`.padEnd(9)}${String(deploys).padStart(7)}` +
    `${b.total.toFixed(1).padStart(14)} min${b.planned.toFixed(1).padStart(9)} min` +
    `${b.failure.toFixed(1).padStart(16)} min${b.count.toFixed(2).padStart(19)}`);
}

const b = budget(TARGET, DEPLOYS);
console.log(`\nplanned work's share of the budget = ${((b.planned / b.total) * 100).toFixed(1)}%`);
console.log(`if the target were 99.99%, planned work would be ${((budget(99.99, DEPLOYS).planned / budget(99.99, DEPLOYS).total) * 100).toFixed(0)}% of the budget`);
target      deploys   monthly budget   planned   left for failure   failures absorbed
99.9%         20          43.2 min     15.0 min            28.2 min               2.82
99.9%         40          43.2 min     20.0 min            23.2 min               2.32
99.99%        20           4.3 min     15.0 min           -10.7 min              -1.07

planned work's share of the budget = 34.7%
if the target were 99.99%, planned work would be 347% of the budget

The first line shows how the budget splits: 15.0 of the 43.2 minutes goes to planned work, that is, 34.7% of the budget is already spent before a single failure occurs. The 28.2 minutes left for failure, at a ten-minute recovery time, means 2.82 failures a month: two failures fit the budget, a third breaks the target.

The second line is the assumption’s sensitivity: when the deployment count doubles, the failure share drops from 28.2 to 23.2 minutes, the absorbable failures from 2.82 to 2.32. Releasing more often means less headroom for failure — both are paid from the same budget.

The third line is the lesson’s harshest result. When the target is raised to 99.99%, the monthly budget shrinks to 4.3 minutes, and planned work’s 15.0 minutes takes up 347% of the budget; the failure share is negative 10.7 minutes. The target cannot be met even with zero failures. Raising a service availability target requires changing the deployment pattern as well; otherwise the target remains a statement of intent. What makes an outage budget a metric is its ability to flag a target as impossible.

Multiplication in a Composite System

Answering a request requires multiple components to work together. The tracking query arrives at the edge process, checks the cache, goes to the shipment store if needed, and is fed by the state event stream. If the request cannot be answered when any one component goes down, the components are connected in series, and composite service availability is a product.

// service/composite.mjs — multiplication in a series chain, complement in replication: the tracking path's composite service availability
const MONTH = 30 * 24 * 60;        // minutes
const CHAIN = ["edge process", "cache", "shipment store", "event stream"];
const COMPONENT = 0.999;           // assumption: service availability per component

const minutes = (a) => (1 - a) * MONTH;
const series = (a, n) => a ** n;
const replicated = (a, k) => 1 - (1 - a) ** k;

console.log(`chain = ${CHAIN.length} components, ${(COMPONENT * 100).toFixed(1)}% per component`);
console.log("components       composite service availability   monthly outage");
for (let n = 1; n <= CHAIN.length; n += 1) {
  const a = series(COMPONENT, n);
  console.log(`${String(n).padStart(14)}   ${(a * 100).toFixed(4).padStart(21)}%   ${minutes(a).toFixed(1).padStart(9)} min`);
}

console.log("\nreplicas       component outage share   chain outage share   monthly outage");
for (const k of [1, 2, 3]) {
  const b = replicated(COMPONENT, k);
  const z = series(b, CHAIN.length);
  console.log(`${String(k).padStart(12)}   ${(1 - b).toExponential(2).padStart(20)}   ` +
    `${(1 - z).toExponential(2).padStart(19)}   ${(minutes(z) * 60).toFixed(3).padStart(9)} sec`);
}

const better = 0.9999;
console.log(`\nsensitivity: if the component were ${(better * 100).toFixed(2)}%, the unreplicated chain would be ` +
  `${(series(better, CHAIN.length) * 100).toFixed(4)}%, ${minutes(series(better, CHAIN.length)).toFixed(1)} min monthly`);
console.log(`weakest link: the chain cannot exceed any single component (upper bound ${(COMPONENT * 100).toFixed(1)}%)`);
chain = 4 components, 99.9% per component
components       composite service availability   monthly outage
             1                 99.9000%        43.2 min
             2                 99.8001%        86.4 min
             3                 99.7003%       129.5 min
             4                 99.6006%       172.5 min

replicas       component outage share   chain outage share   monthly outage
           1                1.00e-3               3.99e-3   10352.458 sec
           2                1.00e-6               4.00e-6      10.368 sec
           3                1.00e-9               4.00e-9       0.010 sec

sensitivity: if the component were 99.99%, the unreplicated chain would be 99.9600%, 17.3 min monthly
weakest link: the chain cannot exceed any single component (upper bound 99.9%)

The first table gives the direction of the multiplication: even though each component is 99.9%, the four-component chain is 99.6006%, and the monthly outage rises from 43.2 to 172.5 minutes, nearly fourfold. Outage shares add up approximately. The design consequence is direct: adding a component lowers service availability, because in a series chain every new component is a new single point. The Architectural Styles course counted a microservice layout’s bill as a state count that grows with the unit count; its counterpart here is multiplication.

The same table also gives an upper bound. A series chain cannot exceed its weakest component: without replication, a path containing a 99.9% component cannot be 99.95%. If a target is set without being broken down into component targets, whether it holds cannot be tested by looking at the components.

The second table shows the one tool that works in the opposite direction: placing multiple copies of the same component. Copies are connected in parallel — the component stays up unless all of them go down together — and the outage share is raised to a power by the copy count. A component’s outage share falls from one in a thousand to one in a million with a second copy, one in a billion with a third; the four-component chain’s monthly outage drops from 10,352 seconds to 10.4 seconds, then to 0.010 seconds.

This second table carries an assumption that does not fully hold in reality: that copies fail independently. Two copies running the same faulty release go down together on the same request; the same configuration error breaks both at once. To the extent independence does not hold, real service availability falls below the computed value, so these numbers are an upper bound. Adding a copy is also not as cheap as it looks in the arithmetic; keeping copies unaware of each other’s failures incurs another cost, and that cost is the subject of the rest of this topic.

The last line gives the sensitivity: if the component’s service availability were 99.99%, the unreplicated chain would be 99.9600% and the monthly outage 17.3 minutes. The same outcome can be reached two ways — improving the component or replicating it — and the cost of each is compared using the previous lesson’s cost curves.

Summary

  • Service availability is the ratio of the time the system can respond and is a concept separate from interface accessibility; time-based and request-based measurement show the same outage differently.
  • 99.9% is 8.8 hours a year, 43.2 minutes a month, 1.4 minutes a day of outage; adding one nine divides the share by 10, and a 0.09-point gap means 38.9 minutes a month.
  • A target’s period is as decisive as the target: a monthly 43.2 minutes permits a single long outage, a daily 1.4 minutes does not.
  • 34.7% of the budget goes to planned work (20 deployments and 10 minutes of maintenance); the 28.2 minutes left for failure, at a ten-minute recovery time, is 2.82 failures a month.
  • When the target is raised to 99.99%, planned work takes up 347% of the budget and the target cannot be met even with zero failures; raising the target requires changing the deployment pattern.
  • In a series chain, service availabilities multiply: four 99.9% components give 99.6006% and 172.5 minutes a month. In parallel copies, the outage share is raised to a power, but the computation assumes copies fail independently, so it is an upper bound.

Next Step

This lesson converted service availability into a budget and counted how the budget gets spent: deployment, maintenance, failure. The open question is why the budget gets spent. Part of the outage comes from failure — a component goes down, and minutes leave the budget until it recovers. Part comes from a deliberate decision, and that part is invisible in the arithmetic. The bill for adding a copy was left unfinished in the last table: when copies are unaware of each other, the system has to choose between two things. Does it choose to respond, or to respond correctly? Choosing to respond preserves the budget and returns a wrong value; waiting or refusing in order to respond correctly spends from the budget. The next lesson names this choice, measures two policies under the same failure scenario, and counts how many minutes of the outage budget go to which policy.

To keep your progress and take notes, Log in

My notes

Log in to take notes.

Start typing to search.

↑↓ Esc navigate · open · close