Quotas and Service Limits

The previous lesson measured the downward direction: what can be downsized, what can be deleted. There is a number pointing upward too, and it never shows up on the bill. Every item in the rented layer comes with a ceiling; that ceiling is not set by the amount paid, it is set by the quota the provider places.

The quota mechanism itself was measured in M22/K03 and is not repeated here. This lesson’s question belongs to the scaling plan: how many units remain before each limit, which one fills first as growth continues, and what happens once it does.

The Limit Table and the Growth Plan

The example is the fictional regional measurement network: five teams, three environments, a rented machine, network, store, function, and edge layer. Limits, growth rates, and amounts are fictional; the currency is fictional too and is called unit, corresponding to no real price list.

CC25: there are eight limits; every limit has a scope (account, region, network, partition) and a driver that consumes it. CC26: the scope decides how many teams share the limit. CC27: the growth plan is per driver; every driver has its own monthly growth rate. CC28: every limit has an increasable ceiling and an increase-request duration; if the ceiling equals the limit, the limit cannot be increased. CC29: hitting a limit has four behaviors — creation-denial, request-denial, queue, silent throttling — and the day it is noticed depends on the behavior. CC30: a daily loss accrues while a limit stays full; on a limit that cannot be increased, the loss runs for thirty days, that is, until the plan changes.

The limit table and growth plan are a process-internal model; there is no real limit list.

// cost/limit.mjs — quota and service limit table model: no real limit list. Every row is a
// limit, tied to a *driver* that consumes it and to what happens on impact. The quota
// mechanism itself was measured in M22/K03; what is measured here is the growth plan applied
// to this table. The currency is fictional and is called `unit`.

// name, scope, driver, current usage, limit, increasable ceiling, increase-request days,
// behavior, daily loss while the limit stays full (unit)
export const LIMIT = [
  ["machine cores per region", "region", "counter", 412, 640, 2000, 6, "creation-denial", 34],
  ["object writes per second", "partition", "request", 1850, 3500, 12000, 4, "request-denial", 61],
  ["concurrent function executions", "account", "request", 640, 1000, 4000, 3, "queue", 18],
  ["name resolutions per second", "network", "request", 7200, 10000, 10000, 0, "silent throttling", 27],
  ["static addresses per account", "account", "edge", 34, 50, 200, 5, "creation-denial", 9],
  ["virtual networks per account", "account", "environment", 9, 12, 40, 7, "creation-denial", 4],
  ["retention rules per partition", "partition", "data", 62, 100, 100, 0, "creation-denial", 6],
  ["rules per rule set", "network", "service", 148, 200, 400, 9, "creation-denial", 12],
];

// growth plan: each driver's monthly growth rate (fictional)
export const GROWTH = { counter: 0.06, request: 0.11, data: 0.09, edge: 0.05, environment: 0.02,
  service: 0.04 };

// scope decides how many teams share the limit: an account-, region-, or network-scoped limit
// has one team's growth consume the others' headroom
export const SHARED = { account: 5, region: 5, network: 5, partition: 1 };

// behavior on hitting the limit: day noticed, and why it takes that long
export const BEHAVIOR = {
  "creation-denial": [1, "request fails on the spot"],
  "request-denial":  [1, "the caller sees the error"],
  "queue":           [4, "leaks out as latency"],
  "silent throttling": [12, "produces no error at all"],
};

export const fillMonth = (usage, limit, g) =>
  g <= 0 ? Infinity : Math.log(limit / usage) / Math.log(1 + g);
export const sum = (v) => v.reduce((a, b) => a + b, 0);

// cost/plan.mjs — the growth plan applied to the limit table: how many units remain before each
// limit, which limit fills first, whether the first-filling limit's identity changes when request
// growth is scanned, and the daily cost of hitting a limit. Numbers come from the model alone.
import { LIMIT, GROWTH, SHARED, BEHAVIOR, fillMonth, sum } from "./limit.mjs";

const write = (w, ...s) => console.log(s.map((v, i) =>
  (w[i] < 0 ? String(v).padEnd(-w[i]) : String(v).padStart(w[i]))).join(""));
const t0 = process.hrtime.bigint();

console.log(`${LIMIT.length} limits, ${Object.keys(GROWTH).length} drivers ` +
  `(fictional limit table and fictional growth plan)`);

console.log("\n1. how many units remain before each limit");
const A = [-33, -11, 7, 10, 9, 11, 10, 13];
write(A, "limit", "scope", "teams", "usage", "limit", "remaining", "fill", "driver");
for (const [name, scope, driver, use, lim] of LIMIT)
  write(A, name, scope, SHARED[scope], use, lim, lim - use, `${(100 * use / lim).toFixed(1)}%`, driver);
const fillAvg = sum(LIMIT.map(([, , , u, l]) => u / l)) / LIMIT.length;
const isShared = LIMIT.filter(([, scope]) => SHARED[scope] > 1);
console.log(`  average fill ${(100 * fillAvg).toFixed(1)}%; fullest ` +
  `${(100 * Math.max(...LIMIT.map(([, , , u, l]) => u / l))).toFixed(1)}%`);
console.log(`  ${isShared.length}/${LIMIT.length} limits are shared by five teams: one team's ` +
  `growth consumes another's headroom`);
const fixed = LIMIT.filter(([, , , , l, ceiling]) => ceiling <= l);
console.log(`  ${fixed.length} limits cannot be increased (ceiling = limit): ` +
  fixed.map((x) => x[0]).join(", "));

console.log("\n2. applying the growth plan: which limit fills first");
const B = [-33, 9, 12, 11, 11, 11];
write(B, "limit", "growth", "fill month", "month 12", "ceiling x", "increase");
const order = LIMIT.map((x) => [x, fillMonth(x[3], x[4], GROWTH[x[2]])])
  .sort((a, b) => a[1] - b[1]);
for (const [[name, scope, driver, use, lim, ceiling, days], n] of order) {
  const twelve = use * Math.pow(1 + GROWTH[driver], 12);
  write(B, name, `${(100 * GROWTH[driver]).toFixed(0)}%`, n === Infinity ? "-" : n.toFixed(1),
    `${(100 * twelve / lim).toFixed(0)}%`, (ceiling / lim).toFixed(2) + "x",
    ceiling <= lim ? "none" : `${days} days`);
}
const twelveMonth = LIMIT.filter(([, , driver, use, lim]) => use * Math.pow(1 + GROWTH[driver], 12) > lim);
console.log(`  fills first: ${order[0][0][0]} (month ${order[0][1].toFixed(1)}); ` +
  `second ${order[1][0][0]} (month ${order[1][1].toFixed(1)})`);
console.log(`  the first to fill is not the fullest limit: the fullest is ` +
  `${LIMIT.reduce((a, b) => (a[3] / a[4] >= b[3] / b[4] ? a : b))[0]}`);
console.log(`  ${twelveMonth.length}/${LIMIT.length} limits are exceeded by month 12`);

console.log("\n3. scanning request growth: does the first-filling limit change");
const C = [9, 33, 7, 33, 7];
write(C, "request", "first to fill", "month", "second to fill", "month");
for (const g of [0.02, 0.05, 0.08, 0.11, 0.16, 0.25]) {
  const grown = { ...GROWTH, request: g };
  const s = LIMIT.map((x) => [x, fillMonth(x[3], x[4], grown[x[2]])]).sort((a, b2) => a[1] - b2[1]);
  write(C, `${(100 * g).toFixed(0)}%`, s[0][0][0], s[0][1].toFixed(1), s[1][0][0],
    s[1][1].toFixed(1));
}
console.log(`  the request driver feeds three limits; changing its rate changes not just the`);
console.log(`  date but the *identity* of the limit that fills first`);

console.log("\n4. hitting a limit: what happens, and what is lost until it is increased");
const D = [-33, 19, 9, 10, 8, 12];
write(D, "limit", "behavior", "noticed", "increase", "days", "lost units");
let totalLoss = 0, totalDays = 0;
for (const [name, , , , lim, ceiling, days, behavior, gk] of LIMIT) {
  const [notice] = BEHAVIOR[behavior];
  const span = ceiling <= lim ? notice + 30 : notice + days;    // if it cannot be increased, the loss continues
  totalLoss += gk * span; totalDays += span;
  write(D, name, behavior, notice, ceiling <= lim ? "none" : days, span, (gk * span).toFixed(0));
}
console.log(`  total ${totalLoss.toFixed(0)} units lost, ${totalDays} limit-days; ` +
  `average ${(totalDays / LIMIT.length).toFixed(1)} days/limit`);
const silent = LIMIT.filter((x) => x[7] === "silent throttling");
const instant = LIMIT.filter((x) => BEHAVIOR[x[7]][0] === 1);
console.log(`  when it is noticed depends on behavior: ${instant.length} limits are seen the ` +
  `same day, silent throttling on day ${BEHAVIOR["silent throttling"][0]}`);
const sk = sum(silent.map((x) => x[8] * (12 + 30)));
console.log(`  silent throttling's loss is ${sk.toFixed(0)} units, ` +
  `${(100 * sk / totalLoss).toFixed(1)}% of the total; that limit cannot be increased, the plan has to change`);

const ms = Number(process.hrtime.bigint() - t0) / 1e6;   // raw measurement is not printed, only the band is checked
console.log(`\nfour tables within the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`);

8 limits, 6 drivers (fictional limit table and fictional growth plan)

1. how many units remain before each limit
limit                            scope        teams     usage    limit  remaining      fill       driver
machine cores per region         region           5       412      640        228     64.4%      counter
object writes per second         partition        1      1850     3500       1650     52.9%      request
concurrent function executions   account          5       640     1000        360     64.0%      request
name resolutions per second      network          5      7200    10000       2800     72.0%      request
static addresses per account     account          5        34       50         16     68.0%         edge
virtual networks per account     account          5         9       12          3     75.0%  environment
retention rules per partition    partition        1        62      100         38     62.0%         data
rules per rule set               network          5       148      200         52     74.0%      service
  average fill 66.5%; fullest 75.0%
  6/8 limits are shared by five teams: one team's growth consumes another's headroom
  2 limits cannot be increased (ceiling = limit): name resolutions per second, retention rules per partition

2. applying the growth plan: which limit fills first
limit                               growth  fill month   month 12  ceiling x   increase
name resolutions per second            11%         3.1       252%      1.00x       none
concurrent function executions         11%         4.3       224%      4.00x     3 days
retention rules per partition           9%         5.5       174%      1.00x       none
object writes per second               11%         6.1       185%      3.43x     4 days
machine cores per region                6%         7.6       130%      3.13x     6 days
rules per rule set                      4%         7.7       118%      2.00x     9 days
static addresses per account            5%         7.9       122%      4.00x     5 days
virtual networks per account            2%        14.5        95%      3.33x     7 days
  fills first: name resolutions per second (month 3.1); second concurrent function executions (month 4.3)
  the first to fill is not the fullest limit: the fullest is virtual networks per account
  7/8 limits are exceeded by month 12

3. scanning request growth: does the first-filling limit change
  request                    first to fill  month                   second to fill  month
       2%    retention rules per partition    5.5         machine cores per region    7.6
       5%    retention rules per partition    5.5      name resolutions per second    6.7
       8%      name resolutions per second    4.3    retention rules per partition    5.5
      11%      name resolutions per second    3.1   concurrent function executions    4.3
      16%      name resolutions per second    2.2   concurrent function executions    3.0
      25%      name resolutions per second    1.5   concurrent function executions    2.0
  the request driver feeds three limits; changing its rate changes not just the
  date but the *identity* of the limit that fills first

4. hitting a limit: what happens, and what is lost until it is increased
limit                                       behavior  noticed  increase    days  lost units
machine cores per region             creation-denial        1         6       7         238
object writes per second              request-denial        1         4       5         305
concurrent function executions                 queue        4         3       7         126
name resolutions per second        silent throttling       12      none      42        1134
static addresses per account         creation-denial        1         5       6          54
virtual networks per account         creation-denial        1         7       8          32
retention rules per partition        creation-denial        1      none      31         186
rules per rule set                   creation-denial        1         9      10         120
  total 2195 units lost, 116 limit-days; average 14.5 days/limit
  when it is noticed depends on behavior: 6 limits are seen the same day, silent throttling on day 12
  silent throttling's loss is 1134 units, 51.7% of the total; that limit cannot be increased, the plan has to change

four tables within the 2000 ms band: yes

The numbers belong to the measurement class; their inputs are the assumptions above.

Whose Name Is on the Remaining Headroom

The first table counts the headroom left on eight limits. Average fill is 66.5%, the fullest is virtual networks per account (75.0%), with three units of headroom remaining — a team wanting a fourth environment instead of three spends those three units.

Who the headroom belongs to is written into scope. Six of the eight limits are scoped to an account, region, or network, and shared by five teams. When one team claims a new static address, everyone else’s headroom shrinks; no one sees anyone else’s consumption. Only two limits are partition-scoped and belong to a single team. This is the reverse of the earlier lessons’ allocation question: there, cost could not be split across teams; here, the limit is already unsplit, and whoever arrives first consumes it.

Two limits’ ceiling equals their own value: name resolutions per second and retention rules per partition cannot be increased. For these, “how many units remain” has only one meaning, because there is no way for the number to grow.

The First Limit to Fill Is Not the Fullest

The second table runs the growth plan. The order comes out not by fill level but by the combination of fill level and growth rate: the first to fill is name resolutions per second, 72.0% full but 11% monthly growth, month 3.1; second is concurrent function executions (64.0% full, 11% growth, month 4.3).

The fullest limit, virtual networks per account, does not fill until month 14.5, because the environment count that consumes it grows only 2% a month. A quota report sorted by fill percentage produces exactly the reverse priority list on this network.

By month twelve, seven of the eight limits are exceeded. The overrun’s size also differs: name resolutions reaches 252% of its limit, function concurrency 224%. The ceiling-multiplier column gives the counterpart: function concurrency can be raised fourfold and survives twelve months, name resolutions cannot be increased at all.

Growth Rate Changes the Limit’s Identity

The third table scans a single driver. The request rate feeds three limits at once, making it the plan’s most uncertain variable.

At 2% and 5%, the first to fill is retention rules per partition (month 5.5), which has nothing to do with requests; it consumes data volume. From 8% on, name resolutions takes first place, dropping to month 1.5 at 25%. What changes is not just the date but the limit’s identity: slow growth means preparing for retention rules, fast growth means name resolutions, and the two preparations share nothing — one wants a retention policy, the other an architecture change.

The growth rate is an estimate. What this table says is that the cost of a wrong estimate is not preparing for the wrong date, it is preparing for the wrong limit.

The Behavior of Hitting the Limit and the Time to Increase It

The fourth table counts what happens after a limit fills. The day it is noticed depends on the behavior: creation-denial and request-denial produce an error the caller sees and are noticed the same day; a queue leaks out as latency (4 days); silent throttling produces no error at all and runs 12 days.

The increase request’s duration adds on top: between 3 and 9 days. The total comes to 116 limit-days and 2,195 units lost, averaging 14.5 days per limit. Object write requests close out fastest (5 days, 305 units): noticed immediately, increased within four days.

51.7% of the number comes from a single row: name resolutions per second. Silent throttling goes unnoticed for 12 days, and since the limit cannot be increased, the loss is not cut short by a single request; it runs 42 days and comes to 1,134 units. A limit that cannot be increased has no “increase duration” — instead there is a plan-change duration, not measured here.

Summary

• Eight limits average 66.5% full; the fullest has three units of headroom remaining. Six are shared by five teams: the limit is unsplit, and whoever grows first consumes the headroom.
• Two limits can never be increased (ceiling = limit); for these, the remaining-units number has only one meaning.
• The first limit to fill is not the fullest: the first is name resolutions, 72.0% full but growing 11% (month 3.1); the fullest limit, at 75.0%, does not fill until month 14.5 because it grows only 2%.
• By the end of twelve months, seven of the eight limits are exceeded; name resolutions reaches 252% of its limit.
• Scanning request growth from 2% to 25% changes the identity of the first limit to fill — from retention rules to name resolutions. A wrong estimate’s cost is not preparing for the wrong date, it is preparing for the wrong limit.
• The cost of hitting a limit is behavior plus increase duration: 116 limit-days, 2,195 units. 51.7% of that comes from a single silent-throttling limit — unnoticed for 12 days, unable to be increased, running 42 days and totaling 1,134 units.

Next Step

The same thing sits in every row: the rented side sets the limit’s value, whether it can be increased, and how many days that takes. On the two limits that cannot be increased, the only way out is changing the plan, and that change has stopped being the network’s own decision. The next lesson reads this dependency’s total: how many items can be moved, what replaces the ones that cannot, and what moving costs on the bill.