--- title: 'Cloud Governance, Security and Cost' source: 'https://academia.sh/en/courses/cloud-governance' language: en updated: '2026-09-13T05:07:34+00:00' license: 'CC BY-SA 4.0' lessons: 21 outcomes: - 'Reads and narrows a policy in terms of permission, resource, and condition' - 'Measures encryption, audit, and isolation decisions in terms of surface' - 'Ties cost to ownership and folds limits into the scaling plan' --- # Cloud Governance, Security and Cost Who owns everything that is rented and who receives the bill: the user, group, and role identity types, the policy structure built from permission, resource, and condition statements, the two evaluation directions of identity-based and resource-based policies, assuming a role with temporary credentials, giving an application an identity, detecting and narrowing excess permission, separating workloads through account and organization structure, encryption at rest and in transit, secret storage and rotation, audit logs, configuration compliance auditing, network isolation and private endpoints, threat detection, how compliance requirements map onto the cloud, the cost model, tagging and cost allocation, budget alerts, resource sizing, quotas and service limits, and vendor lock-in. ## Cloud Identity Writing down who can access what: the distinction between user, group, and role identity types, the policy structure built from permission, resource, and condition statements, evaluating identity-based and resource-based policies from both directions, assuming a role with temporary credentials, giving an application an identity, detecting unused permission and narrowing to least privilege, and separating workload and environment through account and organization structure. ### Users, Groups, and Roles Source: https://academia.sh/en/courses/cloud-governance/users-groups-and-roles The needs of five teams on the fictional regional measurement network are built and compared using three identity types: the connection count, excess permission, new-member cost, and permission left behind on departure for permission granted directly to a user, to a group, and to a role. Across four courses, every layer got rented: the machine, the network, the store, the function, and the edge. Each rental handed a decision to the provider and got an item back in return. The one thing never asked was whose name all that renting was done under. This lesson's question is who owns the rented thing, and how many connections that ownership is written with. There are three ways: a permission can be written directly to a person, to a group people are collected into, or to a role independent of any person. All three set up the same access; the difference is in the numbers — how many connections get written, how many people get more than they need, where a new arrival has to be touched, and what is left behind when someone leaves. - **CI1.** The model carries the five teams, three environments, and seven resource types of the fictional regional measurement network. No real account is run; what is measured is the model built with `node`. - **CI2.** The need table was written by hand and is derived from the (team, duty) pair, not from the person. In reality, need splits per person; since it is per duty here, the excess permission measured is a lower bound. - **CI3.** In this lesson a permission connection is only a (who, what) pair; which action can be taken is not counted. That distinction enters the policy document in the next lesson. - **CI4.** The ages of long-lived credentials come from a sequence we generate ourselves; the seed is written into the code as `20260802`. The distribution is fictional. - **CI5.** The limits are fictional and defined within the lesson: sixty groups, forty roles per account, ten policy documents per identity. - **CI6.** Departed people, which identity automations run under, and the daily action counts were written by hand. A permission matrix in the same form was measured earlier over cluster objects; the object here is not the cluster object but the resource of the rented account, and those numbers do not carry over. #### Team, Resource, and Need ```js // identity.mjs — fictional regional measurement network identity model; not a real account, only a model export const ENV = ['development', 'staging', 'production']; export const TEAM_RESOURCE = { // team -> resource types it touches ingest: ['edge-node-group', 'measurement-store', 'function-package', 'record-stream'], validation: ['machine-cluster', 'function-package', 'measurement-store', 'record-stream'], billing: ['object-store', 'measurement-store', 'record-stream'], field: ['edge-node-group', 'network-config', 'machine-cluster'], dashboard: ['measurement-store', 'record-stream'], }; export const DUTY_ENV = { // duty -> environments it works in developer: ['development', 'staging'], 'on-call': ['staging', 'production'], analyst: ['production'], lead: ENV, }; const ROSTER = { // twenty-three people; written by hand ingest: ['developer', 'developer', 'developer', 'on-call', 'analyst', 'lead'], validation: ['developer', 'developer', 'on-call', 'analyst', 'lead'], billing: ['developer', 'developer', 'analyst', 'lead'], field: ['developer', 'developer', 'on-call', 'on-call', 'lead'], dashboard: ['developer', 'analyst', 'lead'], }; export const PERSON = Object.entries(ROSTER).flatMap(([t, ds]) => ds.map((d, i) => ({ name: `${t}-${i + 1}`, team: t, duty: d }))); export const cross = (types, envs) => // the "what" side of a (who, what) pair new Set(types.flatMap((t) => envs.map((e) => `${t}@${e}`))); export const need = (p) => cross(TEAM_RESOURCE[p.team], DUTY_ENV[p.duty]); export const coarseGroup = (p) => cross(TEAM_RESOURCE[p.team], ENV); export const AUTOMATION = [ // name, identity, personal?, daily actions ['measurement-compaction', 'ingest-1', true, 2450], ['edge-health-scan', 'field-4', true, 720], ['nightly-billing-export', 'billing-1', true, 1840], ['dashboard-refresh', 'dashboard-2', true, 96], ['release-rollout', 'workload-identity-1', false, 310], ['backup-verification', 'workload-identity-2', false, 144], ].map(([name, identity, personal, daily]) => ({ name, identity, personal, daily })); export const DEPARTED = ['ingest-1', 'field-4', 'billing-3', 'dashboard-2']; export const PERSON_DAILY = 40; // actions per person per day, fictional export const LIMIT = { group: 60, role: 40, document: 10 }; // fictional limits, defined in the lesson export function* generator(seed) { // our own generator; the seed is visible let x = seed % 2147483647; for (;;) { x = (x * 48271) % 2147483647; yield x; } } ``` Need is the product of two factors: which resource types a team touches, and which environments a duty runs in. The coarse group carries the same types but not the environment split — belonging to a team means reaching all its resources across all three environments at once. #### Three Identity Types ```js // run1.mjs — the same need is built with three identity types and the connection count is compared import { PERSON, TEAM_RESOURCE, ENV, need, coarseGroup } from './identity.mjs'; const roleName = (p) => `${p.team}-${p.duty}`; const ROLE = new Map(), GROUP = new Map(); for (const p of PERSON) { if (!ROLE.has(roleName(p))) ROLE.set(roleName(p), need(p)); if (!GROUP.has(p.team)) GROUP.set(p.team, coarseGroup(p)); } const sum = (f) => PERSON.reduce((a, p) => a + f(p), 0); const sumSizes = (m) => [...m.values()].reduce((a, s) => a + s.size, 0); const newcomer = { team: 'ingest', duty: 'developer' }; // a new team member joining const MODEL = [ { name: 'direct user', defs: PERSON.length, idLink: 0, resLink: sum((p) => need(p).size), granted: need, newWrite: need(newcomer).size }, { name: 'group', defs: GROUP.size, idLink: PERSON.length, resLink: sumSizes(GROUP), granted: coarseGroup, newWrite: 1 }, { name: 'role', defs: ROLE.size, idLink: PERSON.length, resLink: sumSizes(ROLE), granted: need, newWrite: 1 }, ]; console.log(`${PERSON.length} people, ${Object.keys(TEAM_RESOURCE).length} teams, ${ENV.length} environments, ` + `${new Set(Object.values(TEAM_RESOURCE).flat()).size} resource types`); console.log(`need derived from duty: ${sum((p) => need(p).size)} (who, what) pairs\n`); console.log('model defs identity link resource link total granted excess new member'); for (const m of MODEL) { const granted = sum((p) => m.granted(p).size); const excess = sum((p) => [...m.granted(p)].filter((x) => !need(p).has(x)).length); console.log(`${m.name.padEnd(19)} ${String(m.defs).padEnd(6)} ${String(m.idLink).padEnd(16)} ` + `${String(m.resLink).padEnd(16)} ${String(m.idLink + m.resLink).padEnd(7)} ` + `${String(granted).padEnd(11)} ${String(excess).padEnd(7)} ${m.newWrite}`); } ``` ``` 23 people, 5 teams, 3 environments, 7 resource types need derived from duty: 157 (who, what) pairs model defs identity link resource link total granted excess new member direct user 23 0 157 157 157 0 8 group 5 23 48 71 231 74 1 role 17 23 115 138 157 0 1 ``` The need drawn from the duties of twenty-three people is a hundred fifty-seven pairs, and that number is independent of the model — what changes is how many connections build those pairs. In the direct write it is one-to-one: a hundred fifty-seven needed, a hundred fifty-seven connections. The group brings that down to seventy-one — eighty-six fewer connections — but granted permission climbs to two hundred thirty-one. Seventy-four permissions sit open that nobody asked for. This is where the group's cheapness comes from: the five group definitions never write the environment split at all, and not writing the environment means handing every member all three environments at once. An analyst who only works in production gets exactly three times what they need, because they reach every one of their team's resource types across all three environments. The role sits between the two: seventeen definitions, a hundred fifteen resource connections, twenty-three identity connections — a hundred thirty-eight total. Nineteen connections cheaper than the direct write, sixty-seven connections pricier than the group, but its excess permission is zero. Its real payoff shows in the last column: when a new team member arrives, the direct write sets up eight separate connections, the group and the role set up one. The direct write's cost is not paid once at setup — it is paid again with every arrival. #### Arriving, Leaving, and What Remains ```js // run2.mjs — departed people, permissions left behind and the subject seen in the audit record import { PERSON, DEPARTED, AUTOMATION, PERSON_DAILY, need, coarseGroup, generator } from './identity.mjs'; const lookup = (name) => PERSON.find((p) => p.name === name); const personal = AUTOMATION.filter((o) => o.personal); const linked = new Map(personal.map((o) => [o.identity, o])); console.log('departed duty need group permission automation running on it'); for (const name of DEPARTED) { const p = lookup(name), o = linked.get(name); console.log(`${name.padEnd(13)} ${p.duty.padEnd(12)} ${String(need(p).size).padEnd(8)} ` + `${String(coarseGroup(p).size).padEnd(19)} ${o ? `${o.name} (${o.daily} actions/day)` : '-'}`); } const cannotClose = DEPARTED.filter((a) => linked.has(a)); const stalled = cannotClose.reduce((a, x) => a + linked.get(x).daily, 0); const remaining = (f) => cannotClose.reduce((a, x) => a + f(lookup(x)).size, 0); console.log(`\n${cannotClose.length} identities cannot be closed, ${stalled} actions/day run on top of them`); console.log('model connections to delete permissions alive if left open actions/day stalled if closed'); const row = (name, del, perm) => console.log(`${name.padEnd(19)} ${String(del).padEnd(24)} ${String(perm).padEnd(33)} ${stalled}`); row('direct user', DEPARTED.reduce((a, x) => a + need(lookup(x)).size, 0), remaining(need)); row('group', DEPARTED.length, remaining(coarseGroup)); row('role', DEPARTED.length, remaining(need)); // Long-lived credentials: every identity has one key, personal keys have been copied const identityCount = PERSON.length + AUTOMATION.filter((o) => !o.personal).length; const gen = generator(20260802); const age = Array.from({ length: identityCount }, () => 45 + (gen.next().value % 856)); console.log(`\nlong-lived key: ${identityCount} identities + ${personal.length} copies = ` + `${identityCount + personal.length} storage locations; oldest ${Math.max(...age)} days, ` + `${age.filter((y) => y > 365).length} older than 365 days`); // Audit record: which line is written under whose name const automationTotal = AUTOMATION.reduce((a, o) => a + o.daily, 0); const personalTotal = personal.reduce((a, o) => a + o.daily, 0); const humanTotal = PERSON.length * PERSON_DAILY; const nameCarrying = personalTotal + humanTotal; console.log(`\ndaily audit record ${automationTotal + humanTotal} lines: human ${humanTotal}, automation ${automationTotal}`); console.log(`of the ${nameCarrying} lines written under a person's name, automation produced ${personalTotal} of them ` + `(${((100 * personalTotal) / nameCarrying).toFixed(1)}%)`); console.log(`the names of the ${cannotClose.length} departed people keep appearing in ${30 * stalled} lines over thirty days`); ``` ``` departed duty need group permission automation running on it ingest-1 developer 8 12 measurement-compaction (2450 actions/day) field-4 on-call 6 9 edge-health-scan (720 actions/day) billing-3 analyst 3 9 - dashboard-2 analyst 2 6 dashboard-refresh (96 actions/day) 3 identities cannot be closed, 3266 actions/day run on top of them model connections to delete permissions alive if left open actions/day stalled if closed direct user 19 16 3266 group 4 27 3266 role 4 16 3266 long-lived key: 25 identities + 4 copies = 29 storage locations; oldest 880 days, 16 older than 365 days daily audit record 6480 lines: human 920, automation 5560 of the 6026 lines written under a person's name, automation produced 5106 of them (84.7%) the names of the 3 departed people keep appearing in 97980 lines over thirty days ``` Four people left. The direct write should delete nineteen connections, the group and the role, four. The second column gives the unexpected part: three of the four identities cannot be closed, because automation runs on top of them. As long as these three stay open, sixteen permissions in the direct write stay alive, twenty-seven in the group. The third column gives the flip side. Closing the identities stops three thousand two hundred sixty-six legitimate actions per day, none of which involve the people who left. None of the three models solves the dilemma — the same number stalls in all three: the identity type changes the cleanup cost and the breadth of permission left behind, but not the machine work bound to a person's identity. The key line gives the surface's size. Each of the twenty-five identities has one long-lived key, four of them copied onto automation machines; twenty-nine storage locations in total. Sixteen keys are over a year old, the oldest eight hundred eighty days — a key's age is the upper bound on how long it stays valid once it leaks. The last three lines close out the "who can see" column. Eighty-four point seven percent of the six thousand twenty-six audit lines carrying a person's name were not produced by a human, and the names of the three departed people keep appearing in ninety-seven thousand nine hundred eighty lines over thirty days. The audit record is not giving a wrong answer here; it is answering the question it was asked, and the question is wrong. #### The Limit, What Remains, and the Cost of Narrowing ```js // run3.mjs — the limit, what remains and the cost of narrowing the coarse group import { PERSON, TEAM_RESOURCE, DUTY_ENV, ENV, LIMIT, need, coarseGroup, cross } from './identity.mjs'; const roleName = (p) => `${p.team}-${p.duty}`; const role = new Set(PERSON.map(roleName)); const split = PERSON.reduce((s, p) => { // if every role were split by its own environment for (const e of DUTY_ENV[p.duty]) s.add(`${roleName(p)}@${e}`); return s; }, new Set()); const mostDocs = Math.max(...PERSON.map((p) => TEAM_RESOURCE[p.team].length)); console.log(`fictional limits: ${LIMIT.group} groups, ${LIMIT.role} roles per account; ${LIMIT.document} policy documents per identity`); console.log('definition used remaining'); const limitRow = (name, k, s) => console.log(`${name.padEnd(29)} ${String(k).padEnd(11)} ${s - k}`); limitRow('group (one per team)', new Set(PERSON.map((p) => p.team)).size, LIMIT.group); limitRow('role (team × duty)', role.size, LIMIT.role); limitRow('role (split by environment)', split.size, LIMIT.role); limitRow('document (widest identity)', mostDocs, LIMIT.document); const rolesPerTeam = role.size / new Set(PERSON.map((p) => p.team)).size; console.log(`each team accounts for ${rolesPerTeam} roles; ` + `${Math.floor((LIMIT.role - role.size) / rolesPerTeam)} teams remain before the role limit, ` + `${Math.floor((LIMIT.role - split.size) / (split.size / 5))} teams in the environment-split definition`); // Narrowing the coarse group to its narrowest member: take the environment intersection console.log('\nteam members shared env coarse permission per member narrow permission per member stalled legitimate permission'); let totalDropped = 0, totalKept = 0, totalNeed = 0; for (const t of Object.keys(TEAM_RESOURCE)) { const members = PERSON.filter((p) => p.team === t); const intersection = ENV.filter((e) => members.every((p) => DUTY_ENV[p.duty].includes(e))); const narrow = cross(TEAM_RESOURCE[t], intersection); const needed = members.reduce((a, p) => a + need(p).size, 0); const kept = members.reduce((a, p) => a + [...need(p)].filter((x) => narrow.has(x)).length, 0); totalDropped += needed - kept; totalKept += kept; totalNeed += needed; console.log(`${t.padEnd(11)} ${String(members.length).padEnd(9)} ${String(intersection.length || '-').padEnd(12)} ` + `${String(coarseGroup(members[0]).size).padEnd(30)} ${String(narrow.size).padEnd(30)} ${needed - kept}`); } console.log(`\ntotal: coarse group grants ${PERSON.reduce((a, p) => a + coarseGroup(p).size, 0)} permissions, ` + `narrowed group ${totalKept}; of ${totalNeed} legitimate permissions ${totalDropped} are dropped ` + `(${((100 * totalDropped) / totalNeed).toFixed(1)}%)`); ``` ``` fictional limits: 60 groups, 40 roles per account; 10 policy documents per identity definition used remaining group (one per team) 5 55 role (team × duty) 17 23 role (split by environment) 35 5 document (widest identity) 4 6 each team accounts for 3.4 roles; 6 teams remain before the role limit, 0 teams in the environment-split definition team members shared env coarse permission per member narrow permission per member stalled legitimate permission ingest 6 - 12 0 48 validation 5 - 12 0 40 billing 4 - 9 0 24 field 5 1 9 3 18 dashboard 3 - 6 0 12 total: coarse group grants 231 permissions, narrowed group 15; of 157 legitimate permissions 142 are dropped (90.4%) ``` The limit table tells two separate stories. Nothing is tight on the group side: five groups, fifty- five slots free. On the role side, seventeen roles exist with twenty-three slots left; at three point four roles per team, six more teams could still be added. Splitting roles by environment — putting production permission in its own role — pushes the definition count to thirty-five and the slots left down to five: a narrowing that looks reasonable on its own closes the account off from growth. The lower table counts narrowing's other cost. Closing the coarse group's excess permission by touching the group means shrinking it to its narrowest member: the members' shared environment. Four of the five teams share no environment — a developer works in development and staging, an analyst only in production — so the intersection comes up empty. The coarse group's two hundred thirty-one permissions drop to fifteen, but a hundred forty-two of what dropped was legitimate need: ninety percent. The seventy-four excess permissions cannot be cleaned by touching the group definition. The excess is not inside the group — it is in its coarseness, and the only way to close it is to split the group; splitting it along duty lines produces exactly the role model. The group is cheap because it draws no distinction, the role is expensive because it writes the distinction, and the direct write rewrites that same distinction for every single person. #### Summary - The 157 (who, what) pairs drawn from the duties of twenty-three people are built with 157 connections by the direct write, 71 by the coarse group, 138 by the role; a new member costs 8 connections in the direct write, 1 in the group and the role. - The group's 86-connection cheapness is paid for with 74 excess permissions: granted permission climbs from 157 to 231, because the group never writes the environment split. - Three of four departed people cannot have their identity closed because 3,266 actions/day of automation run on top of them; while the identity stays open, 27 permissions stay alive in the group, 16 in the direct write and the role. - Of the 6,026 audit lines carrying a person's name, 5,106 were not produced by a human; the long-lived key of 25 identities sits in 29 places, and 16 of them are over a year old. - Narrowing the coarse group to its narrowest member closes the excess permission but drops 142 of 157 legitimate permissions; the excess is not in the group's content, it is in its coarseness. #### Next Step This lesson's connections carry only the (who, what) pair: the ingest team has access to the measurement store. Which access was not written — read, write, or delete the store itself. Until that set is written, the sentence "has access" does not measure anything. The next lesson opens up what lies inside the connection: the effect, action, resource pattern, and condition. It counts how many actions different phrasings of the same intent actually cover, pulls apart the scope a wildcard opens item by item, and shows why an explicit deny always wins. ### Policy Structure Source: https://academia.sh/en/courses/cloud-governance/policy-structure A policy document evaluator is written and the same intent is expressed four separate ways to count how many actions each phrasing actually covers: the scope a wildcard opens is pulled apart item by item, an explicit deny is shown to always win, and how far a condition narrows the scope is measured. The previous lesson's connections said "the ingest team has access to the measurement store" without saying which access. A connection actually stands for a set of actions, and until that set is written the connection count measures nothing. A policy document writes that set with four fields: **effect** (allow or deny), **action**, **resource**, and **condition**. This lesson's measure is what a phrasing actually covers. The gap sits between the writer's intent and what the evaluator reads. Intent can be narrow while the phrasing stays broad, and the gap produces no warning anywhere, because unused extra permission is not an error. It can only be seen by counting. - **CI7.** The model carries the fictional regional measurement network's seven resource types, forty-four action types, and forty-two resource instances. No real service is run; what is measured is the evaluator built with `node`. - **CI8.** A policy document carries four fields. The field set is defined within the lesson; richer phrasings add more fields, the measurement itself does not change shape. - **CI9.** The wildcard is only `*`, and its matching rule is defined in the code. As the pattern language grows, so does the scope; the numbers here are the scope this one character opens. - **CI10.** The destructive action set was marked by hand: delete verbs plus the verbs that change retention, access, or environment settings. The marking is debatable; the counted value depends on it. - **CI11.** Context was reduced to four profiles — internal and external network, fresh and stale auth. The condition measurement is read against these four profiles. - **CI12.** Document-order shuffling used a generator we wrote ourselves; the seed `31071923` is written into the code, and a hundred twenty orders were tried. #### The Evaluator ```js // policy.mjs — policy document evaluator; not a real service, only a model export const TYPE_ACTION = { // resource type -> actions defined on it 'machine-cluster': ['list', 'read', 'create', 'start', 'stop', 'resize', 'delete', 'tag-write'], 'network-config': ['list', 'read', 'rule-add', 'rule-delete', 'route-write', 'endpoint-open'], 'object-store': ['list', 'read', 'write', 'delete', 'version-read', 'lifecycle-write', 'access-write'], 'measurement-store': ['list', 'read', 'write', 'delete', 'query', 'retention-write'], 'function-package': ['list', 'read', 'publish', 'invoke', 'version-delete', 'env-var-write'], 'edge-node-group': ['list', 'read', 'record-add', 'record-delete', 'firmware-update'], 'record-stream': ['list', 'read', 'write', 'consume', 'retention-write', 'delete'], }; export const ENV = ['development', 'staging', 'production']; const INSTANCE = ['region-1', 'region-2']; // two instances per type and environment export const DESTRUCTIVE = new Set(['delete', 'rule-delete', 'record-delete', 'version-delete', 'access-write', 'env-var-write', 'retention-write', 'lifecycle-write', 'stop']); export const ACTION = Object.entries(TYPE_ACTION).flatMap(([t, fs]) => fs.map((f) => `${t}:${f}`)); export const RESOURCE = Object.keys(TYPE_ACTION).flatMap((t) => ENV.flatMap((e) => INSTANCE.map((i) => ({ name: `${t}/${e}/${i}`, type: t, env: e })))); export const PAIR = ACTION.flatMap((a) => // (action, resource) universe RESOURCE.filter((r) => r.type === a.split(':')[0]).map((r) => ({ action: a, resource: r }))); export const CONTEXT = [ // request context; written by hand { name: 'internal-fresh', net: 'internal', authAge: 5 }, { name: 'internal-stale', net: 'internal', authAge: 240 }, { name: 'external-fresh', net: 'external', authAge: 5 }, { name: 'external-stale', net: 'external', authAge: 240 }, ]; export const REQUEST = PAIR.flatMap((p) => CONTEXT.map((c) => ({ ...p, context: c }))); export const pattern = (p, s) => // the wildcard is only * new RegExp(`^${p.split('*').map((x) => x.replace(/[.+?^${}()|[\]\\]/g, '\\$&')).join('.*')}$`).test(s); const conditionHolds = (c, r) => (c.net === undefined || c.net === r.context.net) && (c.authAgeAtMost === undefined || r.context.authAge <= c.authAgeAtMost) && (c.env === undefined || c.env === r.resource.env); export const matches = (s, r) => s.action.some((p) => pattern(p, r.action)) && s.resource.some((p) => pattern(p, r.resource.name)) && (!s.condition || conditionHolds(s.condition, r)); // Rule: explicit deny always wins; document order does not change the outcome export function evaluate(statements, request) { const matched = statements.filter((s) => matches(s, request)); if (matched.some((s) => s.effect === 'deny')) return 'explicit-deny'; return matched.some((s) => s.effect === 'allow') ? 'allow' : 'implicit-deny'; } // For comparison: the first match wins export function firstMatch(statements, request) { for (const s of statements) if (matches(s, request)) return s.effect === 'deny' ? 'explicit-deny' : 'allow'; return 'implicit-deny'; } export function* generator(seed) { // our own generator; the seed is visible let x = seed % 2147483647; for (;;) { x = (x * 48271) % 2147483647; yield x; } } ``` The evaluator produces three outcomes. **Allow** means a matching allow statement exists and no matching deny statement does. **Explicit deny** means a deny statement matches, and it wins no matter how many allow statements also match. **Implicit deny** means no statement matches at all — what is not written is forbidden. The second evaluator stands for comparison: a rule where the first matching statement wins. #### The Same Intent, Four Phrasings ```js // run1.mjs — the same intent is phrased four ways and the actions it actually covers are counted import { ACTION, RESOURCE, PAIR, DESTRUCTIVE, matches, CONTEXT } from './policy.mjs'; const intent = (p) => ['read', 'list', 'query'].includes(p.action.split(':')[1]) && p.resource.type === 'measurement-store' && p.resource.env === 'production'; const PHRASING = [ ['A action *, resource *', { effect: 'allow', action: ['*'], resource: ['*'] }], ['B action measurement-store:*', { effect: 'allow', action: ['measurement-store:*'], resource: ['*'] }], ['C action *:read *:list *:query', { effect: 'allow', action: ['*:read', '*:list', '*:query'], resource: ['measurement-store/*'] }], ['D action and resource written in full', { effect: 'allow', action: ['measurement-store:read', 'measurement-store:list', 'measurement-store:query'], resource: ['measurement-store/production/*'] }], ]; console.log(`action types ${ACTION.length}, resources ${RESOURCE.length}, (action, resource) universe ${PAIR.length}`); console.log(`contexts ${CONTEXT.length}; pairs the intent requires ${PAIR.filter(intent).length}\n`); console.log('phrasing covered excess destructive action types touched'); for (const [label, s] of PHRASING) { const covered = PAIR.filter((p) => matches(s, { ...p, context: CONTEXT[0] })); const excess = covered.filter((p) => !intent(p)).length; const destructive = covered.filter((p) => DESTRUCTIVE.has(p.action.split(':')[1])).length; const types = new Set(covered.map((p) => p.resource.type)).size; console.log(`${label.padEnd(40)} ${String(covered.length).padEnd(9)} ${String(excess).padEnd(7)} ` + `${String(destructive).padEnd(20)} ${types}`); } ``` ``` action types 44, resources 42, (action, resource) universe 264 contexts 4; pairs the intent requires 6 phrasing covered excess destructive action types touched A action *, resource * 264 258 78 7 B action measurement-store:* 36 30 12 1 C action *:read *:list *:query 18 12 0 1 D action and resource written in full 6 0 0 1 ``` All four cover the same intent: the ingest team should be able to read the measurement stores in production. The intent requires exactly six pairs. Phrasing A opens two hundred sixty-four of them — the entire universe — and seventy-eight of the two hundred fifty-eight extra pairs land on a destructive action. The writer never wanted that; they left fields unfilled without meaning to. The gap between B and C shows which field was loosened. B leaves the action field open and covers thirty-six pairs; twelve are destructive, because two of the measurement store's six actions are delete and write-retention. C writes the action field and loosens the resource field instead: eighteen pairs, none destructive, but all three environments are inside. The only difference between C and D is one segment of the resource pattern, and scope changes threefold. The number says: loosening the action field brings in destructive actions, loosening the resource field removes the environment boundary. Two separate risks, written two separate ways. #### The Scope a Wildcard Opens ```js // run2.mjs — the scope a wildcard opens is pulled apart item by item import { ACTION, RESOURCE, PAIR, DESTRUCTIVE, pattern } from './policy.mjs'; const pairCount = (ap, rp) => PAIR.filter((p) => pattern(ap, p.action) && pattern(rp, p.resource.name)).length; console.log('action pattern matching action types destructive pairs with every resource'); for (const p of ['*', 'measurement-store:*', '*:read', '*:delete', '*:*-write', 'measurement-store:read']) { const e = ACTION.filter((a) => pattern(p, a)); console.log(`${p.padEnd(24)} ${String(e.length).padEnd(23)} ` + `${String(e.filter((a) => DESTRUCTIVE.has(a.split(':')[1])).length).padEnd(18)} ${pairCount(p, '*')}`); } console.log('\nresource pattern matching resources envs pairs with measurement-store:*'); for (const p of ['*', '*/production/*', 'measurement-store/*', 'measurement-store/production/*']) { const e = RESOURCE.filter((r) => pattern(p, r.name)); console.log(`${p.padEnd(32)} ${String(e.length).padEnd(20)} ${String(new Set(e.map((r) => r.env)).size).padEnd(6)} ` + `${pairCount('measurement-store:*', p)}`); } console.log('\n"measurement-store:*" item by item:'); console.log(' ' + ACTION.filter((a) => pattern('measurement-store:*', a)) .map((a) => `${a.split(':')[1]}${DESTRUCTIVE.has(a.split(':')[1]) ? ' (destructive)' : ''}`).join(', ')); const full = pairCount('measurement-store:read', 'measurement-store/production/*'); console.log('\nmultiple of the fully written form:'); for (const [ap, rp] of [['measurement-store:read', 'measurement-store/production/*'], ['measurement-store:read', 'measurement-store/*'], ['measurement-store:*', 'measurement-store/production/*'], ['measurement-store:*', 'measurement-store/*'], ['*', '*']]) { console.log(` ${ap.padEnd(24)} ${rp.padEnd(32)} pairs ${String(pairCount(ap, rp)).padEnd(4)} multiple ${pairCount(ap, rp) / full}`); } ``` ``` action pattern matching action types destructive pairs with every resource * 44 13 264 measurement-store:* 6 2 36 *:read 7 0 42 *:delete 4 4 24 *:*-write 7 5 42 measurement-store:read 1 0 6 resource pattern matching resources envs pairs with measurement-store:* * 42 3 36 */production/* 14 1 12 measurement-store/* 6 3 36 measurement-store/production/* 2 1 12 "measurement-store:*" item by item: list, read, write, delete (destructive), query, retention-write (destructive) multiple of the fully written form: measurement-store:read measurement-store/production/* pairs 2 multiple 1 measurement-store:read measurement-store/* pairs 6 multiple 3 measurement-store:* measurement-store/production/* pairs 12 multiple 6 measurement-store:* measurement-store/* pairs 36 multiple 18 * * pairs 264 multiple 132 ``` A single wildcard written alone matches forty-four action types, thirteen of them destructive. `measurement-store:*` means six verbs, and opening it item by item shows two of them are delete and write-retention — both verbs someone granting "read permission" never had in mind. The pattern text is five characters long; the scope it opens is sixfold. The last table shows the two fields behaving like a product. A fully written permission covers two pairs. Turning the environment segment of the resource pattern into a star multiplies that by three, turning the verb segment of the action pattern into a star multiplies it by six, and both together by eighteen. Reducing both fields to a single star multiplies it by a hundred thirty-two. This is why counting how many stars appear in a policy document is not enough on its own: which segment the star sits in matters more than how many there are. #### Explicit Deny and Condition ```js // run3.mjs — explicit deny always wins; how much a condition narrows the scope import { REQUEST, evaluate, firstMatch, matches, generator } from './policy.mjs'; const DOCUMENT = [ { name: 'p1-read', effect: 'allow', action: ['*:read', '*:list', '*:query'], resource: ['*'] }, { name: 'p2-write', effect: 'allow', action: ['measurement-store:write', 'record-stream:write'], resource: ['*/staging/*', '*/development/*'] }, { name: 'p3-maintenance', effect: 'allow', action: ['*'], resource: ['machine-cluster/*'] }, { name: 'p4-delete-ban', effect: 'deny', action: ['*:delete', '*:rule-delete', '*:record-delete', '*:version-delete'], resource: ['*/production/*'] }, { name: 'p5-external-net-ban', effect: 'deny', action: ['*'], resource: ['*/production/*'], condition: { net: 'external' } }, ]; const count = (ds, f = evaluate) => REQUEST.reduce((a, r) => (a[f(ds, r)]++, a), { allow: 0, 'explicit-deny': 0, 'implicit-deny': 0 }); console.log(`request universe ${REQUEST.length} = 264 (action, resource) x 4 contexts\n`); console.log('document set allow explicit deny implicit deny'); const base = count(DOCUMENT); console.log(`${'five documents (full set)'.padEnd(28)} ${String(base.allow).padEnd(6)} ` + `${String(base['explicit-deny']).padEnd(15)} ${base['implicit-deny']}`); const overlapping = REQUEST.filter((r) => DOCUMENT.some((d) => d.effect === 'allow' && matches(d, r)) && DOCUMENT.some((d) => d.effect === 'deny' && matches(d, r))).length; console.log(`requests matching both an allow and a deny document: ${overlapping} — deny wins in all of them`); console.log('\ndocument removed allow explicit deny implicit deny'); for (const name of ['p4-delete-ban', 'p5-external-net-ban']) { const s = count(DOCUMENT.filter((d) => d.name !== name)); console.log(`${(name + ' removed').padEnd(32)} ${String(s.allow).padEnd(6)} ${String(s['explicit-deny']).padEnd(15)} ${s['implicit-deny']}`); } const narrowed = DOCUMENT.map((d) => (d.name === 'p1-read' ? { ...d, condition: { authAgeAtMost: 60 } } : d)); const stalled = REQUEST.filter((r) => evaluate(DOCUMENT, r) === 'allow' && evaluate(narrowed, r) !== 'allow'); console.log(`once p1 gets an "auth age at most 60 min" condition, allow becomes ${count(narrowed).allow}; ${stalled.length} requests stall, ` + `actions: ${[...new Set(stalled.map((r) => r.action.split(':')[1]))].join(', ')}`); const gen = generator(31071923); // document order is shuffled with our own generator const shuffle = (a) => { const c = [...a]; for (let i = c.length - 1; i > 0; i--) { const j = gen.next().value % (i + 1); [c[i], c[j]] = [c[j], c[i]]; } return c; }; const baseFirst = REQUEST.map((r) => firstMatch(DOCUMENT, r)); let diffA = 0, diffB = 0; for (let n = 0; n < 120; n++) { const s = shuffle(DOCUMENT); REQUEST.forEach((r, x) => { if (evaluate(s, r) !== evaluate(DOCUMENT, r)) diffA++; if (firstMatch(s, r) !== baseFirst[x]) diffB++; }); } console.log(`\nacross 120 shuffled orders, ${REQUEST.length} requests:`); console.log(` explicit-deny-wins rule outcome changed ${diffA}`); console.log(` first-match-wins rule outcome changed ${diffB}`); ``` ``` request universe 1056 = 264 (action, resource) x 4 contexts document set allow explicit deny implicit deny five documents (full set) 448 204 404 requests matching both an allow and a deny document: 88 — deny wins in all of them document removed allow explicit deny implicit deny p4-delete-ban removed 452 176 428 p5-external-net-ban removed 528 56 472 once p1 gets an "auth age at most 60 min" condition, allow becomes 318; 130 requests stall, actions: list, read, query across 120 shuffled orders, 1056 requests: explicit-deny-wins rule outcome changed 0 first-match-wins rule outcome changed 5588 ``` Four hundred forty-eight of one thousand fifty-six requests are allowed, two hundred four are explicitly denied, four hundred four are implicitly denied because they match no document at all. Eighty-eight requests match both an allow document and a deny document, and deny wins on all of them. That is what explicit deny means: no matter how many allow documents match, no matter how many stars they carry, a single deny document overrides them. The document-removal rows separate how much of that explicit deny is actually doing work. Removing `p4` drops explicit deny from two hundred four to a hundred seventy-six — twenty-eight requests — but allow only rises by four. The remaining twenty-four requests were already falling into implicit deny, because no allow document opened them. `p4` looks like it denies twenty-eight requests but is actually blocking only four. The table for `p5` tells a different story: eighty of its hundred forty-eight explicit denials turn into allows once it is removed. Two deny documents that look alike carry a fivefold difference in weight. The condition row gives the cost of narrowing. Once `p1` gets an auth-age condition, allowed requests drop from four hundred forty-eight to three hundred eighteen; the hundred thirty stalled requests use read, list, and query — none of them destructive. The condition did not stop a destructive action; it stopped legitimate reads that arrived with stale auth. The condition's payoff has to be sought in a different column, because this column shows only its cost. The last two lines measure how much order matters. Across a hundred twenty shuffled orders, one thousand fifty-six requests were re-evaluated each time: zero outcomes changed under the explicit-deny rule. Under the first-match rule, five thousand five hundred eighty-eight changed — roughly four percent of the hundred twenty-six thousand seven hundred twenty evaluations. This is one of explicit deny's advantages: the outcome does not depend on the order documents were written in, so when two teams write documents without knowing about each other, the result is one both of them could have predicted. #### Summary - 44 action types and 42 resources build 264 (action, resource) pairs; a six-pair intent is covered 264, 36, 18, and 6 ways by four different phrasings. - Loosening the action field brings in destructive actions (12 pairs in phrasing B), loosening the resource field removes the environment boundary (threefold between C and D); two separate risks. - Which segment a star sits in matters more than the count: the environment segment is threefold, the verb segment sixfold, both together eighteenfold, a single star a hundred thirty-twofold. - 88 requests match both an allow and a deny document, and deny wins on all of them; across 120 shuffled orders, the outcome changed for 0 under explicit-deny-wins, 5,588 under first-match-wins. - Two deny documents look alike but carry different weight: `p4` denies 28 requests but blocks only 4, while 80 of `p5`'s 148 denials are a real block. - The condition added to `p1` drops allowed requests from 448 to 318; all 130 stalled requests are legitimate reads, none of them destructive. #### Next Step This lesson's five documents all sat in one place, and all of them were attached to a single subject. A resource can carry a policy too: a document written onto the store says who can reach it, from the resource's own side. A request then passes through two separate document sets, and the outcome is their intersection. The next lesson evaluates both directions together: how many requests are denied for being allowed in only one direction, why a resource-based policy becomes mandatory once the account boundary is crossed, and how many documents it takes to find out why an access was denied. ### Identity and Resource-Based Policies Source: https://academia.sh/en/courses/cloud-governance/identity-and-resource-based-policies The same access request is passed through two separate policy sets and their intersection is computed: how many requests are denied for being allowed in only one direction, why a resource-based policy becomes mandatory once the account boundary is crossed, and how many documents it takes to find out why an access was denied. The previous lesson's five documents all sat in one place, and all of them were attached to a single subject. The resource itself can carry a policy too: a document written onto the store says who can reach it, from the resource's own side. Two directions answer the same question from two separate places — one asks "where can this subject reach," the other "who can reach this resource" — and a request passes through both. The two directions exist for different reasons. The identity-based document sits with the team that manages the subject, the resource-based document with the team that operates the resource. Inside the same account the two can substitute for each other; once the account boundary is crossed, they cannot. This lesson's measure is the size of the intersection and the cost of reading it. - **CI13.** The model carries eight subjects, five actions, twenty-one resources, and three accounts. No real account is run; what is measured is the model built with `node`. - **CI14.** The evaluation rule is defined within the lesson: within the same account one direction allowing is enough, once the account boundary is crossed both directions must allow, and an explicit deny wins in either direction. - **CI15.** Eleven identity-based and six resource-based documents were written by hand. The numbers depend on this set; they are not the measurement's shape. - **CI16.** The thirty-day usage trace was written by hand and carries no environment split — a triple that appears in the trace counts as legitimate in all three environments. The unused permission count is therefore a lower bound. - **CI17.** The "documents to check" count is the sum of the subject-bound documents, the resource-matching documents, and the account rule. In reality inherited documents grow the number further; the measured value is a lower bound. - **CI18.** Organization structure was reduced to three accounts; accounts containing each other did not enter the model. #### Two Directions ```js // direction.mjs — two evaluation directions; not a real account, only a model built with node export const ACTION = ['read', 'list', 'write', 'delete', 'configure']; const ENV = ['development', 'staging', 'production']; export const TYPE_ACCOUNT = { // which account a resource type lives in 'measurement-store': 'account-measurement', 'record-stream': 'account-measurement', 'edge-node-group': 'account-measurement', 'object-store': 'account-analytics', 'function-package': 'account-analytics', 'machine-cluster': 'account-shared', 'network-config': 'account-shared', }; export const SUBJECT_ACCOUNT = { 'ingest-service': 'account-measurement', 'validation-service': 'account-measurement', 'edge-collector': 'account-measurement', 'analytics-job': 'account-analytics', 'dashboard-reader': 'account-analytics', 'billing-job': 'account-shared', 'field-maintenance': 'account-shared', 'backup-verification': 'account-shared', }; export const SUBJECT = Object.keys(SUBJECT_ACCOUNT); export const RESOURCE = Object.keys(TYPE_ACCOUNT).flatMap((t) => ENV.map((e) => ({ name: `${t}/${e}`, type: t, account: TYPE_ACCOUNT[t] }))); export const REQUEST = SUBJECT.flatMap((s) => ACTION.flatMap((a) => RESOURCE.map((r) => ({ subject: s, action: a, resource: r })))); const READ = ['read', 'list'], READ_WRITE = ['read', 'list', 'write']; const ID = (name, subject, effect, action, resource) => ({ name, subject, effect, action, resource }); const RD = (name, resource, effect, subject, action) => ({ name, resource, effect, subject, action }); export const IDENTITY_DOCUMENT = [ // attached to a subject: where it can do what ID('k1', 'ingest-service', 'allow', READ_WRITE, ['measurement-store/*', 'record-stream/*']), ID('k2', 'ingest-service', 'deny', ['delete'], ['*']), ID('k3', 'validation-service', 'allow', READ, ['*']), ID('k4', 'edge-collector', 'allow', READ_WRITE, ['edge-node-group/*', 'record-stream/*']), ID('k5', 'analytics-job', 'allow', READ, ['measurement-store/*', 'object-store/*']), ID('k6', 'analytics-job', 'allow', ['write'], ['object-store/staging', 'object-store/development']), ID('k7', 'dashboard-reader', 'allow', READ, ['measurement-store/*', 'record-stream/*']), ID('k8', 'billing-job', 'allow', READ, ['measurement-store/*', 'object-store/*', 'record-stream/*']), ID('k9', 'field-maintenance', 'allow', [...READ_WRITE, 'configure'], ['machine-cluster/*', 'network-config/*', 'edge-node-group/*']), ID('k10', 'field-maintenance', 'deny', ['delete'], ['*/production']), ID('k11', 'backup-verification', 'allow', READ, ['object-store/*', 'measurement-store/*']), ]; export const RESOURCE_DOCUMENT = [ // attached to a resource: who can do what RD('r1', ['measurement-store/*'], 'allow', ['analytics-job', 'billing-job'], READ), RD('r2', ['object-store/*'], 'allow', ['*'], READ), RD('r3', ['record-stream/*'], 'allow', ['ingest-service', 'edge-collector'], READ_WRITE), RD('r4', ['machine-cluster/*', 'network-config/*'], 'allow', ['field-maintenance'], [...READ_WRITE, 'configure']), RD('r5', ['*/production'], 'deny', ['dashboard-reader'], ['write', 'delete', 'configure']), RD('r6', ['edge-node-group/*'], 'allow', ['edge-collector', 'field-maintenance'], READ_WRITE), ]; // Thirty-day usage trace: (subject, action, resource type); written by hand, carries no environment split export const TRACE = new Set([ ['ingest-service', READ_WRITE, ['measurement-store', 'record-stream']], ['validation-service', ['read'], ['measurement-store', 'record-stream', 'machine-cluster']], ['edge-collector', ['read', 'write'], ['edge-node-group', 'record-stream']], ['analytics-job', READ, ['measurement-store', 'object-store']], ['dashboard-reader', READ, ['measurement-store', 'record-stream']], ['billing-job', ['read'], ['measurement-store', 'object-store', 'record-stream']], ['field-maintenance', ['read', 'write', 'configure'], ['machine-cluster', 'network-config']], ['backup-verification', ['read'], ['object-store', 'measurement-store']], ].flatMap(([s, as, ts]) => as.flatMap((a) => ts.map((t) => `${s}|${a}|${t}`)))); export const inTrace = (i) => TRACE.has(`${i.subject}|${i.action}|${i.resource.type}`); export const pattern = (p, s) => new RegExp(`^${p.split('*').map((x) => x.replace(/[.+?^${}()|[\]\\]/g, '\\$&')).join('.*')}$`).test(s); export const identityDocuments = (i, kb = IDENTITY_DOCUMENT) => kb.filter((b) => b.subject === i.subject); export const resourceDocuments = (i, rb = RESOURCE_DOCUMENT) => rb.filter((b) => b.resource.some((p) => pattern(p, i.resource.name))); const matches = (b, i) => b.action.includes(i.action) && b.resource.some((p) => pattern(p, i.resource.name)) && (typeof b.subject === 'string' ? b.subject === i.subject : b.subject.includes('*') || b.subject.includes(i.subject)); export function direction(documents, i) { // one direction's outcome const m = documents.filter((b) => matches(b, i)); if (m.some((b) => b.effect === 'deny')) return 'explicit-deny'; return m.some((b) => b.effect === 'allow') ? 'allow' : 'no-match'; } // Model rule: within the same account one direction is enough, across the account boundary both must allow export function evaluate(i, kb = IDENTITY_DOCUMENT, rb = RESOURCE_DOCUMENT) { const a = direction(identityDocuments(i, kb), i), b = direction(resourceDocuments(i, rb), i); if (a === 'explicit-deny' || b === 'explicit-deny') return 'explicit-deny'; const same = SUBJECT_ACCOUNT[i.subject] === i.resource.account; return (same ? a === 'allow' || b === 'allow' : a === 'allow' && b === 'allow') ? 'allow' : 'implicit-deny'; } ``` The two document types share the same fields but carry different lists: the identity-based document is bound to a subject and carries a resource pattern, the resource-based document is bound to a resource and carries a subject list. The `*` subject list in document `r2` is the most visible place this difference shows — instead of naming who can reach it, the resource can open itself to everyone. #### Intersection ```js // run1.mjs — the same request is passed through both directions and the intersection is computed import { REQUEST, SUBJECT, RESOURCE, ACTION, SUBJECT_ACCOUNT, IDENTITY_DOCUMENT, RESOURCE_DOCUMENT, identityDocuments, resourceDocuments, direction, evaluate } from './direction.mjs'; const sameAccount = (i) => SUBJECT_ACCOUNT[i.subject] === i.resource.account; const directions = REQUEST.map((i) => ({ i, a: direction(identityDocuments(i), i), b: direction(resourceDocuments(i), i), same: sameAccount(i), result: evaluate(i) })); console.log(`${SUBJECT.length} subjects x ${ACTION.length} actions x ${RESOURCE.length} resources = ${REQUEST.length} requests`); console.log(`identity-based documents ${IDENTITY_DOCUMENT.length}, resource-based documents ${RESOURCE_DOCUMENT.length}`); console.log(`same account ${directions.filter((y) => y.same).length}, crossing the account boundary ${directions.filter((y) => !y.same).length}\n`); const count = (f) => directions.filter(f).length; console.log('both directions compared same account account boundary allowed result'); const row = (name, f) => console.log(`${name.padEnd(27)} ${String(count((y) => y.same && f(y))).padEnd(13)} ` + `${String(count((y) => !y.same && f(y))).padEnd(17)} ${count((y) => f(y) && y.result === 'allow')}`); row('identity-only allow', (y) => y.a === 'allow' && y.b !== 'allow' && y.b !== 'explicit-deny'); row('resource-only allow', (y) => y.b === 'allow' && y.a !== 'allow' && y.a !== 'explicit-deny'); row('both directions allow', (y) => y.a === 'allow' && y.b === 'allow'); row('neither direction allows', (y) => y.a !== 'allow' && y.b !== 'allow'); row('either direction denies', (y) => y.a === 'explicit-deny' || y.b === 'explicit-deny'); const s = { allow: 0, 'explicit-deny': 0, 'implicit-deny': 0 }; for (const y of directions) s[y.result]++; console.log(`\nresult: allow ${s.allow}, explicit deny ${s['explicit-deny']}, implicit deny ${s['implicit-deny']}`); const oneDirectionWasEnough = count((y) => y.result !== 'allow' && (y.a === 'allow' || y.b === 'allow') && y.a !== 'explicit-deny' && y.b !== 'explicit-deny'); console.log(`allowed in one direction yet denied: ${oneDirectionWasEnough} requests — all of them cross the account boundary`); ``` ``` 8 subjects x 5 actions x 21 resources = 840 requests identity-based documents 11, resource-based documents 6 same account 285, crossing the account boundary 555 both directions compared same account account boundary allowed result identity-only allow 29 45 29 resource-only allow 6 18 6 both directions allow 57 39 96 neither direction allows 193 453 0 either direction denies 17 32 0 result: allow 131, explicit deny 49, implicit deny 660 allowed in one direction yet denied: 63 requests — all of them cross the account boundary ``` A hundred thirty-one of eight hundred forty requests are allowed. The first two rows show why same-account and account-boundary are separate columns: within the same account, twenty-nine requests are allowed in only the identity direction, and all twenty-nine end up allowed; the same holds for the six requests allowed in only the resource direction. Across the account boundary the same rows are forty-five and eighteen, but the allowed result is zero. The last line reduces this to a single number: sixty-three requests are denied even though one direction explicitly allows them. Someone really did write a document, the document matches, but access does not happen because the opposite direction was never written. That is what the account boundary does — it stops one side writing a document from being enough and makes both sides agree mandatory. #### Account Boundary ```js // run2.mjs — account boundary: when a resource-based document becomes mandatory import { REQUEST, RESOURCE_DOCUMENT, IDENTITY_DOCUMENT, TYPE_ACCOUNT, SUBJECT_ACCOUNT, SUBJECT, pattern, identityDocuments, direction, evaluate, inTrace, TRACE } from './direction.mjs'; const crossAccount = REQUEST.filter((i) => SUBJECT_ACCOUNT[i.subject] !== i.resource.account); console.log('resource type account allowing docs boundary requests allow id-allow, denied'); for (const t of Object.keys(TYPE_ACCOUNT)) { const d = crossAccount.filter((i) => i.resource.type === t); const docs = RESOURCE_DOCUMENT.filter((b) => b.effect === 'allow' && b.resource.some((p) => pattern(p, `${t}/production`))).length; const allowed = d.filter((i) => evaluate(i) === 'allow').length; const blocked = d.filter((i) => evaluate(i) !== 'allow' && direction(identityDocuments(i), i) === 'allow').length; console.log(`${t.padEnd(20)} ${TYPE_ACCOUNT[t].padEnd(22)} ${String(docs).padEnd(17)} ${String(d.length).padEnd(18)} ` + `${String(allowed).padEnd(6)} ${blocked}`); } const intersect = (kb, rb) => REQUEST.filter((i) => evaluate(i, kb, rb) === 'allow'); const legitimate = (l) => l.filter(inTrace).length; const full = intersect(IDENTITY_DOCUMENT, RESOURCE_DOCUMENT); console.log(`\nallowed requests ${full.length}; seen in the trace ${legitimate(full)}, unused permission ${full.length - legitimate(full)}`); console.log(`the trace carries ${TRACE.size} (subject, action, type) triples\n`); // Three stages of narrowing the resource-based document const r2Narrow = RESOURCE_DOCUMENT.map((b) => (b.name === 'r2' ? { ...b, subject: SUBJECT.filter((s) => [...TRACE].some((k) => k.startsWith(`${s}|`) && k.endsWith('|object-store'))) } : b)); console.log('resource-based document set allow seen in trace stopped legitimate work'); for (const [name, rb] of [['unchanged (r2 subject *)', RESOURCE_DOCUMENT], ['r2 subject list from trace', r2Narrow], ['r2 removed', RESOURCE_DOCUMENT.filter((b) => b.name !== 'r2')], ['none at all', []]]) { const k = intersect(IDENTITY_DOCUMENT, rb); console.log(`${name.padEnd(34)} ${String(k.length).padEnd(6)} ${String(legitimate(k)).padEnd(15)} ${legitimate(full) - legitimate(k)}`); } ``` ``` resource type account allowing docs boundary requests allow id-allow, denied measurement-store account-measurement 1 75 12 12 record-stream account-measurement 1 75 0 12 edge-node-group account-measurement 1 75 9 3 object-store account-analytics 1 90 18 0 function-package account-analytics 0 90 0 6 machine-cluster account-shared 1 75 0 6 network-config account-shared 1 75 0 6 allowed requests 131; seen in the trace 75, unused permission 56 the trace carries 32 (subject, action, type) triples resource-based document set allow seen in trace stopped legitimate work unchanged (r2 subject *) 131 75 0 r2 subject list from trace 119 75 0 r2 removed 107 69 6 none at all 86 60 15 ``` The `function-package` row shows the rule in its plainest form: no allowing resource document is attached to this type, so all ninety requests that cross the account boundary are denied, and six of them hold permission written on the identity side. Those six permissions sit somewhere, get maintained, show up in an audit, and do nothing. The `record-stream` row has an allowing document, but because the two subjects it names are already in the same account, it does nothing for any of the seventy-five requests that cross the boundary. The lower table gives the stages of narrowing. Once document `r2`'s `*` subject list is replaced with the subjects that appear in the thirty-day trace, allowed requests drop from a hundred thirty-one to a hundred nineteen and stopped legitimate work is zero: all twelve permissions were unused permission. Removing `r2` entirely drops allow to a hundred seven and stops six legitimate jobs; removing every resource-based document drops allow to eighty-six and stopped legitimate work rises to fifteen. Narrowing's first stage is free, the second is not. #### Why It Was Denied ```js // run3.mjs — how many documents finding out why an access was denied requires import { REQUEST, SUBJECT_ACCOUNT, identityDocuments, resourceDocuments, direction, evaluate } from './direction.mjs'; const reason = (i) => { const a = direction(identityDocuments(i), i), b = direction(resourceDocuments(i), i); if (a === 'explicit-deny') return 'explicit deny in identity direction'; if (b === 'explicit-deny') return 'explicit deny in resource direction'; if (a === 'allow' || b === 'allow') return 'account boundary: one direction allows'; return 'neither direction allows'; }; const denied = REQUEST.filter((i) => evaluate(i) !== 'allow'); const docsToCheck = (i) => identityDocuments(i).length + resourceDocuments(i).length + 1; // +1: the account rule const avg = (l, f) => (l.reduce((a, x) => a + f(x), 0) / l.length).toFixed(1); console.log(`denied ${denied.length} requests; docs to check = subject-bound + resource-matching + account rule\n`); console.log('denial reason requests avg docs max'); for (const n of ['explicit deny in identity direction', 'explicit deny in resource direction', 'account boundary: one direction allows', 'neither direction allows']) { const l = denied.filter((i) => reason(i) === n); console.log(`${n.padEnd(41)} ${String(l.length).padEnd(9)} ${String(avg(l, docsToCheck)).padEnd(15)} ` + `${Math.max(...l.map(docsToCheck))}`); } console.log(`${'total'.padEnd(41)} ${String(denied.length).padEnd(9)} ${String(avg(denied, docsToCheck)).padEnd(15)} ` + `${Math.max(...denied.map(docsToCheck))}`); console.log(`if only one direction were read (identity only): avg ${avg(denied, (i) => identityDocuments(i).length)} docs`); // How many requests a reader that only looks at one direction answers wrong const identityOnly = (i) => (direction(identityDocuments(i), i) === 'allow' ? 'allow' : 'deny'); const actual = (i) => (evaluate(i) === 'allow' ? 'allow' : 'deny'); const wrong = REQUEST.filter((i) => identityOnly(i) !== actual(i)); console.log(`\na reader that only looks at the identity direction is wrong on ${wrong.length} of ${REQUEST.length} requests`); console.log(`${wrong.filter((i) => SUBJECT_ACCOUNT[i.subject] !== i.resource.account).length} of them cross the account boundary`); const range = (f) => { const v = REQUEST.map(f); return `${Math.min(...v)}-${Math.max(...v)}`; }; console.log(`subject-bound docs ${range((i) => identityDocuments(i).length)}, ` + `resource-matching docs ${range((i) => resourceDocuments(i).length)}`); ``` ``` denied 709 requests; docs to check = subject-bound + resource-matching + account rule denial reason requests avg docs max explicit deny in identity direction 28 4.4 5 explicit deny in resource direction 21 3.9 4 account boundary: one direction allows 63 3.5 5 neither direction allows 597 3.5 5 total 709 3.5 5 if only one direction were read (identity only): avg 1.4 docs a reader that only looks at the identity direction is wrong on 51 of 840 requests 45 of them cross the account boundary subject-bound docs 1-2, resource-matching docs 0-2 ``` Finding the reason for a denial takes an average of three and a half documents to read, five in the worst case. The number is small because the model is small; the real information is in the ratio. If only one direction were read, the average would be one point four documents: adding the second direction increases the documents to read by two and a half times, and that increase is paid on every denial query. The payoff for that cost is in the last two lines. A reader that only looks at the identity direction gives the wrong answer on fifty-one of eight hundred forty requests — in every case it says "allowed," when the request is actually denied. Forty-five of them are caused by the account boundary, the remaining six have an explicit deny in the resource direction. The document was read correctly; what is wrong is looking at only one document. #### Summary - 131 of 840 requests are allowed; 285 requests are within the same account, 555 cross the account boundary. - 63 requests are denied even though one direction explicitly allows them, and all of them cross the account boundary; within the same account this number is zero, because one direction is enough there. - The `function-package` type has no allowing resource document: all 90 requests that cross the boundary are denied, and 6 identity-based permissions sit unused, doing nothing. - 75 of the 131 allowed requests appear in the trace; writing `r2`'s `*` subject list from the trace closes 12 unused permissions and stops no legitimate work. - The second stage is not free: removing `r2` stops 6 legitimate jobs, removing every resource-based document stops 15. - Finding the reason for a denial takes an average of 3.5 documents, 1.4 if only one direction were read; a reader that only looks at the identity direction is wrong on 51 of 840 requests. #### Next Step This lesson's eight subjects were persistent identities, and each one had a long-lived credential. That was the surface counted in the first lesson: twenty-nine storage locations and a key eight hundred eighty days old. However narrowly a permission connection is written, once the key leaks every document works in the leaker's favor. The next lesson separates access from persistent identity: a subject temporarily assumes another identity. What gets measured is the leak window, the length of the assumption chain, how the trust relationship determines who can assume it, and how many steps it takes before the true subject disappears from the audit record. ### Role Assumption Source: https://academia.sh/en/courses/cloud-governance/role-assumption Access with temporary credentials is compared against a long-lived key: the leak window, the length of the assumption chain, how the trust relationship determines who can assume it, and how many steps it takes before the true subject disappears from the audit record. The previous three lessons narrowed permission and taught how to read both directions together, but every subject was still a persistent identity, and every persistent identity had a long-lived credential. The first lesson counted the size of that surface: twenty-nine storage locations, the oldest eight hundred eighty days old. However narrowly a policy document is written, whoever gets hold of the key gets everything that document grants. Role assumption separates access from persistent identity. A subject authenticates with its own identity, then assumes a role and receives a **temporary credential** that carries that role's authority. Dynamically generating a credential and shortening its lifetime was already measured on the secret-management axis; that is not what is measured here. Three things are counted here: how far the window narrows, the chaining of assumption, and the trace the chain leaves in the audit record. - **CI19.** The model carries five persistent identities, six roles, and nine assumption chains. No real account is run; what is measured is the trust graph built with `node`. - **CI20.** Credential lifetimes and the hourly assumption limit are fictional and defined within the lesson. They are not a provider's values. - **CI21.** The leak's detection time was set at twenty-one days. The long-lived key's window was capped at this duration; in reality an undetected leak has a longer window, so the measured factor is a lower bound. - **CI22.** The leak was assumed to be uniformly distributed across the credential's validity interval; the expected window is therefore half the lifetime. - **CI23.** The thirty-day trace was written by hand and consists of nine chains. The code checks that the chains match the trust relationship. - **CI24.** The audit record is treated as carrying two fields: the final role and the session name. The assumption record additionally carries the caller's session name. Record retention duration did not enter the model. #### Trust Graph and Lifetimes ```js // assumption.mjs — trust relationship and assumption chain; not a real account, only a model built with node export const IDENTITY = ['ingest-service', 'analytics-job', 'field-maintenance', 'release-pipeline', 'on-call-human']; export const TRUST = { // role -> who can assume it (identity or role) 'role-read': ['ingest-service', 'analytics-job', 'role-bridge'], 'role-release': ['release-pipeline'], 'role-maintenance': ['field-maintenance', 'role-release'], 'role-management': ['on-call-human', 'role-maintenance'], 'role-audit': ['*'], 'role-bridge': ['on-call-human', 'release-pipeline'], }; export const ROLE = Object.keys(TRUST); // Thirty-day trace: which chain produced how many operations; written by hand export const TRACE = [ ['ingest-service', ['role-read'], 1200], ['analytics-job', ['role-read'], 800], ['field-maintenance', ['role-maintenance'], 260], ['release-pipeline', ['role-release', 'role-maintenance'], 340], ['on-call-human', ['role-bridge', 'role-read'], 90], ['on-call-human', ['role-management'], 40], ['release-pipeline', ['role-release', 'role-maintenance', 'role-management'], 25], ['field-maintenance', ['role-audit'], 60], ['analytics-job', ['role-audit'], 30], ]; export const trusts = (role, assumer, g = TRUST) => g[role].includes('*') || g[role].includes(assumer); export function chainValid(start, steps, g = TRUST) { let cur = start; for (const r of steps) { if (!trusts(r, cur, g)) return false; cur = r; } return true; } // Roles reachable from an identity and the shortest chain length (breadth-first search) export function reach(start, g = TRUST) { const dist = new Map(); let frontier = [start]; for (let d = 1; frontier.length; d++) { const next = []; for (const r of Object.keys(g)) { if (dist.has(r)) continue; if (frontier.some((s) => trusts(r, s, g))) { dist.set(r, d); next.push(r); } } frontier = next; } return dist; } export const LIFETIME = [ // name, validity duration (minutes) ['long-lived key', null], ['role assumption 12h', 720], ['role assumption 1h', 60], ['role assumption 15min', 15], ]; export const DETECT_MIN = 21 * 24 * 60; // leak detection time, fictional export const LIMIT_HOUR = 500; // max assumption calls per hour, fictional ``` The trust relationship sits on the role itself and carries a list: who can assume this role. The list can hold a persistent identity as well as another role — that is where the chain is born. The `*` on the `role-audit` row is this list's widest form, and is itself a subject of measurement. #### Leak Window ```js // run1.mjs — leak window: long-lived key versus role assumption import { IDENTITY, LIFETIME, DETECT_MIN, LIMIT_HOUR } from './assumption.mjs'; const MONTH = 30 * 24 * 60; // thirty-day window, minutes const longLived = DETECT_MIN; // for long-lived, window = detection time console.log(`thirty days ${MONTH} min; leak detection time ${DETECT_MIN} min (fictional)`); console.log(`${IDENTITY.length} persistent identities, hourly assumption limit ${LIMIT_HOUR} calls (fictional)\n`); console.log('credential type lifetime (min) count in 30d expected window worst window factor'); for (const [name, t] of LIFETIME) { const count = t ? MONTH / t : 1; const expected = t ? Math.min(t / 2, longLived) : longLived; const worst = t ? Math.min(t, longLived) : longLived; console.log(`${name.padEnd(25)} ${String(t ?? 'unlimited').padEnd(15)} ${String(count).padEnd(16)} ` + `${String(expected).padEnd(17)} ${String(worst).padEnd(16)} ${Math.round(longLived / expected)}`); } console.log('\nrenewal cost and limit:'); for (const [name, t] of LIFETIME.filter(([, t]) => t)) { const hourly = (60 / t) * IDENTITY.length; console.log(` ${name.padEnd(21)} calls/hour ${hourly.toFixed(2).padEnd(6)} ` + `left to limit ${(LIMIT_HOUR - hourly).toFixed(2).padEnd(7)} identities at same rate ${Math.floor(LIMIT_HOUR / (60 / t))}`); } ``` ``` thirty days 43200 min; leak detection time 30240 min (fictional) 5 persistent identities, hourly assumption limit 500 calls (fictional) credential type lifetime (min) count in 30d expected window worst window factor long-lived key unlimited 1 30240 30240 1 role assumption 12h 720 60 360 720 84 role assumption 1h 60 720 30 60 1008 role assumption 15min 15 2880 7.5 15 4032 renewal cost and limit: role assumption 12h calls/hour 0.42 left to limit 499.58 identities at same rate 6000 role assumption 1h calls/hour 5.00 left to limit 495.00 identities at same rate 500 role assumption 15min calls/hour 20.00 left to limit 480.00 identities at same rate 125 ``` The long-lived key's window comes not from its lifetime but from when the leak is detected: thirty thousand two hundred forty minutes, or three weeks. Throughout this time the key is still valid and nobody knows. For a one-hour temporary credential, the expected window drops to thirty minutes — a thousand eight times shorter — because the lifetime of what leaked is independent of how long it takes to notice. Even if nobody does anything, the credential invalidates itself. The cost is in the second column. A one-hour lifetime means seven hundred twenty separate credentials over thirty days; a fifteen-minute lifetime means two thousand eight hundred eighty. Each of these is an assumption call, and the call count runs into the quota. With five persistent identities, the fifteen-minute lifetime makes twenty calls an hour, leaving four hundred eighty calls of the fictional five-hundred limit; a hundred twenty-five identities running at the same rate would fit. The cost of narrowing the window fourfold is that the number of identities that fit drops to a quarter. #### Chain and Trust List ```js // run2.mjs — the trust relationship determines who can assume; how far the chain reaches import { IDENTITY, ROLE, TRUST, TRACE, reach, chainValid } from './assumption.mjs'; console.log(`${IDENTITY.length} persistent identities, ${ROLE.length} roles; chains in trace ${TRACE.length}`); console.log(`every chain in the trace matches the trust relationship: ${TRACE.every(([b, a]) => chainValid(b, a))}\n`); console.log('persistent identity direct via chain total longest chain used in trace'); let directTotal = 0, total = 0; for (const k of IDENTITY) { const u = reach(k); const d = [...u.values()].filter((x) => x === 1).length; const used = new Set(TRACE.filter(([b]) => b === k).flatMap(([, a]) => a)).size; directTotal += d; total += u.size; console.log(`${k.padEnd(21)} ${String(d).padEnd(9)} ${String(u.size - d).padEnd(10)} ${String(u.size).padEnd(7)} ` + `${String(Math.max(...u.values())).padEnd(15)} ${used}`); } console.log(`total ${String(directTotal).padEnd(9)} ${String(total - directTotal).padEnd(10)} ${total}`); // If role-audit's wide trust were narrowed to what the trace actually assumes const assumedInTrace = (role) => IDENTITY.filter((k) => TRACE.some(([b, a]) => b === k && a.includes(role))); const narrow = { ...TRUST, 'role-audit': assumedInTrace('role-audit') }; const pairCount = (g) => IDENTITY.reduce((a, k) => a + reach(k, g).size, 0); const legitimate = (g) => TRACE.filter(([b, a]) => chainValid(b, a, g)).reduce((s, [, , n]) => s + n, 0); console.log('\n' + 'trust list'.padEnd(47) + ' (identity, role) pairs stalled chains stalled ops/day'); for (const [name, g] of [['role-audit: *', TRUST], [`role-audit: ${narrow['role-audit'].join(', ')}`, narrow], ['role-audit removed', { ...TRUST, 'role-audit': [] }]]) { const stalled = TRACE.filter(([b, a]) => !chainValid(b, a, g)).length; console.log(`${name.padEnd(47)} ${String(pairCount(g)).padEnd(24)} ${String(stalled).padEnd(15)} ${legitimate(TRUST) - legitimate(g)}`); } ``` ``` 5 persistent identities, 6 roles; chains in trace 9 every chain in the trace matches the trust relationship: true persistent identity direct via chain total longest chain used in trace ingest-service 2 0 2 1 1 analytics-job 2 0 2 1 2 field-maintenance 2 1 3 2 2 release-pipeline 3 3 6 3 3 on-call-human 3 1 4 2 3 total 12 5 17 trust list (identity, role) pairs stalled chains stalled ops/day role-audit: * 17 0 0 role-audit: analytics-job, field-maintenance 14 0 0 role-audit removed 12 2 90 ``` Five identities and six roles form thirty possible pairs; the trust lists open seventeen of them. Twelve are direct assumption, five go through a chain. The chain does the most work for `release-pipeline`: it can directly assume three roles, reaches three more through the chain, and its longest chain is three steps. This identity is never granted `role-management` authority anywhere; it reaches `role-maintenance` through `role-release`, and from there `role-management`. Someone looking at the trust lists one at a time cannot see this, because no list writes `release-pipeline` and `role-management` side by side. The lower table measures the cost of a wide trust list. Because `role-audit` is open to every identity, five pairs come from it; replacing the list with the two names that actually assume it in the trace drops the pair count from seventeen to fourteen and stops zero legitimate work. Three pairs were assumption rights that sat open without ever being used. Going one stage further and closing the role entirely stops two chains and loses ninety operations a day. Narrowing's first stage is free, the second is not — the same result that came out of the resource-based document in the previous lesson, on a different axis. #### The True Subject in the Audit Record ```js // run3.mjs — does the true subject appear in the audit record, and after how many steps does it vanish import { ROLE, TRACE } from './assumption.mjs'; function records(name) { // records are produced from the thirty-day trace const assumption = [], operation = []; for (const [trueSubject, steps, daily] of TRACE) { let cur = trueSubject, prev = null, session = null; for (const [n, r] of steps.entries()) { session = name(trueSubject, r, n); assumption.push({ from: cur, role: r, session, caller: prev }); prev = session; cur = r; } operation.push({ role: steps.at(-1), session, daily, trueSubject, length: steps.length }); } return { assumption, operation }; } // Backtracking: possible true subjects from a (role, session) key function possibleTrueSubjects(assumption, role, session) { const set = new Set(); for (const u of assumption.filter((x) => x.role === role && x.session === session)) { if (!ROLE.includes(u.from)) set.add(u.from); else for (const a of possibleTrueSubjects(assumption, u.from, u.caller)) set.add(a); } return set; } const NAMING = [ // name, readable from last record, generator ['named by role', false, (a, r) => `${r}-session`], ['true subject only in first step', false, (a, r, n) => (n === 0 ? `${a}@${r}` : `${r}-session`)], ['true subject in every step', true, (a, r) => `${a}@${r}`], ]; const totalOps = TRACE.reduce((a, [, , n]) => a + n, 0); console.log(`the thirty-day trace has ${TRACE.length} chains, ${totalOps} operation records per day\n`); console.log('session naming subject in record resolved ops/day percent records to merge max candidates'); const results = new Map(); for (const [name, readable, f] of NAMING) { const { assumption, operation } = records(f); const candidates = operation.map((k) => ({ ...k, candidates: possibleTrueSubjects(assumption, k.role, k.session).size })); results.set(name, candidates); const resolved = candidates.filter((k) => k.candidates === 1); const resolvedOps = resolved.reduce((a, k) => a + k.daily, 0); const toMerge = readable ? '0' : (resolvedOps ? (resolved.reduce((a, k) => a + k.daily * k.length, 0) / resolvedOps).toFixed(2) : '-'); console.log(`${name.padEnd(34)} ${(readable ? 'yes' : 'no').padEnd(19)} ${String(resolvedOps).padEnd(18)} ` + `${(((100 * resolvedOps) / totalOps).toFixed(1) + '%').padEnd(7)} ${toMerge.padEnd(18)} ${Math.max(...candidates.map((k) => k.candidates))}`); } console.log('\nchain length chains ops/day resolved by role name resolved by first-step subject'); for (const u of [1, 2, 3]) { const selected = (name) => results.get(name).filter((k) => k.length === u); const resolvedAt = (name) => selected(name).filter((k) => k.candidates === 1).reduce((a, k) => a + k.daily, 0); console.log(`${String(u).padEnd(16)} ${String(selected('named by role').length).padEnd(7)} ` + `${String(selected('named by role').reduce((a, k) => a + k.daily, 0)).padEnd(10)} ` + `${String(resolvedAt('named by role')).padEnd(23)} ${resolvedAt('true subject only in first step')}`); } const steps = TRACE.reduce((a, [, s]) => a + s.length, 0); console.log(`\nassumption steps ${steps}; with a one-hour temporary credential, ${steps * 24} assumption records per day`); ``` ``` the thirty-day trace has 9 chains, 2845 operation records per day session naming subject in record resolved ops/day percent records to merge max candidates named by role no 0 0.0% - 3 true subject only in first step no 2845 100.0% 1.17 1 true subject in every step yes 2845 100.0% 0 1 chain length chains ops/day resolved by role name resolved by first-step subject 1 6 2390 0 2390 2 2 430 0 430 3 1 25 0 25 assumption steps 13; with a one-hour temporary credential, 312 assumption records per day ``` The operation record carries two fields: which role and which session. In a session named by role, the true subject never appears anywhere, and backtracking does not rescue it either — zero of the two thousand eight hundred forty-five records a day resolve to a single true subject. The reason is in the second table: the result is zero even for one-step chains. Three separate identities reach `role-read`, and because all three use the same session name, there is no way to tell which one a given read record came from. In the worst case there are three candidates. When the session name carries the true subject only in the first step, every record resolves, but the record itself still does not show the true subject: merging an average of one point one seven assumption records is required, and in a three-step chain that number rises to three. When the true subject is carried in every step, merging drops to zero — the question is answered directly from the record itself. The difference is measured during an incident. Merging three record sets requires having all three relevant records on hand and their time ranges overlapping; the choice of session name, meanwhile, is in the hands of whoever performs the assumption. The last line gives the volume: thirteen assumption steps, three hundred twelve assumption records a day with a one-hour lifetime. Narrowing the window also grows the audit record. #### Summary - The long-lived key's leak window equals the detection time (30,240 min); with a one-hour temporary credential the expected window is 30 min, 1,008 times shorter. - The window's cost is production volume: a 15-minute lifetime requires 2,880 credentials over thirty days and 20 calls an hour; that leaves 480 calls of the fictional 500 limit, and 125 identities fit at the same rate. - Five identities and six roles form 30 pairs, the trust lists open 17 of them: 12 direct, 5 via chain. `release-pipeline` reaches `role-management` in three steps even though no list ever writes them together. - Writing `role-audit`'s `*` trust list from the trace drops the pair count from 17 to 14 and stops no legitimate work; closing the role entirely stops 2 chains and 90 operations a day. - When the session name is given by role, zero of 2,845 operation records a day resolve to a single true subject; the result is zero even for one-step chains, and in the worst case there are 3 candidates. - When the true subject is carried only in the first step, every record resolves, but merging an average of 1.17 assumption records is required; carrying it in every step drops the merge to zero. #### Next Step Four of this lesson's five subjects were a program, one was a human, and all of them were handled the same way. The difference between persistent identity and temporary credential was measured, but one question went unanswered: where does a program get its first identity from. Assuming a role requires first authenticating with an identity, and if that first identity sits somewhere, there is still a long-lived secret at the start of the chain. The next lesson measures this first step: how many separate ways there are to give a workload an identity, how many secrets each way requires storing, and what item takes over once the number of stored secrets drops to zero. ### Workload Identities Source: https://academia.sh/en/courses/cloud-governance/workload-identities Three ways of giving an application an identity are measured side by side: a long-lived key sitting in a file, a secret in an environment variable, and an identity issued by the runtime. Leak surface, rotation work, and identity scope are counted; how identity is issued at the edge and function layer is worked out separately. The previous lesson made human identities temporary: authority is obtained by assumption and drops once the work is done. The same cannot be said for code. The measurement network's workload instances write to the data store overnight, and while doing it they read a key that sits somewhere. That key was written the day it was set up; nowhere does it say when it will change. The secret's lifetime and leak window were measured in M22/K07 and are not repeated here. This lesson's question is not lifetime but **location** and **scope**: how many separate places the identity sits in, how many places changing it requires touching, and how much more work opens up under that identity once a process is compromised. Three paths are set side by side — a long-lived key sitting in a file, a secret placed in an environment variable, and an identity issued by the runtime. - **CI25.** Ten workloads, three hosts, three environments, and twelve edge points belong to the fictional regional measurement network. No real key, secret, or credential was written; the names are made up. What is measured is the model written with `node`. - **CI26.** There is no randomness; the numbers are derived from the model's tables, and the output is the same from run to run. - **CI27.** The identity's lifetime and leak window are not measured. Three things are counted: where the identity sits, where rotation touches, and the identity's scope. - **CI28.** A location is either persistent or temporary, either keeps an access record or does not; partial recording is not modeled. - **CI29.** In a host-scoped identity, every process on the host can read the host's identity; isolation between processes is not modeled. In reality isolation lowers the entry, it does not zero it. - **CI30.** The edge layer has no runtime-issued identity; identity travels with the deployment package. This is a model decision; edge points were assumed not to carry a local identity endpoint. #### Where the Identity Sits The model holds ten workloads, which host each runs on, and the permissions each one needs. The seven workloads at the container and machine layers are spread across three hosts; the three at the function and edge layers have no host. The `LOCATION` table gives, for each path, where the identity sits, how many times each location exists in the system, whether it is persistent, and whether it keeps an access record. ```js // identity.mjs — workload, host, and identity-issuance path model // Fictional regional measurement network; no real key, secret, or credential, names are made up. export const ENV = ['development', 'staging', 'production']; export const WORKLOAD = { 'read-collector': { layer: 'container', host: 'k1', permission: ['measurement-write', 'queue-write'] }, 'validator': { layer: 'container', host: 'k1', permission: ['measurement-read', 'queue-read', 'measurement-write'] }, 'log-shipper': { layer: 'container', host: 'k1', permission: ['log-write', 'config-read'] }, 'billing': { layer: 'container', host: 'k2', permission: ['invoice-read', 'invoice-write', 'mail-send'] }, 'work-order': { layer: 'container', host: 'k2', permission: ['work-order-write', 'queue-read', 'mail-send'] }, 'nightly-batch': { layer: 'machine', host: 'm1', permission: ['measurement-read', 'archive-write', 'invoice-read'] }, 'archiver': { layer: 'machine', host: 'm1', permission: ['archive-read', 'archive-write'] }, 'metric-summary': { layer: 'function', host: null, permission: ['measurement-read'] }, 'report-generator': { layer: 'function', host: null, permission: ['invoice-read', 'archive-read'] }, 'edge-validator': { layer: 'edge', host: null, permission: ['measurement-write'] }, }; export const NAME = Object.keys(WORKLOAD); export const HOST = ['k1', 'k2', 'm1']; export const EDGE_POINT = 12; // points the edge layer is deployed to // Scale names: how many times a location exists in the system. export const SCALE = { instance: NAME.length * ENV.length, // (workload, env) pairs host: HOST.length * ENV.length, single: 1, }; // Where the identity sits: [name, scale, persistent?, keeps an access record?, how rotation reaches it] export const LOCATION = { 'key-in-file': [ ['instance-disk', 'instance', true, false, 'manual'], ['machine-image', 'host', true, false, 'manual'], ['config-store', 'instance', true, true, 'manual'], ['backup', 'host', true, false, 'unreachable'], ['developer-laptop', 'single', true, false, 'unreachable']], 'env-variable': [ ['deployment-spec', 'instance', true, true, 'manual'], ['process-env', 'instance', false, false, 'restart'], ['crash-dump', 'host', true, false, 'unreachable']], 'runtime-issued': [ ['process-memory', 'instance', false, true, 'automatic']], }; export const PATH = Object.keys(LOCATION); // Is the identity read when the process starts: if so, rotation needs a restart. export const READ_AT_START = { 'key-in-file': true, 'env-variable': true, 'runtime-issued': false }; export const scaleOf = (y) => SCALE[y[1]]; export const sumBy = (path, pick) => LOCATION[path].filter(pick).reduce((a, y) => a + scaleOf(y), 0); // In a host-scoped identity, the permission a host carries is the union of the workloads on it. export const hostPermission = (k) => [...new Set(NAME.filter((a) => WORKLOAD[a].host === k).flatMap((a) => WORKLOAD[a].permission))]; export const onHost = (k) => NAME.filter((a) => WORKLOAD[a].host === k); // Per layer: whether a runtime-issued identity exists, and its narrowest scope. export const LAYER = { machine: [true, 'host'], container: [true, 'instance'], function: [true, 'call'], edge: [false, 'deployment'], }; ``` ```js // run1.mjs — the leak surface and rotation work of all three paths in one table import { NAME, ENV, SCALE, LOCATION, PATH, sumBy, READ_AT_START } from './identity.mjs'; const G = [11, 8, 12, 11, 8, 10, 13]; // column widths const row = (name, v) => name.padEnd(18) + v.map((x, i) => String(x).padStart(G[i])).join(''); const countBy = (path, mode) => sumBy(path, (y) => y[4] === mode); console.log(`workloads ${NAME.length}, envs ${ENV.length}, instances ${SCALE.instance}, host items ${SCALE.host}\n`); console.log(row('', ['', '', '', '', 'manual', 'restart', 'unreachable'])); console.log(row('path', ['locations', 'copies', 'persistent', 'no record', 'touch', 'restarts', 'copies'])); for (const path of PATH) console.log(row(path, [LOCATION[path].length, sumBy(path, () => true), sumBy(path, (y) => y[2]), sumBy(path, (y) => !y[3]), countBy(path, 'manual'), READ_AT_START[path] ? SCALE.instance : 0, countBy(path, 'unreachable')])); ``` ``` workloads 10, envs 3, instances 30, host items 9 manual restart unreachable path locations copies persistent no record touch restarts copies key-in-file 5 79 79 49 69 30 10 env-variable 3 69 39 39 30 30 9 runtime-issued 1 30 0 0 0 0 0 ``` The key sitting in a file lives in five separate locations, seventy-nine copies in total, and all of them are persistent. Forty-nine copies sit in locations that keep no access record. The environment-variable path brings the location count down to three but only drops the total copies to sixty-nine; the real difference is in the persistent and no-record columns, both of which drop to thirty-nine. The third row puts the measure in its place. For the runtime-issued identity, persistent copies are zero, no-record copies are zero. There are thirty copies — one in every instance's memory — but every one of them gets recorded as it is fetched from the identity endpoint. The gain is not the copy count going down; it is the persistent copy count dropping from seventy-nine to zero, and forty-nine silent reads becoming recorded ones. #### Rotation Work The table's right three columns are the cost side. Seeing where these columns come from requires listing the locations out one by one. ```js // run2.mjs — where the identity sits and how rotation reaches those places import { LOCATION, PATH, scaleOf } from './identity.mjs'; for (const path of PATH) { console.log(path); for (const y of LOCATION[path]) console.log(` ${y[0].padEnd(21)}${String(scaleOf(y)).padStart(3)} ${(y[2] ? 'persistent' : 'temporary')}` + ` record ${(y[3] ? 'yes' : 'no').padEnd(4)} rotation: ${y[4]}`); } ``` ``` key-in-file instance-disk 30 persistent record no rotation: manual machine-image 9 persistent record no rotation: manual config-store 30 persistent record yes rotation: manual backup 9 persistent record no rotation: unreachable developer-laptop 1 persistent record no rotation: unreachable env-variable deployment-spec 30 persistent record yes rotation: manual process-env 30 temporary record no rotation: restart crash-dump 9 persistent record no rotation: unreachable runtime-issued process-memory 30 temporary record yes rotation: automatic ``` Changing the key in the file requires manually touching sixty-nine locations and restarting thirty instances; ninety-nine touches in total. For the environment variable, the manual-touch count drops to thirty, because the process environment renews itself automatically on restart. On the third path both columns are zero — there is no separate rotation job at all. The real finding is in the last column. On the file path, rotation never reaches ten copies at all: nine backups and one developer laptop. On the environment-variable path, nine crash dumps are in the same situation, and none of the three carry a record. Revoking the old one is a separate step, and whether that revocation can be done safely while unreachable copies exist is unknown: a machine restored from a backup boots with a revoked key. At the end of a ninety-nine-touch job, the item that never closes is not zero — it is ten. #### Identity Scope The third measure is scope: is the identity issued to an instance, or to the host that instance runs on. In a host-scoped identity, every process on the host can read the same identity, so the identity's permission is the union of the permissions of every workload on that host. ```js // run3.mjs — identity scope: how much extra permission and how many extra workloads a host-scoped identity opens import { NAME, ENV, WORKLOAD, HOST, hostPermission, onHost } from './identity.mjs'; console.log('host workloads union perms permissions'); for (const k of HOST) { const u = hostPermission(k); console.log(`${k.padEnd(7)}${String(onHost(k).length).padStart(7)}${String(u.length).padStart(15)} ${u.join(', ')}`); } console.log('\nworkload scope own host extra extra'); console.log(' perm perm perm workload'); let own = 0, hostScoped = 0, extraWork = 0; for (const a of NAME) { const w = WORKLOAD[a], k = w.host; const hostN = k ? hostPermission(k).length : w.permission.length; const extraWorkY = k ? onHost(k).length - 1 : 0; own += w.permission.length; hostScoped += hostN; extraWork += extraWorkY; console.log(`${a.padEnd(19)}${(k ?? 'no-host').padEnd(9)}${String(w.permission.length).padStart(5)}` + `${String(hostN).padStart(7)}${String(hostN - w.permission.length).padStart(9)}${String(extraWorkY).padStart(10)}`); } const O = ENV.length; console.log(''); for (const [name, v] of [ ['instance-scoped identity (instance, perm)', own * O], ['host-scoped identity (instance, perm)', hostScoped * O], ['extra permission entries granted', `${(hostScoped - own) * O} (${Math.round(((hostScoped - own) / own) * 100)}%)`], ['extra (instance, workload) entries reached', extraWork * O], ['most extra work opened from a single instance', Math.max(...HOST.map((k) => onHost(k).length - 1))], ['instances that open no extra work at all', `${NAME.filter((a) => !WORKLOAD[a].host).length * O} / ${NAME.length * O}`], ]) console.log(`${name.padEnd(47)}${v}`); ``` ``` host workloads union perms permissions k1 3 6 measurement-write, queue-write, measurement-read, queue-read, log-write, config-read k2 2 5 invoice-read, invoice-write, mail-send, work-order-write, queue-read m1 2 4 measurement-read, archive-write, invoice-read, archive-read workload scope own host extra extra perm perm perm workload read-collector k1 2 6 4 2 validator k1 3 6 3 2 log-shipper k1 2 6 4 2 billing k2 3 5 2 1 work-order k2 3 5 2 1 nightly-batch m1 3 4 1 1 archiver m1 2 4 2 1 metric-summary no-host 1 1 0 0 report-generator no-host 2 2 0 0 edge-validator no-host 1 1 0 0 instance-scoped identity (instance, perm) 66 host-scoped identity (instance, perm) 120 extra permission entries granted 54 (82%) extra (instance, workload) entries reached 30 most extra work opened from a single instance 2 instances that open no extra work at all 9 / 30 ``` The upper block shows how much the union grows. `k1` has three workloads on it, none of them needing more than three permissions, and the host's union permission is six. The log shipper needs two permissions but carries six under the host identity; two of the four extra ones are measurement and queue writes — two data paths the log shipper never touches at all. The lower block is the system total. The instance-scoped identity has sixty-six `(instance, permission)` entries, the host-scoped identity a hundred twenty. The fifty-four extra entries granted amount to eighty-two percent of the real need. Compromising a single instance opens extra access in thirty `(instance, workload)` entries; in the worst case a single process opens two more workloads. The last line states the limit of the scope measure: nine of the thirty instances share no host, so the scope question never even arises for them. The scope entry is the entry of host sharing. Moving the identity from a file to the runtime zeroes out the location and rotation columns, but it does not zero out this one — a runtime-issued identity can still be host-scoped. #### Edge and Function Layer Scope changes by layer: the runtime does not issue an identity the same way at every layer, and at some it does not issue one at all. ```js // run4.mjs — how identity is issued at the edge and function layer: scope and persistent copies counted separately import { NAME, ENV, WORKLOAD, LAYER, EDGE_POINT } from './identity.mjs'; const O = ENV.length; const onLayer = (lt) => NAME.filter((a) => WORKLOAD[a].layer === lt); const persistentCopies = (lt) => (LAYER[lt][0] ? 0 : onLayer(lt).length * EDGE_POINT * O); console.log('layer workloads instances runtime-issued narrowest persistent rotation'); console.log(' identity scope copies touch'); for (const lt of Object.keys(LAYER)) { const n = onLayer(lt).length; console.log(`${lt.padEnd(11)}${String(n).padStart(7)}${String(n * O).padStart(7)} ` + `${(LAYER[lt][0] ? 'yes' : 'no').padEnd(17)}${LAYER[lt][1].padEnd(10)}` + `${String(persistentCopies(lt)).padStart(6)}${String(persistentCopies(lt)).padStart(10)}`); } const canIssue = Object.keys(LAYER).filter((k) => LAYER[k][0]); const narrower = canIssue.filter((k) => LAYER[k][1] !== 'host'); const countAt = (ks) => ks.reduce((a, k) => a + onLayer(k).length * O, 0); console.log(''); for (const [name, v] of [ ['instances that can receive a runtime identity', `${countAt(canIssue)} / ${NAME.length * O}`], ['instances that can get instance-or-narrower scope', `${countAt(narrower)} / ${NAME.length * O}`], ['instances that remain host-scoped', `${countAt(canIssue.filter((k) => LAYER[k][1] === 'host'))} / ${NAME.length * O}`], ['instances that get none at all', `${countAt(Object.keys(LAYER).filter((k) => !LAYER[k][0]))} / ${NAME.length * O}`], ['persistent copies left in the system on the best path', Object.keys(LAYER).reduce((a, k) => a + persistentCopies(k), 0)], ]) console.log(`${name.padEnd(56)}${v}`); ``` ``` layer workloads instances runtime-issued narrowest persistent rotation identity scope copies touch machine 2 6 yes host 0 0 container 5 15 yes instance 0 0 function 2 6 yes call 0 0 edge 1 3 no deployment 36 36 instances that can receive a runtime identity 27 / 30 instances that can get instance-or-narrower scope 21 / 30 instances that remain host-scoped 6 / 30 instances that get none at all 3 / 30 persistent copies left in the system on the best path 36 ``` At the machine layer an identity exists, but its narrowest scope is the host: everything running on that machine can read the identity issued to it. At the container layer, instance scope is possible — each workload can get its own identity. The function layer is the narrowest of all — the identity is issued per call, disappears once the call ends, and persistent copies are zero. The edge row stands apart from the rest. Edge points have no local identity endpoint; the identity travels inside the deployment package, and twelve points times three environments makes thirty-six persistent copies. Rotation touches the same thirty-six locations. The layer that carries identity under the worst conditions is the one facing outward. The lower block gives the total. Twenty-seven of thirty instances can receive a runtime-issued identity, but only twenty-one of them can get instance-or-narrower scope; six remain host-scoped, and three get none at all. Even choosing the best path, thirty-six persistent copies remain in the system, all in a single layer. Dropping from seventy-nine to thirty-six is a real gain, but it does not reach zero; what remains is left there by what the layer makes impossible. #### Summary - The key sitting in a file lives in 5 locations, 79 copies, all persistent; 49 of them are in a location that keeps no access record. The environment variable drops persistent copies to 39. The runtime-issued identity has 30 copies but 0 persistent, 0 without a record — the gain is not in the copy count, it is in the copy's persistence and recordability. - Rotation requires 69 manual touches and 30 restarts on the file path, 30 touches and 30 restarts on the environment variable, 0 on the runtime-issued identity. - Copies rotation never reaches: 10 on the file path (9 backups, 1 laptop), 9 on the environment variable (crash dumps); these leave whether revocation can be done safely unresolved. - A host-scoped identity raises the `(instance, permission)` entry count from 66 to 120: 54 extra permissions, 82% of the real need. Compromising a single instance opens extra access in 30 `(instance, workload)` entries; 9 of the 30 instances share no host, so this entry never arises for them. - The layer determines identity: 27 of 30 instances can receive a runtime-issued identity, but only 21 can get instance-or-narrower scope, 6 remain host-scoped, 3 get none at all. - Even on the best path 36 persistent copies remain, all at the edge layer; with no local identity endpoint there, the identity multiplies to 12 points through the deployment package. #### Next Step This lesson changed where identity comes from, but never looked at what that identity can do. The number that came out of measuring scope already says it: identity `k1` carries six permissions, and none of the three workloads on it use all six. Even once instance scope is reached, the same question stands — is the permission set written for a workload the same set that workload actually uses? That is the next lesson's question. Once usage history and the granted policy are set side by side, what does the unused-permission ratio come out to, how many lines does the narrowed policy run to, and how much legitimate work does narrowing stop — the rare path that never showed up in history but is still necessary is the cost of narrowing. ### Least Privilege Practice Source: https://academia.sh/en/courses/cloud-governance/least-privilege-practice Granted permission is set side by side with used permission: the unused permission ratio, how many lines the narrowed policy runs to, and how much legitimate work narrowing stops are counted. The observation window is scanned and the safe narrowing threshold is given as a number. The previous lesson changed where identity comes from; what it can do was left open. When policy documents are written, permissions are usually granted not one at a time but at family level: a single line opens an entire measurement family's permissions, because writing a line is expensive, counting a permission is cheap. This lesson's question is how much of those lines actually gets used. The granted permission set is set side by side with usage history, and three things are counted: **unused permission ratio**, how many lines the narrowed policy runs to, and **how much legitimate work narrowing stops**. The third is as important as the first two: a path that never showed up in history does not mean it is unnecessary. - **CI31.** Eight workloads, eight resource families, and eighteen permissions belong to the fictional regional measurement network; what is measured is the model written with `node`. - **CI32.** The actual need list was not measured, it was chosen; each path's average usage interval is a model decision. In reality this list is unknown — the lesson's whole point is that it is unknown. - **CI33.** The usage history was generated with a linear congruential generator written by us. The seed is visible and fixed; the output stays the same from run to run. - **CI34.** A permission is either used or not used. Condition statements and resource-level narrowing are not modeled; these deepen narrowing, they do not change its direction. - **CI35.** The granted policy is written at family level, the narrowed policy is written at permission level. The line count is therefore a separate measure. - **CI36.** The test period is the second ninety days and comes from the same generator stream. How much legitimate work the narrowed policy stops is counted there. #### Granted Permission and Actual Need The model keeps three things separate: the policy written at family level, the permission set it opens, and the paths each workload actually needs. The third is unknown in a real system; here it is made known so that narrowing's accuracy can be measured. ```js // grant.mjs — granted permission, actual need, and usage history model // Fictional regional measurement network; eight workloads, eight resource families. This model is what is measured. export const FAMILY = { measurement: ['read', 'write', 'delete'], queue: ['read', 'write', 'manage'], archive: ['read', 'write', 'delete'], invoice: ['read', 'write'], 'work-order': ['read', 'write'], log: ['read', 'write'], config: ['read', 'write'], mail: ['send'], }; export const PERMISSION = Object.entries(FAMILY).flatMap(([a, e]) => e.map((x) => `${a}-${x}`)); // The granted policy is written at family level: one line grants an entire family's permissions. export const GRANTED = { 'read-collector': ['measurement', 'queue'], 'validator': ['measurement', 'queue'], 'log-shipper': ['log', 'config'], 'billing': ['invoice', 'mail', 'archive'], 'work-order': ['work-order', 'queue', 'mail'], 'nightly-batch': ['measurement', 'archive', 'invoice'], 'archiver': ['archive'], 'report-generator': ['invoice', 'archive', 'log'], }; export const SUBJECT = Object.keys(GRANTED); // Actual need: permission -> average usage interval (days). A model decision; not measured, chosen. export const ACTUAL = { 'read-collector': { 'measurement-write': 1, 'queue-write': 1, 'measurement-read': 3 }, 'validator': { 'measurement-read': 1, 'queue-read': 1, 'measurement-write': 2, 'measurement-delete': 30 }, 'log-shipper': { 'log-write': 1, 'config-read': 7 }, 'billing': { 'invoice-read': 1, 'invoice-write': 1, 'mail-send': 2, 'archive-read': 45 }, 'work-order': { 'work-order-write': 1, 'work-order-read': 1, 'queue-read': 1, 'mail-send': 5 }, 'nightly-batch': { 'measurement-read': 1, 'archive-write': 1, 'invoice-read': 7, 'measurement-delete': 20 }, 'archiver': { 'archive-read': 1, 'archive-write': 1, 'archive-delete': 60 }, 'report-generator': { 'invoice-read': 1, 'archive-read': 7, 'log-read': 30 }, }; export const PERIOD = 90; // observed number of days export const SEED = 20260802; // the generator's seed; visible and fixed export const WINDOW = [1, 7, 14, 30, 60, 90]; export const grantedPermission = (o) => GRANTED[o].flatMap((a) => FAMILY[a].map((x) => `${a}-${x}`)); export const grantedPair = SUBJECT.flatMap((o) => grantedPermission(o).map((i) => `${o}|${i}`)); export const actualPair = SUBJECT.flatMap((o) => Object.keys(ACTUAL[o]).map((i) => `${o}|${i}`)); export const interval = (c) => ACTUAL[c.split('|')[0]][c.split('|')[1]]; // Our own generator: a linear congruential generator; starts from the seed, gives the same sequence run to run. export function generator(seed) { let s = seed >>> 0; return () => { s = (Math.imul(s, 1664525) + 1013904223) >>> 0; return s / 4294967296; }; } // History: each real day, each actual pair is used with probability 1/interval. export function historyGen(days, seed) { const r = generator(seed), h = []; for (let d = 0; d < days; d++) for (const c of actualPair) if (r() < 1 / interval(c)) h.push([d, c]); return h; } ``` ```js // run1.mjs — granted permission side by side with actual need: unused permission ratio import { SUBJECT, GRANTED, ACTUAL, PERMISSION, grantedPermission, grantedPair, actualPair } from './grant.mjs'; console.log('subject lines granted actual unused'); let lines = 0; for (const o of SUBJECT) { const v = grantedPermission(o), g = Object.keys(ACTUAL[o]); lines += GRANTED[o].length; console.log(`${o.padEnd(19)}${String(GRANTED[o].length).padStart(5)}${String(v.length).padStart(9)}` + `${String(g.length).padStart(8)}${String(v.length - g.length).padStart(15)}`); } const unused = grantedPair.length - actualPair.length; console.log(''); for (const [name, v] of [ ['permission catalog', PERMISSION.length], ['granted policy lines (subject, family)', lines], ['granted (subject, permission) entries', grantedPair.length], ['actually needed entries', actualPair.length], ['unused permission entries', `${unused} (${Math.round((unused / grantedPair.length) * 100)}%)`], ]) console.log(`${name.padEnd(39)}${v}`); ``` ``` subject lines granted actual unused read-collector 2 6 3 3 validator 2 6 4 2 log-shipper 2 4 2 2 billing 3 6 4 2 work-order 3 6 4 2 nightly-batch 3 8 4 4 archiver 1 3 3 0 report-generator 3 7 3 4 permission catalog 18 granted policy lines (subject, family) 19 granted (subject, permission) entries 46 actually needed entries 27 unused permission entries 19 (41%) ``` Nineteen lines of policy open forty-six `(subject, permission)` entries; twenty-seven are actually needed. Nineteen entries, forty-one percent of granted permission, are never used at all. These entries were not added by mistake — not one of them was written individually. A single line written at family level brought in the whole family's permissions together. The lines column shows where the gap comes from. The archiver gets three permissions with one line and uses all three; its unused permission is zero. The report generator gets seven permissions with three lines and uses three; four sit idle. The nightly batch gets eight permissions with three lines, four idle. The wider a family a line touches, the bigger the surplus grows; the measurement and archive families each carry three permissions, and most of the surplus piles up in these two families' delete permission. #### Usage History The only way to find unused permission in a real system is to look at history. The model generates a ninety-day usage history; the following ninety days are set aside as the test period. ```js // run2.mjs — a 90-day usage history is generated; rare paths are counted separately import { PERIOD, SEED, actualPair, interval, historyGen } from './grant.mjs'; const full = historyGen(2 * PERIOD, SEED); // first period is observation, second is the test const history = full.filter(([d]) => d < PERIOD); const tally = new Map(actualPair.map((c) => [c, 0])); for (const [, c] of history) tally.set(c, tally.get(c) + 1); console.log(`seed ${SEED}, period ${PERIOD} days, events ${history.length}, test period ${full.length - history.length} events\n`); console.log('actual paths with an interval of 7 days or more:'); console.log('pair interval in 90d first seen'); for (const c of actualPair.filter((x) => interval(x) >= 7)) { const first = history.find(([, x]) => x === c); console.log(`${c.padEnd(34)}${String(interval(c)).padStart(10)}${String(tally.get(c)).padStart(10)}` + `${(first ? `day ${first[0] + 1}` : 'not seen').padStart(13)}`); } const seen = new Set(history.map(([, c]) => c)); console.log(''); for (const [name, v] of [ ['actual path', actualPair.length], ['seen at least once in 90 days', seen.size], ['actual path never seen in 90 days', actualPair.filter((c) => !seen.has(c)).join(', ') || 'none'], ]) console.log(`${name.padEnd(36)}${v}`); ``` ``` seed 20260802, period 90 days, events 1545, test period 1521 events actual paths with an interval of 7 days or more: pair interval in 90d first seen validator|measurement-delete 30 1 day 8 log-shipper|config-read 7 12 day 1 billing|archive-read 45 2 day 25 nightly-batch|invoice-read 7 15 day 3 nightly-batch|measurement-delete 20 7 day 5 archiver|archive-delete 60 0 not seen report-generator|archive-read 7 9 day 6 report-generator|log-read 30 3 day 25 actual path 27 seen at least once in 90 days 26 actual path never seen in 90 days archiver|archive-delete ``` Fifteen hundred forty-five events accumulate over ninety days, but that number is misleading: the overwhelming majority of events come from the fifteen paths that run daily. The table only lists the ones with an interval longer than a week. The validator's delete permission was used once in ninety days, on day eight; billing read the archive twice, the first time on day twenty-five. The last line is this lesson's harsh finding. The archiver's delete permission was not used on any of the ninety days. Twenty-six of twenty-seven actual paths were seen, one was not — and that unseen path is a real path. A narrowing that looks at history counts it as unused permission and cuts it. What happens after it is cut depends on when that path next runs; for a path that runs once every sixty days, that could be any day after the cut. #### Scanning the Observation Window The observation window is a choice: a short window cuts more, a long window is wrong less often. The trade-off between the two can be worked out with numbers. ```js // run3.mjs — the observation window is scanned: how much it cuts, how much it wrongly cuts import { PERIOD, SEED, WINDOW, grantedPair, actualPair, historyGen } from './grant.mjs'; const full = historyGen(2 * PERIOD, SEED); const history = full.filter(([d]) => d < PERIOD), test = full.filter(([d]) => d >= PERIOD); console.log('window narrowed unused wrong blocked in blocked'); console.log('(days) lines perm ratio cuts test period workload'); for (const p of WINDOW) { const visible = new Set(history.filter(([d]) => d >= PERIOD - p).map(([, c]) => c)); const wrongCut = actualPair.filter((c) => !visible.has(c)); const blocked = test.filter(([, c]) => !visible.has(c)); const ratio = Math.round(((grantedPair.length - visible.size) / grantedPair.length) * 100); console.log(`${String(p).padEnd(9)}${String(visible.size).padEnd(13)}${`${ratio}%`.padEnd(15)}` + `${String(wrongCut.length).padEnd(9)}${String(blocked.length).padEnd(12)}` + `${new Set(blocked.map(([, c]) => c.split('|')[0])).size}`); } console.log('\nwhich actual paths get cut as the window narrows:'); for (const p of WINDOW) { const visible = new Set(history.filter(([d]) => d >= PERIOD - p).map(([, c]) => c)); const y = actualPair.filter((c) => !visible.has(c)); console.log(` ${String(p).padStart(2)}d ${y.length ? y.slice(0, 4).join(', ') + (y.length > 4 ? ` (+${y.length - 4})` : '') : 'none'}`); } ``` ``` window narrowed unused wrong blocked in blocked (days) lines perm ratio cuts test period workload 1 19 59% 8 69 7 7 22 52% 5 12 4 14 23 50% 4 10 3 30 24 48% 3 5 2 60 25 46% 2 4 1 90 26 43% 1 0 0 which actual paths get cut as the window narrows: 1d read-collector|measurement-read, validator|measurement-delete, log-shipper|config-read, billing|archive-read (+4) 7d validator|measurement-delete, billing|archive-read, nightly-batch|measurement-delete, archiver|archive-delete (+1) 14d validator|measurement-delete, billing|archive-read, nightly-batch|measurement-delete, archiver|archive-delete 30d validator|measurement-delete, billing|archive-read, archiver|archive-delete 60d validator|measurement-delete, archiver|archive-delete 90d archiver|archive-delete ``` The table sets two outcomes of a single choice side by side. A one-day window cuts fifty-nine percent of forty-six entries, and the narrowed policy drops to nineteen lines — the same number as the granted policy, but this time at permission level and far narrower. The cost is in the two right-hand columns: eight legitimate paths get cut, and in the test period sixty-nine jobs stall across seven separate workloads. As the window grows, both sides soften. At the thirty-day window the cut ratio drops to forty-eight percent, wrong cuts drop to three, blocked jobs drop to five. At the ninety-day window wrong cuts drop to one, blocked jobs drop to zero. The cut-permission ratio fell from fifty-nine percent to forty-three percent; the cost of the long window is not cutting a sixteen-point slice of permission that could have been cut. The difference between the last two lines deserves attention. At the ninety-day window, wrong cuts are one, blocked jobs are zero. The path cut is the archiver's delete permission, and it never ran during the test period either. The bill for a wrong cut does not arrive on the day it is cut — it arrives on the day the path runs. The lower block gives the order: as the window narrows, first the rare paths get cut, then the weekly paths, and last everything that is not daily. #### Safe Narrowing Threshold If ninety observed days is enough to miss a path, how many days is not enough? This question is answered by looking not at history but at the interval: for a path with an interval of `d` days, the probability of it never appearing within a `p`-day window is `(1 − 1/d)^p`. ```js // run4.mjs — safe narrowing threshold: how a path's miss probability drops with the window import { PERIOD, actualPair, interval } from './grant.mjs'; const RISK = 0.01; // accepted probability of missing const missProb = (d, p) => Math.pow(1 - 1 / d, p); const required = (d) => (d === 1 ? 1 : Math.ceil(Math.log(RISK) / Math.log(1 - 1 / d))); const intervals = [...new Set(actualPair.map(interval))].sort((a, b) => a - b); console.log(' interval paths miss at 90d window for 1% risk'); for (const d of intervals) { const n = actualPair.filter((c) => interval(c) === d).length; console.log(`${String(d).padStart(10)}${String(n).padStart(7)}` + `${`${(missProb(d, PERIOD) * 100).toFixed(1)}%`.padStart(18)}${String(required(d)).padStart(20)}d`); } console.log(''); for (const [name, v] of [ ["rarest actual path's interval", `${Math.max(...intervals)}d`], ['safe narrowing threshold (1% risk)', `${Math.max(...intervals.map(required))}d`], ['observed window', `${PERIOD}d`], ['expected wrong cuts at 90d', actualPair.reduce((a, c) => a + missProb(interval(c), PERIOD), 0).toFixed(2)], ]) console.log(`${name.padEnd(37)}${v}`); ``` ``` interval paths miss at 90d window for 1% risk 1 15 0.0% 1d 2 2 0.0% 7d 3 1 0.0% 12d 5 1 0.0% 21d 7 3 0.0% 30d 20 1 1.0% 90d 30 2 4.7% 136d 45 1 13.2% 205d 60 1 22.0% 275d rarest actual path's interval 60d safe narrowing threshold (1% risk) 275d observed window 90d expected wrong cuts at 90d 0.46 ``` The table gives the threshold interval by interval. For the fifteen paths that run daily, one day is enough; for the three paths that run weekly, thirty days. At a twenty-day interval, the required window rises to ninety days — the entire observed period, for a single path. At a sixty-day interval, the required window is two hundred seventy-five days, three times the observed period. The second column from the right gives the risk remaining at a ninety-day window. The sixty-day path's miss probability is twenty-two percent, the forty-five-day path's is thirteen percent, each of the two thirty-day paths five percent. Summed up, the expected number of wrong cuts at ninety days is zero point four six paths; one path actually happened in the run. Model and run are the same order of magnitude. The safe narrowing threshold is this table's one-line answer: two hundred seventy-five days. This number is not a system property, it is a function of the rarest legitimate path's interval; in a system whose interval is unknown, the threshold is unknown too. Narrowing with a ninety-day window means a forty-three percent cut and half a wrong cut — whether that is acceptable depends on which workload that half lands on. #### Summary - A 19-line policy written at family level opens 46 `(subject, permission)` entries; 27 are actually needed. Unused permission is 19 entries, 41% of granted permission — and none of them were written one at a time, all of them came from the width of a line. - 1,545 events accumulate over a 90-day history, but only 26 of 27 actual paths are seen. The unseen path is the archiver's delete permission; its interval is 60 days, and a narrowing that looks at history cuts it. - The window scan gives the trade-off: a 1-day window cuts 59% but stops 8 legitimate paths, and in the test period 69 jobs are blocked across 7 workloads; a 90-day window cuts 43%, makes 1 wrong cut, blocks 0 jobs. - The bill for a wrong cut does not arrive on the day it is cut: the one path cut at a 90-day window did not run during the test period either, and blocked jobs showed up as 0. - Narrowing does not shrink the line count: a 19-line family-level policy turns into a 26-line permission-level policy at a 90-day window. A narrow policy is a longer policy. - The safe narrowing threshold for a 1% miss risk is 275 days — about 4.6 times the rarest path's 60-day interval. Over the observed 90 days, that path's miss probability is 22% and the expected wrong cut is 0.46 paths. #### Next Step This lesson measured narrowing a policy set, and showed where narrowing stops. But every table stands on a single assumption: all these workloads sit in one place, inside the same boundary. If all forty-six entries are written in a single account, compromising one identity is a door open to that entire account, and narrowing only reshapes the key, not the width of the door. The next lesson's question is therefore the boundary itself: when the same set of workloads is set up in a single account, in accounts split by environment, and in accounts split by workload, how many resources does the blast radius come to, how is the quota shared, does the bill split apart, and how many accounts and how many policy copies does it take to manage. ### Account and Organization Structure Source: https://academia.sh/en/courses/cloud-governance/account-and-organization-structure The same workload set is set up in three structures — single account, account by environment, account by workload. Blast radius, quota sharing, how the bill splits apart, and management overhead are counted; what the organization-level protection band cuts and cannot cut is measured. The previous two lessons measured where identity comes from and what it can do; both stayed within a single boundary. Even after policy is narrowed, one question remains unanswered: inside which boundary do these workloads sit? The same eight workloads and three environments can be set up in three separate structures — all in a single account, in accounts split by environment, or in accounts split by workload. This lesson sets up all three side by side and counts four things: blast radius, how many workloads share the quota, how many lines the bill splits into, and how many accounts and how many policy copies are managed. At the end, what the organization-level protection band cuts and cannot cut is measured. - **CI37.** Eight workloads, three environments, eight resource families, and twenty-four resources belong to the fictional regional measurement network; what is measured is the model written with `node`. - **CI38.** Blast radius is the union of two sets: the resources the identity reaches through policy, and the resources inside its account. Escalation within the account is assumed possible, crossing the account boundary is assumed impossible. - **CI39.** Quota is granted per account and modeled as a single number. It is a fictional unit; it is not a real price or limit list. - **CI40.** Demand is constant; spikes, backoff, and queuing are not modeled. These change the moment the limit is hit, not which account hits it. - **CI41.** A workload's bill share is counted as directly readable only if it is alone in its account; distribution through tagging is not measured in this lesson. - **CI42.** The organization-level protection band is attached to the account and cannot separate what is inside it. This is a model decision; whether the band can cut by environment depends on the account boundary separating the environment. #### Same Set, Three Structures The model names resources by a family-and-environment pair, every family has an owning workload, and the three structures answer a single question differently: which account a workload falls into. Everything else — policy, resources, demand — is the same across all three structures. ```js // account.mjs — the same workload set is set up in three account structures; fictional measurement network model export const ENV = ['development', 'staging', 'production']; export const FAMILY = ['measurement', 'queue', 'archive', 'invoice', 'work-order', 'log', 'config', 'mail']; export const PERMISSION = { // family -> permissions the family opens measurement: ['read', 'write', 'delete'], queue: ['read', 'write', 'manage'], archive: ['read', 'write', 'delete'], invoice: ['read', 'write'], 'work-order': ['read', 'write'], log: ['read', 'write'], config: ['read', 'write'], mail: ['send'], }; export const GRANTED = { // workload -> families it has access to 'read-collector': ['measurement', 'queue'], 'validator': ['measurement', 'queue'], 'log-shipper': ['log', 'config'], 'billing': ['invoice', 'mail', 'archive'], 'work-order': ['work-order', 'queue', 'mail'], 'nightly-batch': ['measurement', 'archive', 'invoice'], 'archiver': ['archive'], 'report-generator': ['invoice', 'archive', 'log'], }; export const OWNER = { // owning workload of a resource family measurement: 'read-collector', queue: 'validator', archive: 'archiver', invoice: 'billing', 'work-order': 'work-order', log: 'log-shipper', config: 'log-shipper', mail: 'billing', }; export const ACTUAL = { // paths actually used 'read-collector': ['measurement-write', 'queue-write', 'measurement-read'], 'validator': ['measurement-read', 'queue-read', 'measurement-write', 'measurement-delete'], 'log-shipper': ['log-write', 'config-read'], 'billing': ['invoice-read', 'invoice-write', 'mail-send', 'archive-read'], 'work-order': ['work-order-write', 'work-order-read', 'queue-read', 'mail-send'], 'nightly-batch': ['measurement-read', 'archive-write', 'invoice-read', 'measurement-delete'], 'archiver': ['archive-read', 'archive-write', 'archive-delete'], 'report-generator': ['invoice-read', 'archive-read', 'log-read'], }; export const SUBJECT = Object.keys(GRANTED); export const DEMAND = { // fictional unit: requests/minute, at production scale 'read-collector': 420, 'validator': 260, 'log-shipper': 90, 'billing': 140, 'work-order': 110, 'nightly-batch': 380, 'archiver': 70, 'report-generator': 60, }; export const ENV_SCALE = { development: 0.1, staging: 0.3, production: 1 }; export const QUOTA = 1000; // quota per account, fictional unit export const RESOURCE = FAMILY.flatMap((a) => ENV.map((e) => `${a}@${e}`)); export const IDENTITY = SUBJECT.flatMap((s) => ENV.map((e) => `${s}@${e}`)); // Three structures: which account a workload falls into. export const STRUCTURE = { 'single-account': () => 'single', 'by-environment': (s, e) => e, 'by-workload': (s, e) => `${s}@${e}`, }; export const identityAccount = (y, k) => STRUCTURE[y](k.split('@')[0], k.split('@')[1]); export const resourceAccount = (y, r) => STRUCTURE[y](OWNER[r.split('@')[0]], r.split('@')[1]); // Resources reached through policy: the families granted in the identity's own environment. export const policyResource = (k) => GRANTED[k.split('@')[0]].map((a) => `${a}@${k.split('@')[1]}`); // Resources within the account: the set opened if a compromised identity escalates inside the account. export const accountResource = (y, k) => RESOURCE.filter((r) => resourceAccount(y, r) === identityAccount(y, k)); export const blastRadius = (y, k) => new Set([...accountResource(y, k), ...policyResource(k)]).size; // Organization-level permission boundary: [permission, which environment it cuts in] export const BOUNDARY = [ ['measurement-delete', (e) => e !== 'production'], ['archive-delete', (e) => e !== 'production'], ['config-write', () => true], ]; export const grantedPermission = (s) => GRANTED[s].flatMap((a) => PERMISSION[a].map((x) => `${a}-${x}`)); ``` ```js // run1.mjs — blast radius: how many resources open up if an identity is compromised import { RESOURCE, IDENTITY, STRUCTURE, identityAccount, accountResource, blastRadius, policyResource } from './account.mjs'; console.log(`resources ${RESOURCE.length}, identities ${IDENTITY.length}\n`); console.log('structure accts widest average (identity, resource) policy'); console.log(' blast blast entries share'); for (const y of Object.keys(STRUCTURE)) { const p = IDENTITY.map((k) => blastRadius(y, k)); const accts = new Set(IDENTITY.map((k) => identityAccount(y, k))).size; const pol = IDENTITY.reduce((a, k) => a + policyResource(k).length, 0); console.log(`${y.padEnd(18)}${String(accts).padStart(5)}${String(Math.max(...p)).padStart(10)}` + `${(p.reduce((a, x) => a + x, 0) / p.length).toFixed(1).padStart(10)}` + `${String(p.reduce((a, x) => a + x, 0)).padStart(23)}${String(pol).padStart(10)}`); } console.log('\nproduction identities in by-workload in account via policy blast'); for (const k of IDENTITY.filter((x) => x.endsWith('production'))) console.log(` ${k.padEnd(38)}${String(accountResource('by-workload', k).length).padStart(9)}` + `${String(policyResource(k).length).padStart(13)}${String(blastRadius('by-workload', k)).padStart(9)}`); ``` ``` resources 24, identities 24 structure accts widest average (identity, resource) policy blast blast entries share single-account 1 24 24.0 576 57 by-environment 3 8 8.0 192 57 by-workload 24 3 2.4 57 57 production identities in by-workload in account via policy blast read-collector@production 1 2 2 validator@production 1 2 2 log-shipper@production 2 2 2 billing@production 2 3 3 work-order@production 1 3 3 nightly-batch@production 0 3 3 archiver@production 1 1 1 report-generator@production 0 3 3 ``` In the single account, every one of the twenty-four identities opens all twenty-four resources if compromised. Split by environment, that number drops to eight; split by workload, to three at most. The total `(identity, resource)` entry count drops from five hundred seventy-six to fifty-seven — tenfold. The last column tells where the structure stops. The access policy alone grants fifty-seven entries in all three structures; the account structure does not change this number, it only determines what gets added on top. In the single account, five hundred nineteen entries come from the account boundary; in the by-workload structure, the addition is zero. A narrowed policy is only truly narrow if the account boundary backs it up. The lower block shows why. The nightly batch and report generator own no resources at all; their accounts hold zero resources, and their radius comes entirely from policy. The archiver owns one family and accesses one family, so its radius is one. How tightly the structure can squeeze is determined by how ownership is distributed. #### Quota Sharing and What Remains An account is not just a security boundary — it is also a quota container. ```js // run2.mjs — quota sharing: which account hits the limit, how much headroom remains import { IDENTITY, STRUCTURE, QUOTA, DEMAND, ENV_SCALE, identityAccount } from './account.mjs'; const demand = (k) => Math.round(DEMAND[k.split('@')[0]] * ENV_SCALE[k.split('@')[1]]); console.log(`quota per account ${QUOTA} fictional units; total demand ${IDENTITY.reduce((a, k) => a + demand(k), 0)}\n`); console.log('structure accts most sharing fullest headroom over'); console.log(' one quota account left limit'); for (const y of Object.keys(STRUCTURE)) { const load = new Map(); for (const k of IDENTITY) { const a = identityAccount(y, k); load.set(a, [(load.get(a)?.[0] ?? 0) + demand(k), (load.get(a)?.[1] ?? 0) + 1]); } const loads = [...load.values()], fullest = Math.max(...loads.map((v) => v[0])); console.log(`${y.padEnd(18)}${String(load.size).padStart(5)}${String(Math.max(...loads.map((v) => v[1]))).padStart(17)}` + `${String(fullest).padStart(10)}${String(QUOTA - fullest).padStart(10)}` + `${String(loads.filter((v) => v[0] > QUOTA).length).padStart(6)}`); } console.log('\naccount load in the by-environment structure:'); for (const e of ['development', 'staging', 'production']) { const t = IDENTITY.filter((k) => k.endsWith(e)).reduce((a, k) => a + demand(k), 0); console.log(` ${e.padEnd(12)}demand ${String(t).padStart(4)} headroom ${String(QUOTA - t).padStart(5)}`); } ``` ``` quota per account 1000 fictional units; total demand 2142 structure accts most sharing fullest headroom over one quota account left limit single-account 1 24 2142 -1142 1 by-environment 3 8 1530 -530 1 by-workload 24 1 420 580 0 account load in the by-environment structure: development demand 153 headroom 847 staging demand 459 headroom 541 production demand 1530 headroom -530 ``` In the single account, twenty-four identities share the same thousand-unit quota, and total demand is two thousand one hundred forty-two. Headroom left is negative eleven hundred forty-two units — the account exceeds the quota by more than double. The result of exceeding it is not one workload slowing down but all of them — a test running in the development environment sits in the same container as the collector running in production. Split by environment, the container splits into three, and the real information is in the lower block. Development has eight hundred forty-seven units idle, staging five hundred forty-one; production, meanwhile, exceeds by five hundred thirty. Total demand did not change, only where it lands changed — and the thirteen hundred eighty-eight idle units do not cover production's shortfall, because quotas cannot be transferred between accounts. In the by-workload structure, no account hits the limit; the fullest account has five hundred eighty units of headroom left. The cost is twenty-four separate quotas — each one a limit that has to be watched, raised, and can be forgotten separately. As quota sharing shrinks, the risk of hitting the limit drops, and the number of limits grows. #### Billing and Management Overhead An account boundary determines a third thing too: how many lines the bill splits into, and how many policy documents have to be maintained. ```js // run3.mjs — how the bill splits apart and the management overhead: how many accounts, how many policy copies import { IDENTITY, STRUCTURE, identityAccount, resourceAccount, policyResource } from './account.mjs'; console.log('structure bill directly identity resource total'); console.log(' lines readable -based -based policy'); for (const y of Object.keys(STRUCTURE)) { const byAccount = new Map(); for (const k of IDENTITY) byAccount.set(identityAccount(y, k), (byAccount.get(identityAccount(y, k)) ?? 0) + 1); const aloneCount = IDENTITY.filter((k) => byAccount.get(identityAccount(y, k)) === 1).length; const cross = IDENTITY.reduce((a, k) => a + policyResource(k).filter((r) => resourceAccount(y, r) !== identityAccount(y, k)).length, 0); console.log(`${y.padEnd(18)}${String(byAccount.size).padStart(6)}${`${aloneCount}/${IDENTITY.length}`.padStart(10)}` + `${String(IDENTITY.length).padStart(10)}${String(cross).padStart(10)}${String(IDENTITY.length + cross).padStart(10)}`); } console.log('\ncross-account access in the by-workload structure:'); for (const k of IDENTITY.filter((x) => x.endsWith('production'))) { const c = policyResource(k).filter((r) => resourceAccount('by-workload', r) !== identityAccount('by-workload', k)); console.log(` ${k.padEnd(30)}${String(c.length).padStart(2)} ${c.map((r) => r.split('@')[0]).join(', ') || '-'}`); } ``` ``` structure bill directly identity resource total lines readable -based -based policy single-account 1 0/24 24 0 24 by-environment 3 0/24 24 0 24 by-workload 24 24/24 24 33 57 cross-account access in the by-workload structure: read-collector@production 1 queue validator@production 1 measurement log-shipper@production 0 - billing@production 1 archive work-order@production 2 queue, mail nightly-batch@production 3 measurement, archive, invoice archiver@production 0 - report-generator@production 3 invoice, archive, log ``` Bill lines equal the account count: one, three, twenty-four. In the first two structures, no workload's share can be read at account level; the by-environment structure lets the environment total be read, but not a workload's share. In the third structure, twenty-four of twenty-four can be read. The three right-hand columns give the cost. Identity-based policy count is twenty-four in all three structures. What changes is the resource-based policy: the by-workload structure produces thirty-three cross-account accesses, each one a document that has to be written separately in the resource's account. Total policy documents rise from twenty-four to fifty-seven. The lower block shows where cross-account access piles up. The two workloads that own no resources carry three cross-account accesses each; the log shipper and the archiver carry none, because they own every family they access. The cost of splitting the structure is exactly the size of the gap between ownership and access. #### Organization-Level Protection Band A permission boundary can be placed on top of accounts: whatever gets written inside the account, the permission the band cuts does not pass. The band's power depends on the account boundary, because the band is attached to the account. ```js // run4.mjs — organization-level protection band: what it cuts, what it cannot cut import { ENV, IDENTITY, STRUCTURE, BOUNDARY, ACTUAL, grantedPermission, identityAccount, blastRadius } from './account.mjs'; const separates = (y) => { // does the account boundary separate the environment const h = new Map(); for (const k of IDENTITY) { const a = identityAccount(y, k); h.set(a, new Set([...(h.get(a) ?? []), k.split('@')[1]])); } return [...h.values()].every((s) => s.size === 1); }; const applicable = (y) => BOUNDARY.filter(([, cond]) => separates(y) || ENV.every(cond)); const grantedTotal = IDENTITY.reduce((a, k) => a + grantedPermission(k.split('@')[0]).length, 0); console.log(`granted (identity, permission) entries ${grantedTotal}\n`); console.log('structure env cut % of stopped widest blast'); console.log(' separated entries granted legit path after band'); for (const y of Object.keys(STRUCTURE)) { let cut = 0, stopped = 0; for (const [perm, cond] of applicable(y)) for (const k of IDENTITY) { const [s, e] = k.split('@'); if (!cond(e) || !grantedPermission(s).includes(perm)) continue; cut++; if (ACTUAL[s].includes(perm)) stopped++; } console.log(`${y.padEnd(18)}${(separates(y) ? 'yes' : 'no').padEnd(11)}${String(cut).padStart(5)}` + `${`${Math.round((cut / grantedTotal) * 100)}%`.padStart(11)}${String(stopped).padStart(11)}` + `${String(Math.max(...IDENTITY.map((k) => blastRadius(y, k)))).padStart(18)}`); } console.log('\nwhat each band rule cuts individually (in an environment-separated structure):'); for (const [perm, cond] of BOUNDARY) { const k = IDENTITY.filter((x) => cond(x.split('@')[1]) && grantedPermission(x.split('@')[0]).includes(perm)); console.log(` ${perm.padEnd(18)}${String(k.length).padStart(3)} entries, legit ` + `${k.filter((x) => ACTUAL[x.split('@')[0]].includes(perm)).length}`); } ``` ``` granted (identity, permission) entries 138 structure env cut % of stopped widest blast separated entries granted legit path after band single-account no 3 2% 0 24 by-environment yes 17 12% 6 8 by-workload yes 17 12% 6 3 what each band rule cuts individually (in an environment-separated structure): measurement-delete 6 entries, legit 4 archive-delete 8 entries, legit 2 config-write 3 entries, legit 0 ``` The band carries three rules: delete measurement outside production, delete archive outside production, and write config everywhere. In the two structures that separate environments, all three apply, and they cut seventeen of a hundred thirty-eight entries — twelve percent. In the single account, only the third rule applies: because the account boundary does not separate the environment, the "outside production" condition cannot be attached anywhere, and the cut stays at three. The fourth column gives the band's cost: six of the seventeen cut entries stop a legitimate path. The lower block breaks this down rule by rule — measurement delete cuts six entries, four legitimate; archive delete cuts eight, two legitimate; config write cuts three, none legitimate. The third rule is free, the first two are not. The last column shows what the band cannot cut. Even after the band is applied, the widest blast radius stays at twenty-four, eight, and three. The band cuts permissions, it does not build boundaries: removing a delete permission does not change how many resources a compromised identity can touch. The only thing that changes that is which account the resources sit in. #### Summary - When the same 24 resources and 24 identities are set up in three structures, the widest blast radius goes 24 → 8 → 3; the total `(identity, resource)` entry count drops from 576 to 57. - What policy alone grants is 57 entries in all three structures; the single account adds 519 entries on top, the by-workload structure adds 0. A narrowed policy is only truly narrow if the account boundary backs it up. - Quota is 1,000 fictional units per account: the single account exceeds the limit by 1,142 units; by environment, production exceeds by 530 while development and staging leave 1,388 units idle and non-transferable; by workload, no account exceeds, and the fullest one has 580 units of headroom. - Bill lines go 1 → 3 → 24; a workload's share is 0/24 in the first two structures, directly readable at 24/24 in the third. The cost is the policy count: 24 documents rise to 57, because 33 cross-account accesses arise. - The organization-level protection band cuts 17 of 138 entries (12%) in the structures that separate environments, and stops 6 legitimate paths; in the single account, only 3 entries are cut because the environment condition cannot be attached anywhere. Its power comes not from its own text but from the account structure — and what it cannot cut is the blast radius: even after the band, the widest radius stays 24 / 8 / 3. #### Next Step This topic carried one question all the way through: who can do what. Users and roles were written, policies were evaluated from both directions, authority became temporary, workloads were given identity, granted permission was compared against used permission, and accounts were split apart. But every one of these measures was looking at a single verb: doing. Even after access is narrowed, subjects remain on the list — the blast radius did not reach zero, it reached three. The question that was never asked is this: what does a subject who can reach those three resources see when it reads the data? Is the record sitting on disk in the open, is the request crossing the wire readable, and who holds the key that decodes the record? The next lesson measures not access but what is accessed. ## Cloud Security Protecting data and access in the rented layer: encryption at rest with a key management service and customer keys, end-to-end encryption enforced in transit, rotation and access control in secret storage services, audit logs that track API calls, configuration compliance auditing that catches drift automatically, taking resources off the open internet with private endpoints, threat detection through anomalous-activity alerts, and how compliance requirements map onto the cloud. ### Encryption at Rest Source: https://academia.sh/en/courses/cloud-governance/encryption-at-rest Data at rest is compared across three options: no encryption, provider key, customer key. How many subjects can decrypt, whether key use lands in the audit log, and the overhead the customer key brings are counted with real envelope encryption; on key loss, 3,973 GB across 14 datasets becomes permanently unreadable. The previous topic wrote down who could do what: the subject–action–resource matrix was drawn, policy documents were evaluated, account boundaries were traced. That ledger skipped one question — when someone with access opens the data, **what do they see?** The access list says whose names open the door; it says nothing about what is inside the box behind it. This lesson compares three options for data at rest on a single measure: no encryption, provider key, **customer key**. The measure is **how many subjects can decrypt how many datasets**; two columns sit alongside it — whether key use shows up in the tenant's audit log, and the overhead the customer key brings. Where encrypted secrets are kept and how the key sits there was measured under repository-based deployment; what is measured here is who holds the key when it sits **inside a rented service**. The measurement runs on the **regional measurement network** and is **fictional**. The datasets, subject catalog, and read counts are a **model** written with `node`; the encryption really runs with `node:crypto`, and the key-derivation seeds and content values are made up. **CS1.** Data at rest sits in 14 datasets: three environments, seven types (raw measurement, summary, invoice, archive, backup, snapshot, log archive). Volumes are fictional. **CS2.** The subject catalog is 23 units: 20 people across five teams and three workload identities. **CS3.** Data access and key-use permission are **two separate axes**; under the customer key option, decryption is the intersection of these two sets. There is one key per environment. **CS4.** Whether each subject's job needs plaintext to do its work is recorded: the backup job and the platform team carry encrypted bytes without opening them. **CS5.** Encryption is in envelope form: every dataset has a data key, and the data key is wrapped with the master key. The wrapping and unwrapping really run. **CS6.** Under the customer key, every decrypt produces one audit-log line; under the provider key, key use never lands in the tenant's log. The key rotation period is 365 days (fictional policy). #### Three Options, One Question: Who Can Decrypt ```js // store.mjs — the datasets at rest, subjects, and three encryption options. MODEL. // The regional measurement network is FICTIONAL: team names, dataset names, and read counts are made up. export const ENV = ["development", "staging", "production"]; const BASE = ["raw-measurement", "summary", "invoice"]; const EXTRA = { development: [], staging: ["backup"], production: ["archive", "backup", "snapshot", "log-archive"] }; const GB = { "raw-measurement": 420, summary: 18, invoice: 6, archive: 900, backup: 1340, snapshot: 760, "log-archive": 210 }; const READS = { "raw-measurement": 4, summary: 9, invoice: 6, archive: 1, backup: 0, snapshot: 0, "log-archive": 2 }; // daily plaintext reads per subject export const DATASET = ENV.flatMap((e) => [...BASE, ...EXTRA[e]].map((t) => ({ name: `${e}/${t}`, env: e, type: t, gb: e === "production" ? GB[t] : Math.round(GB[t] / (e === "staging" ? 6 : 20)) }))); // needsPlaintext: does the subject's job require decrypted data. The backup job copies encrypted bytes. export const SUBJECT = [ { name: "measurement team", n: 6, needsPlaintext: true, keyEnv: ["development", "staging"], access: [["*", "raw-measurement"], ["*", "summary"]] }, { name: "billing team", n: 4, needsPlaintext: true, keyEnv: ["development", "staging"], access: [["*", "invoice"]] }, { name: "data team", n: 5, needsPlaintext: true, keyEnv: ["production"], access: [["production", "*"]] }, { name: "platform team", n: 3, needsPlaintext: false, keyEnv: [], access: [["*", "*"]] }, { name: "production on-call", n: 2, needsPlaintext: true, keyEnv: ["production"], access: [["production", "summary"], ["production", "invoice"]] }, { name: "collector identity", n: 1, needsPlaintext: true, keyEnv: ENV, access: [["*", "raw-measurement"]] }, { name: "reporting identity", n: 1, needsPlaintext: true, keyEnv: ["production"], access: [["*", "summary"], ["*", "invoice"]] }, { name: "backup job", n: 1, needsPlaintext: false, keyEnv: [], access: [["*", "backup"], ["*", "archive"], ["*", "snapshot"]] }, ]; export const AGE = { development: 288, staging: 141, production: 322 }; // customer key age export const ROTATION_PERIOD = 365; // fictional rotation period, days const matches = (s, [e, t]) => (e === "*" || s.env === e) && (t === "*" || s.type === t); export const canAccess = (z, k) => z.access.some((c) => matches(k, c)); export const hasKeyAccess = (z, k) => z.keyEnv.includes(k.env); export const SUBJECT_COUNT = SUBJECT.reduce((a, z) => a + z.n, 0); export const reads = (z, k) => (z.needsPlaintext ? z.n * READS[k.type] : 0); // Three options: what condition decides decryption. "complete" = with the missing key grants filled in. export const OPTION = { "no encryption": (z, k) => canAccess(z, k), "provider key": (z, k) => canAccess(z, k), "customer key": (z, k) => canAccess(z, k) && hasKeyAccess(z, k), "customer key (complete)": (z, k) => canAccess(z, k) && (hasKeyAccess(z, k) || z.needsPlaintext), }; export const pairs = (f) => SUBJECT.flatMap((z) => DATASET.filter((k) => f(z, k)).map((k) => ({ z, k }))); export const decrypters = (f) => SUBJECT.filter((z) => DATASET.some((k) => f(z, k))) .reduce((a, z) => a + z.n, 0); if (import.meta.url === `file://${process.argv[1]}`) { const totalGb = DATASET.reduce((a, k) => a + k.gb, 0); console.log(`model: ${DATASET.length} datasets (${totalGb} GB, fictional), ` + `${SUBJECT_COUNT} subjects / ${SUBJECT.length} groups, ${SUBJECT_COUNT * DATASET.length} possible pairs`); const B = [26, 15, 15, 10, 14, 17]; const row = (h) => h.map((v, j) => j ? String(v).padStart(B[j]) : String(v).padEnd(B[0])).join(""); console.log("\n" + row(["option", "can decrypt", "decrypt pairs", "widest", "logs/day", "infra path"])); for (const [name, f] of Object.entries(OPTION)) { const c = pairs(f); const logs = name === "no encryption" || name === "provider key" ? 0 : c.reduce((a, x) => a + reads(x.z, x.k), 0); console.log(row([name, decrypters(f), c.length, Math.max(...SUBJECT.map((z) => DATASET.filter((k) => f(z, k)).length)), logs, name === "no encryption" ? "open, uncounted" : "closed"])); } const open = pairs(OPTION["no encryption"]); const customer = new Set(pairs(OPTION["customer key"]).map((x) => `${x.z.name}|${x.k.name}`)); const closed = open.filter((x) => !customer.has(`${x.z.name}|${x.k.name}`)); console.log(`\npairs closed by moving to customer key: ${closed.length}; ` + `${closed.filter((x) => x.z.needsPlaintext).length} of these are legitimate work needing plaintext`); for (const z of SUBJECT) { const d = closed.filter((x) => x.z.name === z.name); if (d.length) console.log(` ${z.name.padEnd(21)} ${String(d.length).padStart(3)} pairs, ` + `${String(d.reduce((a, x) => a + reads(x.z, x.k), 0)).padStart(3)} daily reads, ` + `needs plaintext: ${z.needsPlaintext}`); } const every = pairs(OPTION["customer key"]).length; console.log(`\npairs no option ever closes: ${every} ` + `(the authorized identity already reads it); once grants are completed ${pairs(OPTION["customer key (complete)"]).length}`); const full = pairs(OPTION["customer key (complete)"]); console.log("\n" + row(["key (env)", "age (days)", "days left", "datasets", "decrypts/day"])); for (const e of ENV) { const k = full.filter((x) => x.k.env === e); console.log(row([e, AGE[e], ROTATION_PERIOD - AGE[e], DATASET.filter((x) => x.env === e).length, k.reduce((a, x) => a + reads(x.z, x.k), 0)])); } } ``` ``` model: 14 datasets (3973 GB, fictional), 23 subjects / 8 groups, 322 possible pairs option can decrypt decrypt pairs widest logs/day infra path no encryption 23 45 14 0 open, uncounted provider key 23 45 14 0 closed customer key 19 20 7 371 closed customer key (complete) 19 27 7 503 closed pairs closed by moving to customer key: 25; 7 of these are legitimate work needing plaintext measurement team 2 pairs, 78 daily reads, needs plaintext: true billing team 1 pairs, 24 daily reads, needs plaintext: true platform team 14 pairs, 0 daily reads, needs plaintext: false reporting identity 4 pairs, 30 daily reads, needs plaintext: true backup job 4 pairs, 0 daily reads, needs plaintext: false pairs no option ever closes: 20 (the authorized identity already reads it); once grants are completed 27 key (env) age (days) days left datasets decrypts/day development 288 77 3 121 staging 141 224 4 121 production 322 43 7 261 ``` The table's first two rows are identical, and the lesson's first finding sits there. Switching to the provider key does not reduce the number of subjects who can decrypt **by even one**: 23 subjects, 45 pairs, the widest subject holding all 14 datasets. Under the provider key, decryption happens automatically alongside data access; for the subject making the read request, encryption is an invisible layer. The one column it does change is the infrastructure path: disk, backup media, decommissioned hardware, and the provider's own operational access all close. That column's "open, uncounted" value is a line item in its own right; how many subjects can read through that path never appears in the tenant's ledger. The customer key row moves the measure for the first time: 23 subjects drop to 19, 45 pairs to 20, the widest grant from 14 datasets to 7. The cost of that sits in the breakdown below. Of the 25 pairs that close, **7 are legitimate work that genuinely needs plaintext**: the measurement team's production raw measurement and summary, the billing team's production invoice, the reporting identity's development and staging datasets — 132 reads a day stop. Once the missing key grants are added, this work reopens and the pair count rises from 20 to 27. Measured against 45, the remaining gap is 18 pairs, and all of them are subjects that do not need plaintext: the platform team's 14 pairs and the backup job's 4. **The customer key's real gain is those 18 pairs** — from 45 to 27, forty percent. The rest ends up turning key permission into a copy of access permission. The last column delivers the second gain: the same 503 decrypts produce 0 log lines under the provider key and 503 under the customer key. Key-use visibility is a line item independent of the narrowing in decrypt authority. The limit table counts key age: the production key is 322 days old, **43 days remain** in the 365-day period, and that rotation concerns 7 datasets. #### The Key's Own Overhead: Rotation, Backup, and Loss ```js // envelope.mjs — envelope encryption really runs: the data key is wrapped, rotated, lost. // KEY AND CONTENT VALUES ARE FICTIONAL. The derivation seed is visible so the output can be reproduced. import { scryptSync, createCipheriv, createDecipheriv, createHash } from "node:crypto"; import { DATASET, pairs, reads, OPTION } from "./store.mjs"; const masterKey = (s) => scryptSync(`fictional-master-passphrase-s${s}`, "fictional-salt-0001", 32, { N: 16384 }); const dataKey = (k) => createHash("sha256").update(`fictional-data-key|${k.name}`).digest(); const iv = (label) => createHash("sha256").update(`iv|${label}`).digest().subarray(0, 12); const content = (k) => Buffer.from(`fictional-record|${k.name}|`.repeat(Math.max(4, k.gb))); const wrap = (k, s) => { // wraps the data key with the master key const c = createCipheriv("aes-256-gcm", masterKey(s), iv(`${k.name}#${s}`)); const g = Buffer.concat([c.update(dataKey(k)), c.final()]); return { dataset: k.name, version: s, body: g, tag: c.getAuthTag() }; }; const unwrap = (wrapped, s) => { // unwraps a wrapped key; wrong key throws const d = createDecipheriv("aes-256-gcm", masterKey(s), iv(`${wrapped.dataset}#${wrapped.version}`)); d.setAuthTag(wrapped.tag); return Buffer.concat([d.update(wrapped.body), d.final()]); }; const encrypt = (k, dk) => { const c = createCipheriv("aes-256-gcm", dk, iv(k.name)); return { body: Buffer.concat([c.update(content(k)), c.final()]), tag: c.getAuthTag() }; }; const decrypt = (k, dk, box) => { const d = createDecipheriv("aes-256-gcm", dk, iv(k.name)); d.setAuthTag(box.tag); return Buffer.concat([d.update(box.body), d.final()]); }; const wrapped = DATASET.map((k) => wrap(k, 1)); const box = DATASET.map((k, i) => encrypt(k, unwrap(wrapped[i], 1))); const objectBytes = box.reduce((a, x) => a + x.body.length, 0); const wrappedBytes = wrapped.reduce((a, x) => a + x.body.length + x.tag.length, 0); const correct = DATASET.every((k, i) => decrypt(k, unwrap(wrapped[i], 1), box[i]).equals(content(k))); console.log(`${DATASET.length} datasets encrypted: ${objectBytes} bytes object, ${wrappedBytes} bytes ` + `wrapped key; all decrypted back correctly: ${correct}`); console.log(`sample wrapped key ${wrapped[0].dataset} -> ${wrapped[0].body.toString("base64")}`); const B = [36, 8, 9, 16]; const row = (h) => h.map((v, j) => j ? String(v).padStart(B[j]) : String(v).padEnd(B[0])).join(""); console.log("\n" + row(["rotation type", "wraps", "objects", "bytes touched"])); const newWrapped = DATASET.map((k) => wrap(k, 2)); // master key 1 -> 2 const opened = DATASET.map((k, i) => decrypt(k, unwrap(newWrapped[i], 2), box[i]).equals(content(k))); console.log(row(["master key (wrap only)", DATASET.length, 0, wrappedBytes])); console.log(row(["data key (re-encrypt)", DATASET.length, DATASET.length, objectBytes + wrappedBytes])); console.log(`objects still open without corruption after the master key rotates: ` + `${opened.every(Boolean)} (${opened.filter(Boolean).length}/${DATASET.length})`); // key loss: wrapped keys remain, master key version 2 is gone. Tried with a fake version. let lostDatasets = 0, lostGb = 0; for (const [i, s] of newWrapped.entries()) { try { decrypt(DATASET[i], unwrap(s, 9), box[i]); } catch { lostDatasets++; lostGb += DATASET[i].gb; } } console.log(`\nmaster key loss: wrapped keys will not open, ${lostDatasets}/${DATASET.length} datasets ` + `unreadable, ${lostGb} GB permanently lost`); const restored = newWrapped.map((s, i) => decrypt(DATASET[i], unwrap(s, 2), box[i]).equals(content(DATASET[i]))); console.log(`restored from the key backup: ${restored.filter(Boolean).length}/` + `${DATASET.length} datasets decrypt correctly — an encrypted-byte backup alone is not enough`); const dailyReads = pairs(OPTION["customer key (complete)"]).reduce((a, x) => a + reads(x.z, x.k), 0); console.log(`\naudit log: ${dailyReads} key uses/day under customer key, ` + `${dailyReads * 30} lines in 30 days; under provider key the same ${dailyReads} decrypts produce 0 lines`); ``` ``` 14 datasets encrypted: 146675 bytes object, 672 bytes wrapped key; all decrypted back correctly: true sample wrapped key development/raw-measurement -> MpZT6WZqv/IAudsxlXHD004z+NRBc5+ITK0DIDlIBdI= rotation type wraps objects bytes touched master key (wrap only) 14 0 672 data key (re-encrypt) 14 14 147347 objects still open without corruption after the master key rotates: true (14/14) master key loss: wrapped keys will not open, 14/14 datasets unreadable, 3973 GB permanently lost restored from the key backup: 14/14 datasets decrypt correctly — an encrypted-byte backup alone is not enough audit log: 503 key uses/day under customer key, 15090 lines in 30 days; under provider key the same 503 decrypts produce 0 lines ``` The encryption really ran: 14 datasets were encrypted, the wrapped keys were unwrapped, and the content came back byte for byte. Rotating the master key touches 14 wrap operations and 672 bytes; **it never touches object bytes.** Rotating the data key re-encrypts all 14 objects in full on top of the same 14 wraps: 147,347 bytes, **roughly 219 times as many.** This is the wrapping layer's reason for existing — a frequently rotated key touches the key, not the data. In exchange, this follows: after the master key rotates, the data key **is still the same data key.** Its wrapped form changed; what it wraps did not. If the data key was left exposed somewhere, rotation does not close that exposure; closing it requires moving to the 147,347-byte row. Cheap rotation and closing rotation are not the same operation, and they sit on separate rows in the table. The loss row is the customer key's harshest line item, and it is a trial that actually ran: with master key version 2 unavailable, **none** of the 14 datasets open, and 3,973 GB becomes permanently unreadable. The encrypted-byte backup was sitting there and did no good; the same 14 datasets decrypted correctly once version 2 came back from the key backup. Moving to the customer key means adding the **key** to the backup plan's scope, not the data. #### The Path Encryption Does Not Close The 20 pairs that stay open under all three options — 27 once grants are completed — are the line item this lesson needs to name outright. These are **reads by authorized identities.** The data team's 5 members open all seven production datasets, because both data access and key permission are concentrated in them. Encryption at rest does nothing against this path, and it is not meant to: the subject has passed authentication, the policy document has granted permission, the key service has decrypted. The path encryption **does** close is a single column: the infrastructure side the tenant cannot read, count, or audit. The provider key closes that column and asks nothing in return. The customer key adds two more things — it separates decrypt authority from access authority (45 pairs down to 27) and drops key use into the tenant's own log (from 0 lines to 503 a day) — and the price it asks is also counted: 7 legitimate jobs stopping, a new secret that needs backing up, and a key that, once lost, makes 3,973 GB unrecoverable. #### Summary - The provider key does not lower the number of subjects who can decrypt from 23, and leaves pairs at 45; the only thing it closes is the infrastructure path the tenant cannot count. - The customer key drops pairs from 45 to 20; once missing grants are completed it rises to 27, and the lasting gain is the 18 pairs of subjects that do not need plaintext. - The cost of narrowing: 7 of the 25 pairs that close are legitimate work needing plaintext, and 132 reads a day stop; the widest subject's grant drops from 14 datasets to 7. - Master key rotation is 14 wraps and 672 bytes, data key rotation is 14 objects and 147,347 bytes; because cheap rotation does not change the data key, it does not close a data key that has been left exposed. - Key loss was demonstrated by actually running it: 14/14 datasets fail to open, 3,973 GB is permanently lost; the encrypted-byte backup is not enough, the key backup restores 14/14 datasets. - Key use shows up only under the customer key: the same 503 decrypts produce 0 lines versus 503 a day, 15,090 lines in 30 days; 43 days remain in the production key's 365-day period. #### Next Step The question for data at rest is answered. But data does not only sit at rest — it keeps moving between datasets, environments, and meter endpoints and the collector. The next lesson measures that movement: how many connection legs are encrypted, how many are left open on an "internal network" rationale, how many connections break when encryption is enforced by policy, and how many configuration items change. ### Encryption in Transit Source: https://academia.sh/en/courses/cloud-governance/encryption-in-transit What happens when transit encryption is enforced is counted: 9 of 16 connection legs are open, 4 have unknown status. Three-stage enforcement changes 23 configuration items and breaks 79 connections; 34 cannot be upgraded. Only one of six paths becomes proven end to end, and reaching all of them needs encryption in its own layer. The question for data at rest is answered. When data moves, though, there is not a single box in front of you — there is a **chain**: a measurement network record travels from the edge device to the intake endpoint, from there to the gateway, to the collector, and to the database; every hop is a separate leg, and every leg has its own status. This lesson's question is not the cost of the handshake — that was measured earlier — nor where encryption terminates; that was measured too. The question here is **enforcement**: which legs are encrypted, which are left open on an "internal network" rationale, how many connections break when policy enforces it, and how many configuration items change. There is a third state as well: whether transport inside the provider's own network is encrypted **cannot be verified by the tenant**. Those legs sit as "unknown," the provider's default is written down as an **assumption**, and not knowing itself counts as a line item. The measurement runs on the **regional measurement network**, is **fictional**, and is a **model** written with `node`. **CS7.** The data path breaks into 16 connection legs; the leg catalog, client counts, configuration items, and request volumes are fictional. **CS8.** Every leg is on one of three network types: public network, dedicated line, provider-internal network. **CS9.** A leg's status is three-valued — encrypted, open, **unknown**. The third value is used only for provider-internal transport and cannot be verified. **CS10.** For every open leg, the number of clients that do not support encryption and how many of them can be upgraded is recorded; a client that cannot be upgraded breaks permanently under enforcement. **CS11.** Enforcement is in three stages: public network first, then dedicated line, then provider-internal network. The transition window is 90 days (fictional), and 61 days have passed in warning mode. **CS12.** That "unknown" legs are encrypted can only be proven by the tenant encrypting in its own layer; the configuration cost of that is recorded per leg. #### Legs, Paths, and Staged Enforcement ```js // leg.mjs — transit legs, paths, and staged enforcement. MODEL. // The regional measurement network is FICTIONAL: the leg catalog, client counts, and request volumes are made up. // status "unknown" = provider-internal transport; the tenant cannot verify it, it stands as an ASSUMPTION. export const LEG = [ // id, name, network, status, clients, unsupported, configItems, requestsK, proofItems, upgradable ["b1", "edge device -> intake endpoint", "public", "open", 340, 54, 3, 4820, 0, 38], ["b2", "intake endpoint -> gateway", "provider-internal", "encrypted", 6, 0, 1, 4820, 0, 0], ["b3", "gateway -> collector", "provider-internal", "unknown", 12, 0, 2, 4820, 4, 0], ["b4", "collector -> database", "provider-internal", "open", 12, 0, 4, 1960, 0, 0], ["b5", "collector -> queue", "provider-internal", "open", 12, 2, 2, 3100, 0, 2], ["b6", "processor -> queue", "provider-internal", "open", 9, 0, 2, 3100, 0, 0], ["b7", "processor -> object store", "provider-internal", "unknown", 9, 0, 1, 640, 3, 0], ["b8", "processor -> cache", "provider-internal", "open", 9, 9, 2, 5400, 0, 0], ["b9", "service -> log collector", "provider-internal", "open", 21, 0, 3, 8900, 0, 0], ["b10", "log collector -> archive", "provider-internal", "unknown", 2, 0, 1, 190, 2, 0], ["b11", "archive -> secondary region", "provider-internal", "unknown", 2, 0, 1, 190, 2, 0], ["b12", "invoice export -> reconciliation", "public", "encrypted", 1, 0, 1, 12, 0, 0], ["b13", "partner feed -> intake", "public", "open", 7, 3, 2, 260, 0, 1], ["b14", "operator -> admin endpoint", "public", "encrypted", 20, 0, 1, 45, 0, 0], ["b15", "admin endpoint -> service", "provider-internal", "open", 20, 0, 3, 45, 0, 0], ["b16", "measurement network maintenance channel", "dedicated-line", "open", 28, 11, 2, 70, 0, 4], ].map(([id, name, net, status, clients, broken, cfg, req, proof, up]) => ({ id, name, net, status, clients, broken, cfg, req, proof, up })); export const PATH = { // the leg sequence a request travels "measurement intake": ["b1", "b2", "b3", "b4"], "summary generation": ["b5", "b6", "b7"], "log flow": ["b9", "b10", "b11"], "invoice export": ["b4", "b7", "b12"], "admin access": ["b14", "b15"], "partner intake": ["b13", "b3", "b4"], }; export const find = (id) => LEG.find((b) => b.id === id); export const NET = ["public", "dedicated-line", "provider-internal"]; // order of enforcement stages export const WINDOW = 90, ELAPSED = 61; // fictional transition window, days const total = (l, f) => l.reduce((a, b) => a + f(b), 0); const B = [28, 7, 11, 10, 13, 13, 15]; export const row = (h) => h.map((v, j) => j ? String(v).padStart(B[j]) : String(v).padEnd(B[0])) .join(""); if (import.meta.url === `file://${process.argv[1]}`) { console.log(`model: ${LEG.length} legs, ${Object.keys(PATH).length} paths, ` + `${total(LEG, (b) => b.req)} thousand requests/day (fictional)`); console.log("\n" + row(["network type", "legs", "encrypted", "open", "unknown", "clients", "requests(k)"])); for (const n of NET) { const l = LEG.filter((b) => b.net === n); const s = (d) => l.filter((b) => b.status === d).length; console.log(row([n, l.length, s("encrypted"), s("open"), s("unknown"), total(l, (b) => b.clients), total(l, (b) => b.req)])); } // staged enforcement: at each stage, the open legs of that network type move to encrypted. const status = new Map(LEG.map((b) => [b.id, b.status])); console.log("\n" + row(["stage", "legs", "cfg items", "broken", "requests(k)", "still open", "unknown"])); const emit = (name, l) => { for (const b of l) status.set(b.id, "encrypted"); console.log(row([name, l.length, total(l, (b) => b.cfg), total(l, (b) => b.broken), total(l, (b) => b.req), [...status.values()].filter((d) => d === "open").length, [...status.values()].filter((d) => d === "unknown").length])); }; emit("0. warning mode", []); for (const n of NET) emit(`${NET.indexOf(n) + 1}. ${n}`, LEG.filter((b) => b.net === n && status.get(b.id) === "open")); // per path, end to end: a path counts as proven only if every one of its legs is encrypted. const pathRow = (name, d) => { const l = PATH[name].map(find); const count = (x) => l.filter((b) => d(b) === x).length; console.log(row([name, l.length, count("encrypted"), count("open"), count("unknown"), count("unknown") === 0 && count("open") === 0 ? "proven" : "no", total(l.filter((b) => d(b) !== "encrypted"), (b) => b.req)])); }; console.log("\n" + row(["path (before enforcement)", "legs", "encrypted", "open", "unknown", "end to end", "unproven(k)"])); for (const name of Object.keys(PATH)) pathRow(name, (b) => b.status); console.log("\n" + row(["path (after three stages)", "legs", "encrypted", "open", "unknown", "end to end", "unproven(k)"])); for (const name of Object.keys(PATH)) pathRow(name, (b) => status.get(b.id)); } ``` ``` model: 16 legs, 6 paths, 38372 thousand requests/day (fictional) network type legs encrypted open unknown clients requests(k) public 4 2 2 0 368 5137 dedicated-line 1 0 1 0 28 70 provider-internal 11 1 6 4 114 33165 stage legs cfg items broken requests(k) still open unknown 0. warning mode 0 0 0 0 9 4 1. public 2 5 57 5080 7 4 2. dedicated-line 1 2 11 70 6 4 3. provider-internal 6 16 11 22505 0 4 path (before enforcement) legs encrypted open unknown end to end unproven(k) measurement intake 4 1 2 1 no 11600 summary generation 3 0 2 1 no 6840 log flow 3 0 1 2 no 9280 invoice export 3 1 1 1 no 2600 admin access 2 1 1 0 no 45 partner intake 3 0 2 1 no 7040 path (after three stages) legs encrypted open unknown end to end unproven(k) measurement intake 4 3 0 1 no 4820 summary generation 3 2 0 1 no 640 log flow 3 1 0 2 no 380 invoice export 3 2 0 1 no 640 admin access 2 2 0 0 proven 0 partner intake 3 2 0 1 no 4820 ``` The first table gives away a pattern immediately: 2 of the 4 legs on the public network are encrypted, only 1 of 11 provider-internal legs. Encryption has been applied to the outward-facing side and not applied on the inside — the rationale is written down: "internal network." The volume that rationale carries is counted: 33,165 thousand requests a day pass through provider-internal legs, **eighty-six percent of the total.** #### The Enforcement Ledger The stage table splits the bill into three. The public-network stage is cheap — 2 legs, 5 configuration items — but it **breaks 57 connections**, because 54 edge devices and 3 clients of the partner feed do not support encrypted transport. The dedicated-line stage is 1 leg, 2 items, 11 broken. The provider-internal stage is the most expensive: 6 legs, 16 items, and the path for 22,505 thousand requests a day; yet it breaks only 11 connections. The distribution runs against intuition — the stage that changes the most configuration is the stage that breaks the fewest connections. The path tables build the real sentence. Before enforcement, **none of the six paths are encrypted end to end.** After three stages, only one becomes proven: admin access, because both legs it passes through can be verified. The remaining five paths cannot be proven because each carries at least one "unknown" leg. Unproven volume drops from 11,600 to 4,820 thousand requests but does not hit zero; the reason is a single leg — the hop between the gateway and the collector, a shared part of both measurement intake and partner intake. ```js // ledger.mjs — the enforcement ledger: broken connections, permanent breaks, exceptions, and proof cost. import { LEG, PATH, WINDOW, ELAPSED } from "./leg.mjs"; const gen = (B) => (h) => h.map((v, j) => j ? String(v).padStart(B[j]) : String(v).padEnd(B[0])) .join(""); const row = gen([41, 9, 13, 10, 11, 11, 13]); // broken-connection table const dr = gen([27, 10, 17, 13, 13, 13, 15]); // decision table const LIMIT = 2; // fictional policy: permanent exception cap const total = (l, f) => l.reduce((a, b) => a + f(b), 0); const OPEN = LEG.filter((b) => b.status === "open"); const UNKNOWN = LEG.filter((b) => b.status === "unknown"); const permanent = (b) => b.broken - b.up; console.log(row(["broken leg", "clients", "unsupported", "upgrade", "permanent", "cfg items", "requests(k)"])); for (const b of OPEN.filter((x) => x.broken > 0)) console.log(row([b.name, b.clients, b.broken, b.up, permanent(b), b.cfg, b.req])); console.log(row(["total", total(OPEN, (b) => b.clients), total(OPEN, (b) => b.broken), total(OPEN, (b) => b.up), total(OPEN, permanent), total(OPEN, (b) => b.cfg), total(OPEN, (b) => b.req)])); // A decision: which legs get enforced, which get encrypted in their own layer, is a proxy set up. const evaluate = (enforced, own, proxy) => { const d = new Map(LEG.map((b) => [b.id, b.status])); let broken = 0, cfg = 0; for (const b of LEG) { if (enforced.has(b.id)) { d.set(b.id, "encrypted"); cfg += b.cfg; broken += proxy ? 0 : permanent(b); } if (own.has(b.id)) { d.set(b.id, "encrypted"); cfg += b.proof; } } if (proxy) cfg += 3 * LEG.filter((b) => enforced.has(b.id) && permanent(b) > 0).length; const proven = Object.values(PATH) .filter((y) => y.every((id) => d.get(id) === "encrypted")).length; const unproven = total(LEG.filter((b) => d.get(b.id) !== "encrypted"), (b) => b.req); return { broken, cfg, proven, unproven, exempt: LEG.filter((b) => b.status === "open" && !enforced.has(b.id)).length }; }; const ids = (l) => new Set(l.map((b) => b.id)); const none = new Set(); const permanentBreakers = OPEN.filter((b) => permanent(b) > 0); // legs that produce a permanent break const DECISION = { "0 warning mode": [none, none, false], "1 enforce, no exception": [ids(OPEN), none, false], "2 enforce + proxy layer": [ids(OPEN), none, true], "3 proxy + own encryption": [ids(OPEN), ids(UNKNOWN), true], "4 enforce + exception": [ids(OPEN.filter((b) => permanent(b) === 0)), none, false], }; console.log("\n" + dr(["decision", "enforced", "permanent break", "exempt open", "proven path", "extra cfg", "unproven(k)"])); for (const [name, [z, k, v]] of Object.entries(DECISION)) { const s = evaluate(z, k, v); console.log(dr([name, z.size, s.broken, s.exempt, `${s.proven}/${Object.keys(PATH).length}`, s.cfg, s.unproven])); } const openReq = total(OPEN, (b) => b.req); console.log(`\ntransition window: ${WINDOW} days, ${ELAPSED} days elapsed, ${WINDOW - ELAPSED} days left; ` + `${openReq * ELAPSED} thousand requests passed through open legs during the warning period`); console.log(`permanent exception cap ${LIMIT} legs (fictional policy); requested ${permanentBreakers.length}, ` + `${permanentBreakers.length - LIMIT} legs beyond the cap`); console.log(`client upgrades: ${total(OPEN, (b) => b.up)} clients, ` + `${OPEN.filter((b) => b.up > 0).length} legs; the remaining ${total(OPEN, permanent)} clients ` + `cannot be upgraded`); console.log(`\nunknown item: ${UNKNOWN.length} legs, ` + `${total(UNKNOWN, (b) => b.req)} thousand requests/day, proof cost ` + `${total(UNKNOWN, (b) => b.proof)} config items`); for (const b of UNKNOWN) console.log(` ${b.name.padEnd(28)} ${String(b.req).padStart(6)} k/day, ` + `proof ${b.proof} items, assumption: provider-internal transport is encrypted`); ``` ``` broken leg clients unsupported upgrade permanent cfg items requests(k) edge device -> intake endpoint 340 54 38 16 3 4820 collector -> queue 12 2 2 0 2 3100 processor -> cache 9 9 0 9 2 5400 partner feed -> intake 7 3 1 2 2 260 measurement network maintenance channel 28 11 4 7 2 70 total 458 79 45 34 23 27655 decision enforced permanent break exempt open proven path extra cfg unproven(k) 0 warning mode 0 0 9 0/6 0 33495 1 enforce, no exception 9 34 0 1/6 23 5840 2 enforce + proxy layer 9 0 0 1/6 35 5840 3 proxy + own encryption 9 0 0 6/6 46 0 4 enforce + exception 5 0 4 1/6 14 16390 transition window: 90 days, 61 days elapsed, 29 days left; 1686955 thousand requests passed through open legs during the warning period permanent exception cap 2 legs (fictional policy); requested 4, 2 legs beyond the cap client upgrades: 45 clients, 4 legs; the remaining 34 clients cannot be upgraded unknown item: 4 legs, 5840 thousand requests/day, proof cost 11 config items gateway -> collector 4820 k/day, proof 4 items, assumption: provider-internal transport is encrypted processor -> object store 640 k/day, proof 3 items, assumption: provider-internal transport is encrypted log collector -> archive 190 k/day, proof 2 items, assumption: provider-internal transport is encrypted archive -> secondary region 190 k/day, proof 2 items, assumption: provider-internal transport is encrypted ``` The broken-connection table sets one number apart: **79 clients do not support encrypted transport**, 45 can be upgraded, and **34 cannot**. The harshest row is the cache leg: 9 of 9 clients do not support it, none can be upgraded, and that leg carries 5,400 thousand requests a day. Second is the edge devices: 16 of 54 devices are hardware sitting in the field whose software will not change. The decision table lays five approaches side by side. Warning mode breaks nothing and proves nothing; over 61 days, 1,686,955 thousand requests passed through open legs and that cannot be closed retroactively. Enforcing without exception changes 23 items and permanently severs 34 connections. The proxy layer zeroes out breakage and adds 12 items: 35. None of these three decisions pushes the proven path count above 1/6; the one row that does is the fourth — encrypting "unknown" legs in the tenant's own layer adds 11 more items, for a total of 46, and **6/6 paths become proven.** The last decision shows the opposite: exempting the 4 legs that produce a permanent break cuts the load to 14 items and breaks no connections, but pushes unproven volume up to 16,390 thousand requests a day. Fictional policy recognizes 2 permanent exceptions; 4 are requested, and **2 legs fall outside the cap.** Of the transition window's 90 days, 61 have passed — **29 days remain.** Next to each of the four "unknown" legs sits the same sentence: the assumption is that provider-internal transport is encrypted. They carry 5,840 thousand requests a day, and their status cannot be read from any configuration the tenant holds. Being unknown is not the same as being open, but it is not the same as being provable either. #### Summary - 3 of the 16 legs are encrypted, 9 are open, 4 have unknown status; only 1 of 11 provider-internal legs is encrypted, and 33,165 thousand requests a day pass through those legs. - Three-stage enforcement changes 23 items; the most expensive stage (provider-internal, 16 items) breaks the fewest connections (11), and the cheapest stage breaks 57. - 79 clients do not support encrypted transport; 45 can be upgraded, 34 cannot, and the proxy layer adds 12 items while saving those 34. - Before enforcement, no path is encrypted end to end; after three stages it is 1/6, and once encryption is added in the tenant's own layer it is 6/6, with unproven volume dropping from 33,495 to zero. - The exemption path cuts the load to 14 items and pushes unproven volume up to 16,390 thousand requests/day; fictional policy recognizes 2 exceptions, and 2 legs fall outside the cap. 29 days remain in the transition window. #### Next Step Data has been made encrypted both at rest and in transit. Both rest on the same thing: a key or credential is stored somewhere, and services read it. The next lesson returns to that storage layer and measures a single question: when a secret rotates, how large should the **window** be where the old and new values are valid together — how many clients does a short window miss, and how much does a long window extend the leak duration. ### Secret Storage Services Source: https://academia.sh/en/courses/cloud-governance/secret-storage-services The measure of secret rotation is the window: the span where the old and new values are valid together is scanned. A short window misses 44 clients, and the window that misses zero runs 3,624 minutes and leaves an old secret standing for 601,584 minutes a year. Fixing just the 18 clients that read only at startup cuts the window to 227 minutes; 17 of 41 secrets have never been rotated. When encryption at rest and in transit were set up, both rested on the same thing: a key or credential sits somewhere, and services read it. That somewhere is a **secret storage service**, and like every rented layer it hands off a decision. The decision this lesson measures collapses into a single number: the **rotation window**. When a secret rotates, the old value does not become invalid instantly; for a span, the old and new values are both accepted, and clients pick up the new value during that span. Once the window closes, the old value is rejected, and every client that has not read the new value by then breaks. The window is measured in two directions at once, and the two directions point opposite ways. A short window produces **missed clients**; a long window extends the time the **old secret stays valid** — a leaked value keeps working for that whole span. Which paths secrets leak through — log lines, environment variables, build-output layers — was scanned earlier and is not repeated here; this lesson's subject is rotation itself. The measurement runs on the **regional measurement network**, is **fictional**, and is a **model** written with `node`. **No real secret value exists in the model**: only secret names, client counts, and times are carried. **CS13.** The secret storage service holds 41 secrets: three environments, five services, five secret types, and three organization-wide secrets. **CS14.** There are 83 client instances reading secrets. Read mode is three-valued: reads every request, reads cached (cache lifetime 240 minutes), reads only at startup (rollout interval 4,320 minutes). **CS15.** The rotation window is the span where the old and new secret are valid together; a client that has not picked up the new value before the window ends counts as **missed** and breaks. **CS16.** Every client instance makes 12 requests a minute (fictional unit). **CS17.** The rotation period is 90 days (fictional policy). Some secrets are rotated automatically, some manually, and some never. **CS18.** Randomness comes from a self-written linear generator; the seed is 20260415 and appears in the output. Human reads in the last 90 days are counted in the model. #### Scanning the Window ```js // secret.mjs — secret storage service, client inventory, and rotation-window scan. MODEL. // The regional measurement network is FICTIONAL. NO REAL SECRET EXISTS: only secret NAMES and counts are modeled. // Randomness comes from a self-written linear generator; the seed is visible below. export const SEED = 20260415; export const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const ENV = ["development", "staging", "production"]; const CATALOG = [ ["collector", ["database-password", "intake-key"]], ["validator", ["database-password", "signing-key"]], ["billing", ["database-password", "external-endpoint-token", "signing-key"]], ["work-order", ["database-password", "external-endpoint-token"]], ["archivist", ["database-password", "maintenance-identity"]], ]; export const SECRET = [ ...ENV.flatMap((e) => CATALOG.flatMap(([s, tl]) => tl.map((t) => ({ name: `${e}/${s}/${t}`, env: e, service: s, type: t })))), ...CATALOG.map(([s]) => ({ name: `production/${s}/maintenance-identity`, env: "production", service: s, type: "maintenance-identity" })), ...["measurement-network-maintenance", "secondary-region-copy", "partner-feed"].map((a) => ({ name: `org/${a}`, env: "org", service: "shared", type: "external-endpoint-token" })), ]; // Client instances: the processes reading each secret. Read mode determines pickup time. const r = generator(SEED); export const CLIENT = []; for (const s of SECRET) { const n = 1 + Math.floor(r() * ({ production: 6, org: 3, staging: 2, development: 2 })[s.env]); for (let i = 0; i < n; i++) { const u = r(); CLIENT.push({ secret: s.name, env: s.env, phase: r(), mode: u < 0.45 ? "cache" : u < 0.75 ? "every-request" : "startup" }); } } for (const s of SECRET) { // rotation history and human reads s.auto = r() < 0.55; s.age = 40 + Math.floor(r() * 860); // days since the secret was created s.last = s.auto ? Math.floor(r() * 90) : (r() < 0.45 ? Math.floor(r() * 420) : null); s.human = Math.floor(r() * 4); // human reads in the last 90 days } export const REQ_PER_MIN = 12; // requests/min per client instance (fictional) export const PROFILE = { // the client side's pickup behavior "current state": { cache: 240, rollout: 4320, changed: [] }, "cache 60 min": { cache: 60, rollout: 4320, changed: ["cache"] }, "startup readers": { cache: 240, rollout: 60, changed: ["startup"] }, "everyone re-reads": { cache: 60, rollout: 60, changed: ["cache", "startup"] }, }; export const pickup = (c, p) => c.mode === "every-request" ? 1 : Math.ceil(c.phase * (c.mode === "cache" ? p.cache : p.rollout)); export const missed = (p, W) => CLIENT.filter((c) => pickup(c, p) > W); export const PERIOD = 90, YEAR = 365; // fictional rotation period, days const B = [13, 20, 18, 18, 24]; export const row = (h) => h.map((v, j) => j ? String(v).padStart(B[j]) : String(v).padEnd(B[0])) .join(""); if (import.meta.url === `file://${process.argv[1]}`) { const modeCount = (m) => CLIENT.filter((c) => c.mode === m).length; console.log(`model (seed ${SEED}): ${SECRET.length} secrets, ${CLIENT.length} client instances — ` + `${modeCount("every-request")} read every request, ${modeCount("cache")} cached, ` + `${modeCount("startup")} read only at startup`); const p = PROFILE["current state"]; console.log(`\ncurrent state: cache lifetime ${p.cache} min, rollout interval ${p.rollout} min; ` + `latest pickup ${Math.max(...CLIENT.map((c) => pickup(c, p)))} min`); console.log("\n" + row(["window(min)", "missed clients", "secrets affected", "broken req/min", "yearly exposure (min)"])); const yearly = Math.round((YEAR / PERIOD) * SECRET.length); // total rotations per year for (const W of [0, 1, 30, 60, 120, 240, 720, 1440, 2880, 3624]) { const k = missed(p, W); console.log(row([W, k.length, new Set(k.map((c) => c.secret)).size, k.length * REQ_PER_MIN, W * yearly])); } } ``` ``` model (seed 20260415): 41 secrets, 83 client instances — 27 read every request, 38 cached, 18 read only at startup current state: cache lifetime 240 min, rollout interval 4320 min; latest pickup 3624 min window(min) missed clients secrets affected broken req/min yearly exposure (min) 0 83 40 996 0 1 56 34 672 166 30 53 32 636 4980 60 44 30 528 9960 120 38 29 456 19920 240 18 17 216 39840 720 14 14 168 119520 1440 10 10 120 239040 2880 3 3 36 478080 3624 0 0 0 601584 ``` The two ends of the table frame the decision. At the top, the window is zero: the old value is rejected instantly, **all 83 client instances miss**, and 996 requests a minute drop. At the bottom, the window is 3,624 minutes — sixty hours — and **no client misses**. Between the two is not a choice; it is a trade-off curve. The middle rows show the shape of the trade-off. A one-hour window misses 44 clients: it affects 30 secrets and cuts 528 requests a minute. Four hours (240 minutes) drops the missed count to 18, because the cache lifetime is 240 minutes and at that boundary every cached client has already refreshed the value. The remaining 18 are clients that read only at startup, and they do not read at four hours — only at the next rollout. Even a one-day window still misses 10. The last column counts the cost in the opposite direction. 41 secrets, on a 90-day period, rotate 166 times a year; on every rotation the old value stays valid for the length of the window. At the zero-miss window, the yearly total comes to **601,584 minutes** — equivalent to 417 days, meaning that at any moment in the year, an average of 1.14 secrets are still also working with their old value. A leaked secret is accepted for that whole span regardless of when the leak is noticed. At a one-hour window, the same total drops to 9,960 minutes — one sixtieth. Neither number is acceptable: not the 44 broken clients, and not the old secret staying open for 417 days a year. The window is not a knob to turn on its own; **no window value is good unless the client side changes.** #### The Client Side and Secrets That Are Never Rotated ```js // rotation.mjs — the client side of auto-rotation, never-rotated secrets, and read surface. import { SECRET, CLIENT, PROFILE, pickup, missed, REQ_PER_MIN, PERIOD } from "./secret.mjs"; const gen = (B) => (h) => h.map((v, j) => j ? String(v).padStart(B[j]) : String(v).padEnd(B[0])) .join(""); const rowP = gen([21, 10, 14, 24, 15, 17]); const median = (l) => l.length ? [...l].sort((a, b) => a - b)[Math.floor(l.length / 2)] : 0; console.log(rowP(["profile", "changing", "window(min)", "yearly exposure (min)", "60 min missed", "60 min req/min"])); for (const [name, p] of Object.entries(PROFILE)) { const latest = Math.max(...CLIENT.map((c) => pickup(c, p))); const changing = CLIENT.filter((c) => p.changed.includes(c.mode)).length; const m60 = missed(p, 60).length; console.log(rowP([name, changing, latest, latest * Math.round((365 / PERIOD) * SECRET.length), m60, m60 * REQ_PER_MIN])); } const never = SECRET.filter((s) => s.last === null); const auto = SECRET.filter((s) => s.auto); const manual = SECRET.filter((s) => s.last !== null && !s.auto); const rowS = gen([24, 9, 15, 12, 20]); console.log("\n" + rowS(["rotation class", "secrets", "latest (days)", "median", "left/over period"])); const emit = (name, l, field) => { const v = l.map((s) => s[field]); const latest = Math.max(...v, 0); console.log(rowS([name, l.length, latest, median(v), latest <= PERIOD ? `${PERIOD - latest} days left` : `${latest - PERIOD} days over`])); }; emit("auto-rotated", auto, "last"); emit("manually rotated", manual, "last"); emit("never rotated", never, "age"); for (const s of [...never].sort((a, b) => b.age - a.age).slice(0, 4)) console.log(` ${s.name.padEnd(38)} ${String(s.age).padStart(4)} days, ` + `${CLIENT.filter((c) => c.secret === s.name).length} clients, ` + `${s.human} human reads in the last 90 days`); const humanTouched = SECRET.filter((s) => s.human > 0); const intersection = humanTouched.filter((s) => s.last === null); console.log(`\nread surface: ${humanTouched.length}/${SECRET.length} secrets were read by a human in ` + `the last 90 days (${SECRET.reduce((a, s) => a + s.human, 0)} reads, all in the audit log)`); console.log(`${intersection.length} of these are never-rotated secrets; ` + `${CLIENT.filter((c) => intersection.some((s) => s.name === c.secret)).length} client instances ` + `read these ${intersection.length} secrets`); const widest = SECRET.map((s) => [s.name, CLIENT.filter((c) => c.secret === s.name).length]) .sort((a, b) => b[1] - a[1])[0]; console.log(`widest secret: ${widest[0]} — ${widest[1]} client instances; ` + `when it rotates, all ${widest[1]} of them must pick up the new value`); ``` ``` profile changing window(min) yearly exposure (min) 60 min missed 60 min req/min current state 0 3624 601584 44 528 cache 60 min 38 3624 601584 18 216 startup readers 18 227 37682 26 312 everyone re-reads 56 57 9462 0 0 rotation class secrets latest (days) median left/over period auto-rotated 16 88 64 2 days left manually rotated 8 371 209 281 days over never rotated 17 826 442 736 days over production/work-order/maintenance-identity 826 days, 5 clients, 1 human reads in the last 90 days staging/archivist/maintenance-identity 789 days, 1 clients, 1 human reads in the last 90 days org/partner-feed 760 days, 1 clients, 0 human reads in the last 90 days production/validator/signing-key 748 days, 6 clients, 2 human reads in the last 90 days read surface: 34/41 secrets were read by a human in the last 90 days (68 reads, all in the audit log) 15 of these are never-rotated secrets; 34 client instances read these 15 secrets widest secret: production/collector/database-password — 6 client instances; when it rotates, all 6 of them must pick up the new value ``` The profile table delivers this lesson's most counterintuitive result. Pulling the cache lifetime from 240 minutes down to 60 **touches 38 clients and does not shorten the window at all**: the zero-miss window stays at 3,624 minutes, and the yearly exposure stays at 601,584 minutes. The only thing it gains is dropping the one-hour-window miss count from 44 to 18. The reason is plain: the binding constraint is not the cache — it is the **18 clients that read only at startup**; they never listen to the cache lifetime at all. The third row confirms this. Making just those 18 clients re-readable — with less than half the touches — drops the window from 3,624 minutes to **227 minutes**, and the yearly exposure from 601,584 to 37,682 minutes: one sixteenth. The fourth row combines both: 56 clients change, the window drops to 57 minutes, the yearly exposure becomes 9,462 minutes, and the one-hour-window miss count is **zero**. This is what auto-rotation requires on the client side — the ability to re-read a secret comes first, cache lifetime second. Without re-reading, rotation waits for a rollout every single time. The change itself is also a line item and sits in the first column of the table. Adding re-read capability asks for a code change in 18 clients; shortening the cache lifetime asks for a configuration change in 38. Both together make 56 instances, and each of those 56 instances is a separate deployment item. Sequencing matters for this reason: the 18-client work gains sixteen times more than the 38-client work, and done alone it drops the window below four hours. If nothing is done, the rotation period does not function — that is the technical counterpart of 17 secrets having never been rotated. The lower table is the inventory face of the same problem. 16 of the 41 secrets are auto-rotated, and the most overdue one has **2 days left** in its 90-day period. 8 secrets are manually rotated, and the most overdue one has run **281 days past** the period. The remaining **17 secrets have never been rotated** — forty percent of the inventory — and the oldest is 826 days old. Measuring this did not need a separate scan: records with an empty last-rotation field were counted. The last three lines set the order. 34 of the 41 secrets were read by a human at least once in the last 90 days — 68 reads in total, all sitting in the audit log. **15 of those 34 secrets are never-rotated secrets**, and 34 client instances read those 15. The intersection gives the rotation order: a secret that a human has touched **and** has never been rotated comes before the rest of the inventory. The widest secret is the production collector's database password; 6 client instances read it, and when it rotates, all six need to pick up the new value — the window decision is made precisely for those six instances. #### Summary - The rotation window was measured in two directions: at a zero window all 83 clients miss, at a 3,624-minute window none miss; at a one-hour window it is 44 clients and 528 requests a minute. - The cost of a long window was counted: 41 secrets rotate 166 times a year, and a 3,624-minute window leaves an old secret standing for 601,584 minutes a year; at a one-hour window the same total is 9,960 minutes. - Cutting the cache lifetime to a quarter touches 38 clients and does not shorten the zero-miss window at all; the binding constraint is the 18 clients that read only at startup. - Making those 18 clients re-readable drops the window to 227 minutes, and both together to 57 minutes; in the second case, the one-hour-window miss count is zero. - 16 of the 41 secrets are auto-rotated (2 days left in the period), 8 are manual (281 days over), and 17 have never been rotated; the oldest is 826 days old. - 34 secrets were read by a human in the last 90 days (68 reads, all in the log), and 15 of them are never-rotated secrets; the rotation order comes from that intersection. #### Next Step Encryption is in place, secrets have become rotatable, and the same sentence appeared in every measurement: "all in the audit log." That sentence has not been tested yet. The next lesson measures the record itself: a set of events is dropped into the audit log, and how many questions it can answer is counted — who, when, which resource, from which identity. The operation type the record does not cover is named, who can delete the record is counted, and retention period and cost are scanned together. ### Audit Logs Source: https://academia.sh/en/courses/cloud-governance/audit-logs A set of events is dropped into the audit log, and how many of four questions it can answer is counted: 273 of 480 events answer all four, 88 in the default configuration. 3 of 8 investigations get answered. 11 subjects can delete the log; moving it to a separate account stops 6 legitimate jobs, and data-plane logging costs 772 times as much as management-plane logging. The same sentence appeared repeatedly across the previous three lessons: "it all lands in the audit log." Key use landed in the log, secret reads landed in the log, policy changes landed in the log. That sentence has never been tested until now. This lesson tests it and reduces the measure to a single form: a set of events is dropped into the audit log, and **how many questions can be answered** is counted. There are four questions — who, when, which resource, from which identity. There is one operation type the log does not cover, and it has to be named: **data-plane reads and writes.** Management-plane calls (policy changes, key use, resource creation) are logged by default; object reads, object writes, and queue consumption are not. That logging is turned on separately and **billed separately.** The second subject is the log itself: an audit log is also a surface — who can delete it, where it is written, how immutability is achieved. The measurement runs on the **regional measurement network**, is **fictional**, and is a **model** written with `node`. The currency is fictional too: the "u" unit is made up, not a real price. **CS19.** The event sample is 480 operations; daily volume is 2,400 events on the management plane and 1,860,000 on the data plane (fictional). **CS20.** Management-plane logging is on by default; data-plane logging is turned on separately and billed separately. **CS21.** An event's identity is one of four types: user, assumed-role user, workload identity, automated service. Under role assumption, the single subject can only be read if a session tag is present. **CS22.** Batch operations do not name a resource individually; the resource field fills with the container's name, and the "which resource" question goes unanswered. **CS23.** For each destination the log can be written to, the model records the number of subjects who can delete it, modify it, and read it, along with the immutability lock. **CS24.** Cost is calculated in the fictional unit: ingesting a thousand events is 0.4 u, GB-month storage is 12 u, event size is 1.4 KB. Sampled logging does not answer single-resource questions. The generator is self-written; the seed is 20260722. #### How Many Questions Get Answered ```js // event.mjs — the events that land in the audit log and the questions that can be answered. MODEL. // The regional measurement network is FICTIONAL. The generator is self-written; the seed appears below. export const SEED = 20260722; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; export const MANAGEMENT = ["policy-change", "key-use", "assume-role", "network-rule-change", "secret-read", "resource-create"]; export const DATA = ["object-read", "object-write", "object-delete", "queue-consume"]; export const DAILY = { management: 2400, data: 1860000 }; // daily event volume (fictional) const r = generator(SEED); export const EVENT = Array.from({ length: 480 }, () => { const isData = r() < 0.72; const op = isData ? DATA[Math.floor(r() * DATA.length)] : MANAGEMENT[Math.floor(r() * MANAGEMENT.length)]; const u = r(); const identity = u < 0.18 ? "user" : u < 0.52 ? "assumed-role" : u < 0.86 ? "workload" : "automated-service"; return { plane: isData ? "data" : "management", op, identity, tag: identity === "assumed-role" ? r() < 0.38 : true, // session tag gives the single subject detail: r() > (isData ? 0.34 : 0.12) }; // batch operations do not name a resource }); // Four questions: which of them does an event sitting in the log answer. export const QUESTION = { "when": () => true, "from which identity": () => true, "which resource": (o) => o.detail, "who (single subject)": (o) => o.identity !== "assumed-role" || o.tag, }; export const logged = (o, dataOn) => o.plane === "management" || dataOn; // Investigation: a question is tied to an operation, a field, and a lookback period. export const INVESTIGATION = [ ["who changed the production policy document", "policy-change", "who (single subject)", 12], ["which objects were read from the archive", "object-read", "which resource", 5], ["which identity used the customer key", "key-use", "from which identity", 40], ["who deleted the measurement record", "object-delete", "who (single subject)", 3], ["when was the network rule loosened", "network-rule-change", "when", 210], ["which secret did the maintenance identity read", "secret-read", "which resource", 60], ["who read the partner token", "secret-read", "who (single subject)", 150], ["which resource did the backup write to", "object-write", "which resource", 400], ]; export const answer = (s, dataOn, retention) => { const [, op, field, days] = s; const l = EVENT.filter((o) => o.op === op && logged(o, dataOn) && QUESTION[field](o)); if (days > retention) return ["no", `retention ${retention} days < ${days}`]; if (l.length === 0) { const off = EVENT.some((o) => o.op === op) && DATA.includes(op) && !dataOn; return ["no", off ? "data-plane logging off" : `${field} field empty`]; } return ["yes", `${l.length} events`]; }; const gen = (B) => (h) => h.map((v, j) => j ? String(v).padStart(B[j]) : String(v).padEnd(B[0])) .join(""); const row = gen([24, 12, 16, 18, 12]); if (import.meta.url === `file://${process.argv[1]}`) { const m = EVENT.filter((o) => o.plane === "management").length; console.log(`model (seed ${SEED}): ${EVENT.length} event sample — ${m} management plane, ` + `${EVENT.length - m} data plane; daily volume ${DAILY.management} / ${DAILY.data} (fictional)`); console.log("\n" + row(["question", "default", "data plane too", "left unlogged", "share"])); for (const [name, f] of Object.entries(QUESTION)) { const v = EVENT.filter((o) => logged(o, false) && f(o)).length; const t = EVENT.filter((o) => logged(o, true) && f(o)).length; console.log(row([name, v, t, EVENT.length - t, `${Math.round(100 * t / EVENT.length)}%`])); } const allFour = (dataOn) => EVENT.filter((o) => logged(o, dataOn) && Object.values(QUESTION).every((f) => f(o))).length; console.log(row(["all four", allFour(false), allFour(true), EVENT.length - allFour(true), `${Math.round(100 * allFour(true) / EVENT.length)}%`])); for (const [dataOn, retention, name] of [[false, 90, "default (management, 90 days)"], [true, 90, "data plane on (90 days)"], [true, 365, "data plane on (365 days)"]]) { const s = INVESTIGATION.map((x) => answer(x, dataOn, retention)); console.log(`\n${name}: ${s.filter((x) => x[0] === "yes").length}/${INVESTIGATION.length} ` + `questions answered`); if (name.startsWith("default")) for (const [i, x] of INVESTIGATION.entries()) console.log(` ${x[0].padEnd(48)} ${s[i][0].padEnd(6)} ${s[i][1]}`); } } ``` ``` model (seed 20260722): 480 event sample — 132 management plane, 348 data plane; daily volume 2400 / 1860000 (fictional) question default data plane too left unlogged share when 132 480 0 100% from which identity 132 480 0 100% which resource 119 352 128 73% who (single subject) 99 374 106 78% all four 88 273 207 57% default (management, 90 days): 3/8 questions answered who changed the production policy document yes 16 events which objects were read from the archive no data-plane logging off which identity used the customer key yes 17 events who deleted the measurement record no data-plane logging off when was the network rule loosened no retention 90 days < 210 which secret did the maintenance identity read yes 15 events who read the partner token no retention 90 days < 150 which resource did the backup write to no retention 90 days < 400 data plane on (90 days): 5/8 questions answered data plane on (365 days): 7/8 questions answered ``` The first column is the default configuration's coverage, and 348 of the 480 events are not in that column at all — data-plane logging is off. The "when" and "from which identity" questions are answered for all of the remaining 132 events; the log always carries these two fields. The other two do not. Once the data plane is also turned on, "which resource" is answered for 352 of the 480 events: the remaining 128 events are batch operations, and the resource field names the container, not the individual objects. "Who" is answered for 374; the missing 106 events are role-assumed calls that do not carry a session tag. The log has the role's name, not the name of the person who assumed it — if the role is shared, the question stops there. The number of events that answer all four questions at once is 273, or 57 percent. In the default configuration, the same number is 88: 18 percent of the 480 events. This is the measured counterpart of the sentence "it all lands in the audit log." The investigation list delivers the same result with concrete questions: three of the eight get answered. The failures fall into two classes, and each points to a separate decision. Two questions go unanswered because data-plane logging is off, three because retention does not reach far enough. Turning on the data plane brings it to 5/8, and pushing retention to 365 days on top of that brings it to 7/8. Which comes first is a cost question, and the next section counts it. #### The Log's Own Surface and the Bill for Retention ```js // record.mjs — the log's own surface (deletion, immutability) and a retention-cost scan. // The currency is FICTIONAL: the "u" unit is made up, not a real price. import { DAILY, INVESTIGATION, answer } from "./event.mjs"; const DESTINATION = [ // where the log is written. MODEL. ["same account, same bucket", 11, 11, 14, "none", 0, 0], ["same account, locked bucket", 3, 0, 14, "retention lock", 4, 2], ["separate account, locked bucket", 1, 0, 5, "retention lock", 9, 6], ["separate account + second copy", 0, 0, 5, "lock + copy", 14, 6], ].map(([name, del, mod, read, lock, extra, stopped]) => ({ name, del, mod, read, lock, extra, stopped })); const EVENT_KB = 1.4, INGEST_PER_K = 0.4, STORAGE_PER_GB = 12; // fictional unit (u) const gb = (n, days) => (n * days * EVENT_KB) / (1024 * 1024); const monthly = (n, days) => Math.round((n * 30 / 1000) * INGEST_PER_K + gb(n, days) * STORAGE_PER_GB); const gen = (B) => (h) => h.map((v, j) => j ? String(v).padStart(B[j]) : String(v).padEnd(B[0])) .join(""); const rowDest = gen([33, 12, 16, 10, 19, 13, 14]); const rowCost = gen([16, 12, 14, 14, 14, 13, 13]); console.log(rowDest(["log destination", "can delete", "can modify", "can read", "immutability", "extra items", "stopped work"])); for (const h of DESTINATION) console.log(rowDest([h.name, h.del, h.mod, h.read, h.lock, h.extra, h.stopped])); console.log("\n" + rowCost(["retention(days)", "mgmt GB", "mgmt(u/mo)", "data GB", "data(u/mo)", "5% sample", "covered"])); for (const days of [30, 90, 180, 365, 730]) { const t = INVESTIGATION.map((x) => answer(x, true, days)).filter((x) => x[0] === "yes").length; console.log(rowCost([days, gb(DAILY.management, days).toFixed(2), monthly(DAILY.management, days), Math.round(gb(DAILY.data, days)), monthly(DAILY.data, days), monthly(DAILY.data * 0.05, days), `${t}/${INVESTIGATION.length}`])); } const coverage = (dataOn, days) => INVESTIGATION.map((x) => answer(x, dataOn, days)) .filter((x) => x[0] === "yes").length; const spend = (dataOn, days) => monthly(DAILY.management, days) + (dataOn ? monthly(DAILY.data, days) : 0); console.log(`\ncoverage and cost (fictional unit u): default ${coverage(false, 90)}/8 -> ` + `${spend(false, 90)} u/mo; data plane on 90 days ${coverage(true, 90)}/8 -> ${spend(true, 90)} u/mo; ` + `365 days ${coverage(true, 365)}/8 -> ${spend(true, 365)} u/mo`); console.log(`turning on data-plane logging: +${spend(true, 90) - spend(false, 90)} u/mo and ` + `+${coverage(true, 90) - coverage(false, 90)} questions. Retention 90 -> 365 days: ` + `+${spend(true, 365) - spend(true, 90)} u/mo and +${coverage(true, 365) - coverage(true, 90)} questions`); console.log(`two paths to the same coverage: management plane only, 365 days ${coverage(false, 365)}/8 -> ` + `${spend(false, 365)} u/mo; data plane on, 90 days ${coverage(true, 90)}/8 -> ${spend(true, 90)} u/mo`); console.log(`data-plane logging costs ` + `${(monthly(DAILY.data, 365) / monthly(DAILY.management, 365)).toFixed(0)} times as much as ` + `management-plane logging; a 5% sample divides that gap by 20 but does not answer single-resource questions`); ``` ``` log destination can delete can modify can read immutability extra items stopped work same account, same bucket 11 11 14 none 0 0 same account, locked bucket 3 0 14 retention lock 4 2 separate account, locked bucket 1 0 5 retention lock 9 6 separate account + second copy 0 0 5 lock + copy 14 6 retention(days) mgmt GB mgmt(u/mo) data GB data(u/mo) 5% sample covered 30 0.10 30 75 23214 1161 3/8 90 0.29 32 224 25002 1250 5/8 180 0.58 36 447 27684 1384 6/8 365 1.17 43 906 33197 1660 7/8 730 2.34 57 1813 44074 2204 8/8 coverage and cost (fictional unit u): default 3/8 -> 32 u/mo; data plane on 90 days 5/8 -> 25034 u/mo; 365 days 7/8 -> 33240 u/mo turning on data-plane logging: +25002 u/mo and +2 questions. Retention 90 -> 365 days: +8206 u/mo and +2 questions two paths to the same coverage: management plane only, 365 days 5/8 -> 43 u/mo; data plane on, 90 days 5/8 -> 25034 u/mo data-plane logging costs 772 times as much as management-plane logging; a 5% sample divides that gap by 20 but does not answer single-resource questions ``` The top table opens with the lesson's most unsettling row. In the default placement, the audit log sits inside the same account where the events it records took place, and **11 subjects can delete it, 11 can modify it.** This is where the log's meaning comes from: if a subject who can perform an event can also remove that event's record, the log is not evidence — it is a convenience. A retention lock drops the modifiable count to zero and the deletable count to 3; the cost is 4 configuration items and **2 legitimate jobs stopping** — two jobs that operate directly on the bucket do not coexist with the lock. Moving to a separate account drops the deletable count to 1, but it also drops the number of subjects who can read from 14 to 5, and **6 legitimate jobs stop**: teams that query the log directly lose their access, and cross-account reads do not come back without being set up separately. This is the cost of narrowing, and it comes with 9 configuration items. The last row zeroes out the deletable count: lock and second copy, 14 items. Immutability is not a free feature — it is a 14-item configuration and 6 stopped jobs. The bottom table keeps retention and cost in the same row. Management-plane logging is cheap: 43 u/mo at 365 days, 1.17 GB. Data-plane logging over the same span is 33,197 u and 906 GB — **772 times as much.** "The operation type the log does not cover" is not left out because it is free; it is left out because turning it on grows the bill by three digits. A five-percent sample divides that gap by twenty, but it does not answer single-resource questions: "which objects were read from the archive" goes unanswered under sampled logging. The marginal accounting decides the order. Turning on the data plane adds 25,002 u a month and gains 2 questions: 12,501 u per question. Extending retention from 90 to 365 days adds 8,206 u and also gains 2 questions: 4,103 u per question. The last line settles the decision — keeping only management-plane logging for 365 days answers 5/8 questions for 43 u/mo; turning on the data plane and keeping it for 90 days asks 25,034 u for the same 5/8. A five-hundred-eighty-fold difference comes down to a single configuration order. #### Summary - 273 of 480 events answer all four questions at once (57%); in the default configuration that number is 88 (18%), because 348 events are on the data plane and never get logged. - "When" and "from which identity" are answered in every logged event; "which resource" goes unanswered in 128 batch operations, and "who" goes unanswered in 106 role-assumption calls that do not carry a session tag. - 3 of 8 investigations get answered in the default configuration; two questions fall through because data-plane logging is off, three because retention does not reach far enough. - 11 subjects can delete the log in the default placement and 11 can modify it; a lock plus a separate account bring that to 0, at the cost of 14 configuration items and 6 legitimate jobs stopping. - Data-plane logging costs 772 times as much as management-plane logging (365 days, fictional unit); a five-percent sample divides that gap by twenty but does not answer single-resource questions. - The marginal cost sets the order: extending retention costs 4,103 u per question, turning on the data plane costs 12,501 u; keeping only management-plane logging for 365 days answers 5/8 questions for 43 u/mo. #### Next Step Once encryption, rotation, and logging are set up, a problem surfaces: all of it sits there as configuration, and configuration drifts over time. A bucket lock gets removed, a log destination gets changed, a leg's exception becomes permanent. The next lesson measures that drift with a rule set: how many resources pass the compliance rules, how many do not, and how many legitimate configurations break once automatic remediation kicks in. ### Configuration Compliance Auditing Source: https://academia.sh/en/courses/cloud-governance/configuration-compliance-auditing Automatic detection of drift: running 6 rules against 22 rented resources, 40 of 46 firing pairs get flagged, 32 of 38 actual noncompliance pairs get caught and 6 are missed, and across three remediation modes the open resource count drops from 16 to 6 while 8 legitimate configurations break and 5 of 6 legitimate changes get blocked. The previous lesson recorded who did what: the audit log is a record in the past tense, and its question is "did this operation happen." This lesson's question is in the present tense — does the resource's **current configuration** comply with the rule? The log stays silent on whether something is wrong; an operation that opens a store to public access lands in the log cleanly, and the log does not say that operation was wrong. **Configuration compliance auditing** is running a rule set against a set of resources and flagging what is noncompliant. Configuration drift was measured in M22/K01, and the reconciler loop that closes drift was measured in M22/K05; neither is repeated here. The distinction is this: there, the side that wrote the desired state and the side that closed the drift were the same side. Here the resource is rented, someone else writes the rule set, and remediation asks for **write access** into the rented layer — remediation itself becomes an access line item. #### The Rule Set and Noncompliant Resources The example course is the fictional **regional measurement network**: five teams, three environments. **CS25 — resource table:** twenty-two resources; each row carries a resource's owning team, environment, and six configuration fields. **CS26 — rule set:** six rules; each rule is made up of a predicate, the field the fix changes, and the fixed value. **CS27 — exception records:** four records; three are written at the resource level, one at the team level. **CS28 — actual-noncompliance label:** thirty-eight resource-rule pairs are manually labeled noncompliant; two are of a type with no counterpart in the rule set. **CS29 — change stream:** twenty changes, six legit-tagged. **CS30 — unit inputs:** a manual fix costs 25 minutes per item, the auditor's per-scan evaluation quota is 240 (fictional unit). The file below builds the tables, the rules, and the three modes. It is a process-internal model; no real cloud resource, auditor, or remediation exists. ```js // compliance/audit.mjs — configuration compliance auditing in the rented layer. Process-internal // model: the regional measurement network (fictional); no real cloud resource, auditor, or fix exists. const parseRows = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); const row = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); // Resource: name|team|env|public access|encryption at rest|audit logging|management port wide|owner tag|key age (days) const RESOURCE = parseRows(` measurement-store|measurement|production|0|1|1|0|1|120 measurement-archive|measurement|production|0|0|1|0|1|900 measurement-staging|measurement|staging|1|0|0|1|0|500 stream-queue|stream|production|0|1|1|0|1|200 stream-store|stream|production|1|1|0|0|1|410 stream-development|stream|development|1|0|0|1|0|730 catalog-store|catalog|production|0|1|1|0|1|90 catalog-gateway|catalog|production|1|1|1|0|1|150 catalog-staging|catalog|staging|0|0|1|1|1|380 gateway-public|gateway|production|1|1|1|0|1|60 gateway-management|gateway|production|0|1|1|1|1|340 gateway-development|gateway|development|1|1|0|1|0|620 ops-log|ops|production|0|1|1|0|1|280 ops-backup|ops|production|0|0|1|0|1|1100 ops-tool|ops|staging|0|1|0|1|1|450 ops-development|ops|development|1|1|0|0|0|800 measurement-edge|measurement|production|1|1|1|0|1|170 stream-archive|stream|production|0|0|1|0|0|960 catalog-backup|catalog|production|0|1|0|0|1|520 gateway-staging|gateway|staging|0|1|1|1|1|300 ops-quota|ops|production|0|1|1|0|1|110 measurement-report|measurement|staging|0|1|0|0|0|240`) .map(([name, team, env, pub, enc, log, port, tag, age]) => ({ name, team, env, public: +pub, encrypted: +enc, log: +log, port: +port, tag: +tag, age: +age })); // Rule set: rule id -> [predicate, field the fix changes, fixed value] const RULE = { Y1: [(k) => k.public === 1, "public", 0], Y2: [(k) => k.encrypted === 0, "encrypted", 1], Y3: [(k) => k.log === 0, "log", 1], Y4: [(k) => k.port === 1, "port", 0], Y5: [(k) => k.tag === 0, "tag", 1], Y6: [(k) => k.age > 365, "age", 90], }; const LABEL = { Y1: "public access closed", Y2: "encryption at rest", Y3: "audit logging", Y4: "management port narrow", Y5: "owner tag", Y6: "key age <= 365", Y7: "encryption in transit" }; // Declared exception records (model input). Three are per resource, one is at team level. const EXCEPTION = parseRows(` gateway-public|Y1 measurement-edge|Y1 ops-backup|Y6 team:stream|Y6`).map((x) => x.join("|")); const exempted = (k, r) => EXCEPTION.includes(`${k.name}|${r}`) || EXCEPTION.includes(`team:${k.team}|${r}`); // Manually labeled ground-truth noncompliance (model input): resource-rule pairs where the drift is not legitimate. // Y7 is not in the rule set; two actual noncompliance pairs are of this type. const ACTUAL = parseRows(` measurement-staging|Y1 Y2 Y3 Y4 Y5 Y6 stream-store|Y1 Y3 Y6 stream-development|Y1 Y2 Y6 gateway-development|Y1 Y3 Y6 ops-development|Y1 Y3 Y5 Y6 measurement-archive|Y2 catalog-staging|Y2 Y4 Y6 ops-backup|Y2 Y6 stream-archive|Y2 Y5 Y6 ops-tool|Y3 Y6 catalog-backup|Y3 Y6 measurement-report|Y3 Y5 gateway-management|Y4 gateway-staging|Y4 stream-queue|Y7 catalog-store|Y7`).flatMap(([name, r]) => r.split(" ").map((x) => `${name}|${x}`)); const RULES = Object.keys(RULE); const ALL = [...RULES, "Y7"]; // Fired: predicate is true. Flagged: fired and no exception record. const fire = (set) => { const a = [], f = []; for (const k of set) for (const r of RULES) if (RULE[r][0](k)) { a.push(`${k.name}|${r}`); if (!exempted(k, r)) f.push(`${k.name}|${r}`); } return { fired: a, flagged: f }; }; const baseline = fire(RESOURCE); const caught = baseline.flagged.filter((x) => ACTUAL.includes(x)); const falseFlag = baseline.flagged.filter((x) => !ACTUAL.includes(x)); const missed = ACTUAL.filter((x) => !baseline.flagged.includes(x)); const nonCompliant = new Set(baseline.flagged.map((x) => x.split("|")[0])); console.log(`${RESOURCE.length} resources, ${RULES.length} rules, ${EXCEPTION.length} exception records, ` + `${ACTUAL.length} manually labeled actual noncompliance`); console.log(`flagged resources: ${nonCompliant.size}/${RESOURCE.length}, ` + `flagged pairs: ${baseline.flagged.length} (fired ${baseline.fired.length})`); const K = [-8, -25, 11, 12, 11, 8, 15]; console.log(); row(K, "rule", "checks", "fired", "flagged", "caught", "missed", "false flag"); for (const r of ALL) { const at = baseline.fired.filter((x) => x.endsWith(`|${r}`)).length; const fl = baseline.flagged.filter((x) => x.endsWith(`|${r}`)); const y = fl.filter((x) => ACTUAL.includes(x)).length; const mi = ACTUAL.filter((x) => x.endsWith(`|${r}`) && !fl.includes(x)).length; row(K, r, LABEL[r], at, fl.length, y, mi, fl.length - y); } row(K, "total", "", baseline.fired.length, baseline.flagged.length, caught.length, missed.length, falseFlag.length); // Change stream: resource|field|new value|legitimate (model input) const CHANGE = parseRows(` catalog-store|public|1|1 catalog-store|log|0|0 measurement-store|port|1|0 measurement-store|age|900|0 stream-queue|public|1|1 stream-queue|encrypted|0|0 ops-quota|port|1|1 ops-quota|tag|0|0 gateway-public|log|0|0 gateway-public|age|700|0 measurement-edge|port|1|1 catalog-gateway|encrypted|0|0 catalog-gateway|age|800|0 ops-log|public|1|1 ops-log|port|1|0 measurement-store|tag|0|0 stream-queue|age|900|1 gateway-management|log|0|0 ops-quota|age|600|0 catalog-store|tag|0|0`) .map(([name, field, value, legit]) => ({ name, field, value: +value, legit: legit === "1" })); // Three modes: report only, block, fix. Every mode sees the same stream in the same order. const runMode = (mode) => { const state = RESOURCE.map((k) => ({ ...k })); const find = (name) => state.find((k) => k.name === name); let blocked = 0, reverted = 0, broken = 0, fixed = 0; if (mode === "fix") for (const k of state) for (const r of RULES) if (RULE[r][0](k) && !exempted(k, r)) { if (!ACTUAL.includes(`${k.name}|${r}`)) broken++; k[RULE[r][1]] = RULE[r][2]; fixed++; } for (const c of CHANGE) { const k = find(c.name); const before = { ...k }; k[c.field] = c.value; const now = RULES.filter((r) => RULE[r][0](k) && !RULE[r][0](before) && !exempted(k, r)); if (now.length === 0) continue; if (mode === "block") { Object.assign(k, before); if (c.legit) blocked++; } else if (mode === "fix") { for (const r of now) { k[RULE[r][1]] = RULE[r][2]; fixed++; } if (c.legit) reverted++; } } // Left open at the end of the window: resources and pairs still carrying actual noncompliance const open = new Set(), pair = []; for (const k of state) { for (const r of RULES) if (RULE[r][0](k) && ACTUAL.includes(`${k.name}|${r}`)) { open.add(k.name); pair.push(`${k.name}|${r}`); } if (ACTUAL.includes(`${k.name}|Y7`)) { open.add(k.name); pair.push(`${k.name}|Y7`); } } // The auditor's write scope: only fix mode writes fields const scope = mode === "fix" ? RESOURCE.length * RULES.length : 0; return { open: open.size, pair: pair.length, blocked, reverted, broken, fixed, scope }; }; const MODE = ["report", "block", "fix"], RESULT = {}; for (const mode of MODE) RESULT[mode] = runMode(mode); const D = [-10, 17, 13, 11, 23, 22, 15]; console.log(); row(D, "mode", "open resources", "open pairs", "fixes", "blocked/reverted", "broken legit config", "write scope"); for (const mode of MODE) { const s = RESULT[mode]; row(D, mode, s.open, s.pair, s.fixed, s.blocked + s.reverted, s.broken, `${s.scope} fields`); } // Manual-fix load (fictional unit): in report mode, every flagged pair is one work item const MINUTES = 25, personHours = (n) => ((n * MINUTES) / 60).toFixed(1); const legitCount = CHANGE.filter((c) => c.legit).length; console.log(`\nreport-mode queue: ${baseline.flagged.length} pairs x ${MINUTES} min = ` + `${personHours(baseline.flagged.length)} person-hours; ${legitCount} of the ${CHANGE.length} ` + `changes in the stream are legit-tagged`); console.log(`block: ${RESULT.block.blocked}/${legitCount} legit changes were blocked; ` + `fix: ${RESULT.fix.reverted}/${legitCount} were reverted`); console.log(`missed ${missed.length} pairs: ${missed.filter((x) => x.endsWith("|Y7")).length} are a ` + `type with no counterpart in the rule set, ` + `${missed.filter((x) => !x.endsWith("|Y7")).length} are pairs an exception record covers`); const teamException = ACTUAL.filter((x) => { const [name, r] = x.split("|"); const k = RESOURCE.find((z) => z.name === name); return EXCEPTION.includes(`team:${k.team}|${r}`); }).length; console.log(`the single team-level exception record covers ${teamException} actual noncompliance pairs`); // Limit and remaining (fictional): the auditor's per-scan evaluation quota const QUOTA = 240, used = RESOURCE.length * RULES.length; console.log(`\nevaluation quota ${QUOTA}/scan; used ${RESOURCE.length} x ` + `${RULES.length} = ${used}, remaining ${QUOTA - used} -> ` + `${Math.floor((QUOTA - used) / RULES.length)} more resources with the same rule set, or ` + `${Math.floor(QUOTA / RESOURCE.length) - RULES.length} more rules with the same resource set`); ``` ``` 22 resources, 6 rules, 4 exception records, 38 manually labeled actual noncompliance flagged resources: 15/22, flagged pairs: 40 (fired 46) rule checks fired flagged caught missed false flag Y1 public access closed 8 6 5 0 1 Y2 encryption at rest 6 6 6 0 0 Y3 audit logging 8 8 7 0 1 Y4 management port narrow 7 7 4 0 3 Y5 owner tag 6 6 4 0 2 Y6 key age <= 365 11 7 6 4 1 Y7 encryption in transit 0 0 0 2 0 total 46 40 32 6 8 mode open resources open pairs fixes blocked/reverted broken legit config write scope report 16 38 0 0 0 0 fields block 16 38 0 5 0 0 fields fix 6 6 59 5 8 132 fields report-mode queue: 40 pairs x 25 min = 16.7 person-hours; 6 of the 20 changes in the stream are legit-tagged block: 5/6 legit changes were blocked; fix: 5/6 were reverted missed 6 pairs: 2 are a type with no counterpart in the rule set, 4 are pairs an exception record covers the single team-level exception record covers 3 actual noncompliance pairs evaluation quota 240/scan; used 22 x 6 = 132, remaining 108 -> 18 more resources with the same rule set, or 4 more rules with the same resource set ``` The numbers are of the measurement class; their inputs are the assumptions above. Six rules fire in forty-six places across twenty-two resources, the exception records filter out six of them, and forty flagged pairs remain. Looked at by resource, fifteen of the twenty-two look noncompliant — that is the number a first scan reads, and it is misleading on its own, because not every flagged pair is actually noncompliant. #### Caught and Missed, Rule by Rule The table below separates the rules and shows they do not behave alike. Y2 runs clean: it fires in six places, all six are actually noncompliant, no false flags — because its predicate is independent of a resource's intent; there is no legitimate reading of encryption at rest being off. Y4 is written the same way but behaves differently: of seven firings, four are real and three flag a legitimate configuration; a management tool's port being open in a development environment is that environment's own arrangement. Y5 is the same: two of six firings belong to a temporary development resource whose owner tag is deliberately left empty. Of the forty flagged pairs, thirty-two are actual noncompliance and eight are false flags. The six missed pairs come from two separate places, and neither is about a rule's threshold. Four sit under exception records: Y6 fires in eleven places but only seven get flagged, because one exception record is written at the team level and covers three actual noncompliance pairs by itself. The exception's scope is more decisive than the noncompliance itself — write access to an exception record is an audit line item, and who can write it has to be counted. The remaining two missed pairs are of a type with no counterpart in the rule set: no rule is written for encryption in transit, so no number is produced for those two resources either. Tightening the rules does not close this gap; the gap is not in the threshold, it is in the rule set's scope. #### Three Remediation Modes The twenty-change stream runs in the same order under all three modes. Report-only mode touches nothing: thirty-eight open pairs and sixteen open resources remain, the auditor's write access in the rented layer is zero fields, and in exchange a forty-item manual-fix queue is born — 16.7 person-hours at 25 minutes an item. Block mode rejects new changes but does not touch what already exists; the open count does not move, sixteen resources and thirty-eight pairs stand exactly as they were. The cost of blocking sits in a different column: five of the six legitimate changes in the stream were stopped. How the one legitimate change that got through made it is worth saying. The change that extends the queue's key age was not rejected, because that team has a team-level exception record for Y6 — meaning the record that lets legitimate work through and the record that covers three actual noncompliance pairs are the same record. Narrowing the exception would drop missed from four to one and raise blocked legitimate changes from five to six; both columns are tied to the same switch. Fix mode drops the open count from sixteen resources to six, and from thirty-eight pairs to six. The remaining six pairs are already the missed ones: four are covered by the exception record, two are of the type with no counterpart in the rule set. The auditor writes fifty-nine fixes for this and asks for write access to six fields on each of twenty-two resources — a hundred thirty-two fields. In the rented layer this is a scope of access no human holds: the single subject that opens a resource to the network, flips encryption, and rotates the key becomes the audit tool itself. #### The Cost of Wrong Remediation Fix mode's second column is the real measure: eight legitimate configurations broke. These are the entirety of the eight falsely flagged pairs — the catalog gateway that was supposed to stay publicly open got closed, three management ports got narrowed, two temporary resources got an owner tag written onto them, one archive key's lifetime got shortened, one staging resource's logging got turned on. The auditor does not know these are legitimate, because the only carrier of legitimacy information is the exception record, and it was not written for these eight pairs. The rate deserves a name: eight of the forty fixes in the baseline scan, meaning one in every five fixes, break a legitimate configuration. On top of that, five legitimate changes coming from the stream get reverted after being applied. Mode choice is therefore not a choice about rule strictness but a choice between two columns: report-only mode stops no legitimate work and leaves sixteen resources open; fix mode closes ten resources and breaks or reverts thirteen legitimate operations. Block mode in the middle does neither — it does not let new noncompliance in, but it does not close the old noncompliance either; the sixteen existing resources still need a manual-fix queue. #### Limit and Remaining The auditor's own limit is also counted. The per-scan evaluation quota is 240 units; twenty-two resources times six rules uses 132 units and leaves 108. Eighteen more resources could be added with the same rule set, or four more rules could be written with the same resource set; not both at once. An encryption-in-transit rule written to close the scope gap would take up twenty-two evaluations — a fifth of the remaining budget. The quota takes rule-writing out of being free: growing the rule set and growing the resource count run into the same limit. #### Summary - Six rules fire in forty-six places across twenty-two resources, and four exception records filter out six of them; of the forty flagged pairs, thirty-two are actual noncompliance and eight are false flags. - Four of the six missed pairs sit under exception records, and a single team-level record covers three actual noncompliance pairs by itself; two are of a type with no counterpart in the rule set and do not close by tightening the threshold. - Report-only mode leaves sixteen resources open, stops no legitimate work, and produces a forty-item queue (16.7 person-hours, fictional unit). - Block mode does not change the open count but stops five of six legitimate changes; the one legitimate change that gets through does so because of the exception record covering three actual noncompliance pairs. - Fix mode drops the open count from sixteen resources to six; in exchange it breaks eight legitimate configurations, reverts five legitimate changes, and asks for write access to a hundred thirty-two fields. - 132 of the auditor's 240-evaluation per-scan quota are used; the remaining 108 units are enough for eighteen more resources or four more rules, not both. #### Next Step Compliance auditing read the resource's configuration, and the first rule was about public access: Y1 fired in eight places, six got flagged. But this rule reads a single flag for whether a resource is open. Once that flag is closed, how many paths really remain to that resource, which endpoint the service is reached through, and how the name resolves were never asked. The next lesson counts how many paths provide access to a managed service, and measures how many legitimate access paths break once the public endpoint is closed. ### Network Isolation and Private Endpoints Source: https://academia.sh/en/courses/cloud-governance/network-isolation-and-private-endpoints Measuring whether a managed service is reached through the public endpoint or a private one: closing the public endpoint on 25 access records breaks 12 legitimate ones, drops 12 to zero paths, moves the same name from resolving to a private address in 11 places to 14, and closing the gaps costs 12 new private endpoints at 180 fictional units a month. The previous lesson read whether a resource is open to public access with a single flag: Y1 fired in eight places, six got flagged. That flag is a resource's own setting. This lesson's question sits one stage earlier — when a workload reaches a managed service, which path does the packet take? Network isolation and private connectivity themselves were measured in M23/K02; that is not repeated here. The object here is individually rented services: does access to an object store, a queue, or a secret storage service go through the service's **public endpoint**, or through a **private endpoint** placed inside the private network? The reason this distinction is measurable is this: a managed service comes by default with a public name and a public address. A private endpoint changes two things at once — it adds a new path to the network, and it changes the address the name resolves to. Each of the two can break separately, and this lesson counts them separately. #### Service, Network, and Access Table The example course is the fictional **regional measurement network**. **CS31 — managed service catalog:** seven services; six support a private endpoint, one only offers a public endpoint. Every supported service has a monthly private-endpoint rent (fictional unit). **CS32 — three private networks and a single peering record:** the production and staging networks are peered, the shared network is not. **CS33 — installed private endpoints:** all six of six services are installed only in the production network. **CS34 — name-resolution zone:** the zone carrying the private records is attached to the production and shared networks, not to the staging network. **CS35 — access table:** twenty-five access records; twenty-two are legitimate, three are not, three can be converted into a jump point. **CS36 — three scenarios:** A public endpoint open, B public endpoint closed, C public endpoint closed and gaps closed. The file below builds these tables and runs the three scenarios. It is a process-internal model; no real network, name record, or service exists. ```js // isolation/endpoint.mjs — whether a managed service is reached through the public endpoint or a private one. // Process-internal model: the regional measurement network (fictional); no real network, name record, or service exists. const parseRows = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); const row = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); // Managed service: name|private endpoint support|private endpoint monthly rent (fictional unit) const SERVICE = Object.fromEntries(parseRows(` object-store|1|18 relational-store|1|18 queue|1|14 secret-storage|1|12 key-management|1|12 log-collection|1|16 notification-service|0|0`).map(([name, d, k]) => [name, { supports: d === "1", rent: +k }])); // Private networks and peering (model input). shared-network is not peered. const NETWORK = ["production-network", "staging-network", "shared-network"]; const PEERING = [["production-network", "staging-network"]]; const peered = (a, b) => a === b || PEERING.some(([x, y]) => (x === a && y === b) || (x === b && y === a)); // Installed private endpoints: service|network. All six of six services are only in the production network. const INSTALLED_A = parseRows(` object-store|production-network relational-store|production-network queue|production-network secret-storage|production-network key-management|production-network log-collection|production-network`).map((x) => x.join("|")); // Networks the name-resolution zone is attached to: staging-network is not attached. const ZONE_A = ["production-network", "shared-network"]; // Access: client|service|location (network or external)|legit|can be converted into a jump point const ACCESS = parseRows(` measurement-collector|object-store|production-network|1|0 measurement-collector|queue|production-network|1|0 stream-processor|queue|production-network|1|0 stream-processor|relational-store|production-network|1|0 stream-processor|key-management|production-network|1|0 catalog-service|relational-store|production-network|1|0 catalog-service|secret-storage|production-network|1|0 gateway-service|log-collection|production-network|1|0 gateway-service|notification-service|production-network|1|0 staging-runner|object-store|staging-network|1|0 staging-runner|relational-store|staging-network|1|0 staging-runner|queue|staging-network|1|0 shared-reporter|object-store|shared-network|1|0 shared-reporter|relational-store|shared-network|1|0 shared-reporter|log-collection|shared-network|1|0 management-tool|relational-store|external|1|1 management-tool|secret-storage|external|1|1 management-tool|key-management|external|1|1 nightly-job|object-store|external|1|0 nightly-job|queue|external|1|0 edge-uploader|object-store|external|1|0 edge-uploader|notification-service|external|1|0 developer-laptop|object-store|external|0|0 developer-laptop|relational-store|external|0|0 unknown-client|object-store|external|0|0`) .map(([client, service, location, legit, jump]) => ({ client, service, location, legit: legit === "1", jump: jump === "1" })); // The paths of one access record and its name resolution. publicOpen: is the service's public endpoint still open. const measure = (e, { installed, zone, publicOpen, jumpAvailable }) => { const hasPrivate = (net) => installed.includes(`${e.service}|${net}`); const privatePath = e.location !== "external" && NETWORK.some((a) => hasPrivate(a) && peered(a, e.location)); const publicPath = publicOpen(e.service); // Name resolution: seen from a network the internal zone is attached to, with a record for that service, resolves private const recordExists = NETWORK.some((a) => hasPrivate(a)); const resolution = e.location !== "external" && zone.includes(e.location) && recordExists ? "private" : "public"; // Jump point: an external client passes through the internal network, the name resolves from the internal zone const jumpPath = jumpAvailable && e.jump && recordExists; const paths = []; if (privatePath) paths.push("private"); if (publicPath) paths.push("public"); if (jumpPath) paths.push("jump"); const works = (resolution === "private" && privatePath) || (resolution === "public" && publicPath) || jumpPath; const status = works ? (jumpPath && !(resolution === "private" && privatePath) && !(resolution === "public" && publicPath) ? "jump" : resolution === "private" ? "private endpoint" : "public endpoint") : (resolution === "private" ? "name private, no path" : "name public, endpoint closed"); return { paths: paths.length, resolution, works, status }; }; // Three scenarios const supportsPrivate = (h) => SERVICE[h].supports; const INSTALLED_C = NETWORK.flatMap((a) => Object.keys(SERVICE).filter(supportsPrivate).map((h) => `${h}|${a}`)); const SCENARIO = { "A public endpoint open": { installed: INSTALLED_A, zone: ZONE_A, jumpAvailable: false, publicOpen: () => true }, "B public endpoint closed": { installed: INSTALLED_A, zone: ZONE_A, jumpAvailable: false, publicOpen: (h) => !SERVICE[h].supports }, "C gaps closed": { installed: INSTALLED_C, zone: NETWORK, jumpAvailable: true, publicOpen: (h) => !SERVICE[h].supports }, }; const S = [-27, 10, 12, 10, 16, 10, 15]; row(S, "scenario", "private", "public", "jump", "broken legit", "cut", "from internet"); const R = {}; for (const [name, s] of Object.entries(SCENARIO)) { const c = ACCESS.map((e) => ({ e, ...measure(e, s) })); const count = (d) => c.filter((x) => x.status === d).length; R[name] = { priv: count("private endpoint"), pub: count("public endpoint"), jump: count("jump"), broken: c.filter((x) => !x.works && x.e.legit).length, cut: c.filter((x) => !x.works && !x.e.legit).length, internet: c.filter((x) => x.works && x.status === "public endpoint").length, c, }; row(S, name, R[name].priv, R[name].pub, R[name].jump, R[name].broken, R[name].cut, R[name].internet); } // Name resolution: which address the same name resolves to in each scenario const C = [-27, 18, 17]; console.log(); row(C, "scenario", "to private addr", "to public addr"); for (const name of Object.keys(SCENARIO)) row(C, name, R[name].c.filter((x) => x.resolution === "private").length, R[name].c.filter((x) => x.resolution === "public").length); // How many paths remain once the public endpoint closes: distribution of path count per access const Y = [-27, 9, 9, 9, 14]; console.log(); row(Y, "scenario", "0 paths", "1 path", "2 paths", "total access"); for (const name of Object.keys(SCENARIO)) { const d = [0, 1, 2].map((n) => R[name].c.filter((x) => x.paths === n).length); row(Y, name, ...d, ACCESS.length); } // Why legitimate access breaks in scenario B const B = [-31, 10]; console.log(); row(B, "B: break reason", "access"); for (const n of ["name private, no path", "name public, endpoint closed"]) row(B, n, R["B public endpoint closed"].c.filter((x) => !x.works && x.e.legit && x.status === n).length); const locations = [...new Set(ACCESS.map((e) => e.location))]; const KN = [-27, 10]; console.log(); row(KN, "B: broken legit, location", "access"); for (const k of locations) { const n = R["B public endpoint closed"].c.filter((x) => !x.works && x.e.legit && x.e.location === k); if (n.length) row(KN, k, n.length); } // The private endpoint's cost: separate per service and per network (fictional unit) const monthlyRent = (installed) => installed.reduce((t, x) => t + SERVICE[x.split("|")[0]].rent, 0); const supported = Object.keys(SERVICE).filter(supportsPrivate).length; console.log(`\nprivate endpoints: A/B ${INSTALLED_A.length} installed, ${monthlyRent(INSTALLED_A)} units/mo; ` + `C ${INSTALLED_C.length} installed (${supported} services x ${NETWORK.length} networks), ${monthlyRent(INSTALLED_C)} units/mo`); console.log(`C adds ${INSTALLED_C.length - INSTALLED_A.length} new private endpoints and ` + `${monthlyRent(INSTALLED_C) - monthlyRent(INSTALLED_A)} units/mo over B; ` + `broken legitimate access ${R["B public endpoint closed"].broken} -> ${R["C gaps closed"].broken}`); const cannotClose = Object.keys(SERVICE).filter((h) => !SERVICE[h].supports); console.log(`${cannotClose.length} service does not support private endpoints: ` + `${ACCESS.filter((e) => !SERVICE[e.service].supports).length} access records go through the public ` + `endpoint in every scenario; average rent ${(monthlyRent(INSTALLED_C) / INSTALLED_C.length).toFixed(1)} ` + `units per endpoint`); // Every installed private endpoint is for one service and one network; how many are actually used const NEEDED = [...new Set(ACCESS.filter((e) => e.legit && e.location !== "external" && SERVICE[e.service].supports).map((e) => `${e.service}|${e.location}`))]; const idle = INSTALLED_C.filter((x) => !NEEDED.includes(x)); console.log(`of the ${INSTALLED_C.length} private endpoints in C, ${NEEDED.length} are used by a ` + `legitimate access record; ${idle.length} sit idle and eat ${monthlyRent(idle)} units/mo`); console.log(`installing only the needed ${NEEDED.length} would run ${monthlyRent(NEEDED)} units/mo: ` + `the ceiling is ${supported} services x ${NETWORK.length} networks = ${supported * NETWORK.length}, ` + `${supported * NETWORK.length - NEEDED.length} installations remaining`); ``` ``` scenario private public jump broken legit cut from internet A public endpoint open 8 14 0 3 0 14 B public endpoint closed 8 2 0 12 3 2 C gaps closed 14 2 3 3 3 2 scenario to private addr to public addr A public endpoint open 11 14 B public endpoint closed 11 14 C gaps closed 14 11 scenario 0 paths 1 path 2 paths total access A public endpoint open 0 14 11 25 B public endpoint closed 12 13 0 25 C gaps closed 6 19 0 25 B: break reason access name private, no path 3 name public, endpoint closed 9 B: broken legit, location access staging-network 3 shared-network 3 external 6 private endpoints: A/B 6 installed, 90 units/mo; C 18 installed (6 services x 3 networks), 270 units/mo C adds 12 new private endpoints and 180 units/mo over B; broken legitimate access 12 -> 3 1 service does not support private endpoints: 2 access records go through the public endpoint in every scenario; average rent 15.0 units per endpoint of the 18 private endpoints in C, 12 are used by a legitimate access record; 6 sit idle and eat 78 units/mo installing only the needed 12 would run 192 units/mo: the ceiling is 6 services x 3 networks = 18, 6 installations remaining ``` The numbers are of the measurement class; their inputs are the assumptions above. The first row worth reading is scenario A, and it is unsettling: private endpoints are installed, the public endpoint is still open, and **fourteen of twenty-five access records go through the open internet.** A private endpoint being installed does not mean traffic goes through it. In the same row, even in scenario A, three legitimate access records already sit broken. #### Separating Access by Path Count The path-distribution table carries this distinction. In scenario A, no access record sits at zero paths: fourteen sit at one path, eleven at two paths. That means both the private and the public endpoint are open for the eleven access records in the production and staging networks. Even so, three access records in the staging network go through the public endpoint, because the name-resolution zone is not attached to the staging network. The path exists; the name does not point to it. This is what the measure separates: installing a private endpoint opens a path, attaching the name record makes that path usable, and the two are separate jobs. The three access records in the shared network sit on the opposite side. The name zone is attached to the shared network, so the name resolves to the private address; but the shared network is not peered with the production network, and there is no path to the private address. Even in scenario A, these three access records are broken, and the reason is not the public endpoint being closed. Name resolution produces its result before routing does; once the address resolves to the wrong side, even a public endpoint left open does not save it. #### What's Left When the Public Endpoint Closes In scenario B, the public endpoint closes on the six supported services. Twelve of the twenty-five access records drop to zero paths, thirteen stay at one path, and none remain at two paths. Broken legitimate access rises from three to twelve; in exchange, three access records that are not legitimate — the developer laptop's two direct reads of the production store and one client of unknown origin — get cut off. This is the gain column, and it is three items. The reasons behind the twelve broken records fall into two groups. Three are in the "name private, no path" state and were already broken in A; nine are in the "name public, endpoint closed" state and are the direct result of closing the endpoint. By location, three break in the staging network, three in the shared network, and six from external. The six from external are this lesson's real warning: the nightly job, the edge uploader, and the management tool are not inside the private network, and never were. Closing the public endpoint cuts them without seeing them, because the decision to close is made on the service side, and the service side does not know where the client is. #### The Bill for Closing the Gaps In scenario C, three things happen at once: a private endpoint is installed in every network for every supported service, the name zone is attached to all three networks, and a jump point is added for the management tool. The result drops broken legitimate access from twelve to three; access going through a private endpoint rises from eight to fourteen, three access records go through the jump point, and the count of names resolving to a public address drops from fourteen to eleven. The remaining three broken access records are the ones that cannot be closed: the nightly job's two access records and the edge uploader's store access. These run from external, cannot be converted into a jump point, and cannot reach the service by a private endpoint's very definition. A private endpoint resolves access from inside the network; for a client sitting outside the network, it has nothing to resolve. These three jobs will either have to move inside the network, or the public endpoint will have to stay open. The bill itself is a separate line item, and it is not a one-time cost. **A private endpoint is needed separately per service and per network:** six supported services times three networks, eighteen installations, at 270 fictional units a month. That is twelve new private endpoints and 180 units/mo more than B — the cost of rescuing nine legitimate access records. Of the eighteen installations, only twelve are used by a legitimate access record; the remaining six sit idle and eat 78 units a month. Installing only the needed twelve would have made the bill 192 units. The ceiling is eighteen and twelve of it is full; every new service or new network multiplies this product, because a single private endpoint does not serve more than one service. One line item does not close under any scenario. The notification service does not support a private endpoint; the two access records reaching it go through the public endpoint in all three scenarios. The rented layer's limit becomes visible here: how far isolation can be pushed is decided by the capability the provider gives per service, and that item cannot be negotiated. #### Summary - Private endpoints being installed does not mean traffic goes through them: in scenario A, fourteen of twenty-five access records go through the open internet, and three are already broken. - Path and name resolution are measured separately. Two paths are open in the staging network, but three access records go through the public endpoint because the name zone is not attached; in the shared network the name resolves to private, but there is no path because there is no peering. - Once the public endpoint closes, twelve of twenty-five access records drop to zero paths; broken legitimate access rises from three to twelve (staging 3, shared 3, external 6), and three non-legitimate access records get cut off. - Closing the gaps drops broken access from twelve to three; the cost is twelve new private endpoints and 180 fictional units a month. The remaining three access records are outside the network and cannot be resolved with a private endpoint. - A private endpoint is needed separately per service and per network: six services times three networks, twelve of the eighteen-installation ceiling are used, six sit idle and eat 78 units a month. - One service does not support a private endpoint; the two access records reaching it go through the public endpoint in every scenario, and this limit is not a decision the tenant can make. #### Next Step The public endpoint was closed, paths were counted, and the three access records left outside were named. But the closed path was meant to keep out not only the expected clients but also the unexpected ones; the three cut non-legitimate access records showed that. The problem is that those three could only be counted because they were written into the table. An event that is not in the table, that comes from inside the private network, and that carries valid credentials never showed up in any table. The next lesson measures anomalous activity: in a known set of events, it counts how many the detection rules catch, how many they miss, and how many false alarms they produce. ### Threat Detection Source: https://academia.sh/en/courses/cloud-governance/threat-detection Measuring anomalous-activity alerts against a known event set: 9 detection rules over 30 events catch 9 of 11 malicious events, produce 13 false alarms, 8 events get caught by only a single signal source, three sources cost 612 fictional units a month, and once the false-alarm threshold is crossed two rules close and caught drops to 7. The previous lesson closed the public endpoint and cut the three non-legitimate access records written into the table. Those three could be cut because they were in the table. An event that comes from inside the private network, carries valid credentials, and performs a permitted operation shows up in no access table; access control counts it as legitimate, because by definition it is legitimate. **Threat detection** targets this gap: it asks whether the activity that authorization let through is **unusual**. The shape of the question is therefore different. An access decision is a binary predicate with a defined correct answer; detection is a guess, and it is measured with three numbers: caught, missed, false alarm. This lesson runs detection rules against a known event set, derives the three numbers, then counts two costs — the charge for collecting signal, and the behavior a false alarm produces in the team. #### Event Set, Signal Sources, and Rules The example course is the fictional **regional measurement network**. **CS37 — known event set:** thirty events, eleven malicious-tagged. The nineteen benign ones are deliberately chosen as **hard look-alikes**; each shares one surface with a malicious event, so the false-alarm rate will run higher than in an easy set. **CS38 — three signal sources:** audit log (operation, operation count, hour), network flow (outbound volume, previously unseen destination, unusual port), and identity behavior (new location, failed attempt, first-seen subject). **CS39 — nine detection rules:** each rule looks at a single source, three rules per source. **CS40 — collection charge:** daily volume per source and unit per GB (fictional unit). **CS41 — false-alarm threshold:** a rule producing three or more false alarms gets closed. **CS42 — collection cap:** 60 GB a day (fictional unit). The file below builds the set, the rules, and the two costs. It is a process-internal model; no real event, signal stream, or cost exists. ```js // detection/rule.mjs — running detection rules against a known event set. Process-internal model: // the regional measurement network (fictional); no real event, signal stream, or cost exists. const parseRows = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); const row = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); // Event: name|malicious|D op|D count|D hour|N egress MB|N new dest|N odd port| // I new location|I failed attempt|I new subject const EVENT = parseRows(` bulk-archive-delete|1|delete|40|3|20|0|0|0|0|0 policy-expansion|1|policy-change|2|4|5|0|0|1|0|0 nightly-role-chain|1|assume-role|9|2|10|0|0|1|2|0 store-outbound-copy|1|read|60|14|900|1|0|0|0|0 leak-to-new-destination|1|read|12|11|120|1|0|0|0|0 key-exfiltration|1|key-create|3|1|8|0|0|1|0|1 brute-force-login|1|read|4|22|3|0|0|1|18|0 quiet-side-channel|1|read|6|13|40|0|1|0|0|0 shared-account-takeover|1|assume-role|2|10|15|0|0|1|1|1 logging-shutdown|1|policy-change|1|13|4|0|0|0|0|0 slow-exfiltration|1|read|9|15|45|0|0|0|0|0 nightly-backup-restore|0|delete|30|3|400|0|0|0|0|0 bulk-cleanup|0|delete|25|11|10|0|0|0|0|0 release-rollout|0|policy-change|1|10|6|0|0|0|0|0 measurement-ingest|0|write|80|9|650|0|0|0|0|0 nightly-report-pull|0|read|20|4|300|1|0|0|0|0 new-team-member|0|read|5|9|12|0|0|1|1|1 login-from-travel|0|read|7|8|9|0|0|1|2|0 report-from-shared-network|0|read|11|16|80|1|0|0|0|0 edge-upload|0|write|14|20|140|1|0|0|0|0 staging-run|0|assume-role|6|13|20|0|0|0|0|0 tool-update|0|write|3|2|18|0|1|0|0|0 password-reset|0|read|2|9|2|0|0|0|11|0 maintenance-window|0|delete|22|1|30|0|0|0|0|0 backup-verification|0|read|18|5|520|0|0|0|0|0 setup-job|0|key-create|2|14|7|0|0|0|0|1 catalog-refresh|0|read|15|10|60|0|0|0|0|0 staging-data-delete|0|delete|8|14|5|0|0|0|0|0 log-archiving|0|write|19|12|210|0|0|0|0|0 scheduled-job|0|read|10|7|25|0|0|0|0|0`) .map(([name, m, di, ds, dh, ac, ay, ap, kk, kb, ko]) => ({ name, malicious: m === "1", auditOp: di, auditCount: +ds, auditHour: +dh, netEgress: +ac, netNewDest: +ay, netOddPort: +ap, idNewLocation: +kk, idFailedAttempt: +kb, idNewSubject: +ko })); // Detection rule: id|signal source|predicate. Every rule looks at a single source. const RULE = { T1: ["audit log", (o) => o.auditOp === "policy-change" && o.auditHour < 6], T2: ["audit log", (o) => o.auditOp === "delete" && o.auditCount >= 20], T3: ["audit log", (o) => o.auditOp === "assume-role" && o.auditCount >= 5], T4: ["network flow", (o) => o.netEgress >= 500], T5: ["network flow", (o) => o.netNewDest === 1 && o.netEgress >= 50], T6: ["network flow", (o) => o.netOddPort === 1], T7: ["identity behavior", (o) => o.idNewLocation === 1 && o.idFailedAttempt >= 1], T8: ["identity behavior", (o) => o.idFailedAttempt >= 10], T9: ["identity behavior", (o) => o.idNewSubject === 1 && o.idNewLocation === 1], }; const SOURCE = ["audit log", "network flow", "identity behavior"]; const RULES = Object.keys(RULE); const alert = (rules) => EVENT.map((o) => ({ o, rule: rules.filter((t) => RULE[t][1](o)) })); const summary = (rules) => { const a = alert(rules); const malicious = a.filter((x) => x.o.malicious); return { caught: malicious.filter((x) => x.rule.length > 0).length, missed: malicious.filter((x) => x.rule.length === 0).length, falseAlarm: a.filter((x) => !x.o.malicious && x.rule.length > 0).length, quiet: a.filter((x) => !x.o.malicious && x.rule.length === 0).length, }; }; const maliciousCount = EVENT.filter((o) => o.malicious).length; console.log(`${EVENT.length} known events (${maliciousCount} malicious-tagged), ` + `${RULES.length} detection rules, ${SOURCE.length} signal sources`); // Rule by rule const K = [-6, -20, 9, 11, 14]; console.log(); row(K, "rule", "signal source", "alerts", "caught", "false alarm"); for (const t of RULES) { const a = alert([t]).filter((x) => x.rule.length > 0); const y = a.filter((x) => x.o.malicious).length; row(K, t, RULE[t][0], a.length, y, a.length - y); } const T = summary(RULES); row(K, "total", "", T.caught + T.falseAlarm, `${T.caught}/${maliciousCount}`, T.falseAlarm); // Signal source by signal source: source alone and source removed const S = [-20, 13, 14, 14, 19]; console.log(); row(S, "signal source", "alone", "false alarm", "removed", "only this source"); for (const s of SOURCE) { const own = RULES.filter((t) => RULE[t][0] === s); const others = RULES.filter((t) => RULE[t][0] !== s); const alone = summary(own), without = summary(others); row(S, s, `${alone.caught}/${maliciousCount}`, alone.falseAlarm, `${without.caught}/${maliciousCount}`, T.caught - without.caught); } // Events caught by only a single source are named console.log(); for (const x of alert(RULES).filter((x) => x.o.malicious && x.rule.length > 0)) { const sources = [...new Set(x.rule.map((t) => RULE[t][0]))]; if (sources.length === 1) console.log(`"${sources[0]}" only: ${x.o.name}`); } for (const x of alert(RULES).filter((x) => x.o.malicious && x.rule.length === 0)) console.log(`no source catches: ${x.o.name}`); // Collection cost (fictional unit): source|daily GB|unit per GB const COST = Object.fromEntries(parseRows(` audit log|12|0.5 network flow|40|0.3 identity behavior|3|0.8`).map(([s, gb, u]) => [s, { gb: +gb, unit: +u }])); const U = [-20, 10, 12, 13, 18]; console.log(); row(U, "signal source", "GB/day", "unit/GB", "unit/mo", "per only-this"); let totalMonth = 0, totalGb = 0; for (const s of SOURCE) { const month = COST[s].gb * COST[s].unit * 30; const others = RULES.filter((t) => RULE[t][0] !== s); const only = T.caught - summary(others).caught; totalMonth += month; totalGb += COST[s].gb; row(U, s, COST[s].gb, COST[s].unit.toFixed(1), month.toFixed(0), only ? (month / only).toFixed(0) : "-"); } row(U, "total", totalGb, "", totalMonth.toFixed(0), ""); // The behavior a false alarm produces: a rule over the threshold gets closed (same measure as M20/K04) const THRESHOLD = 3; const closedRules = RULES.filter((t) => { const a = alert([t]).filter((x) => x.rule.length > 0); return a.length - a.filter((x) => x.o.malicious).length >= THRESHOLD; }); const after = summary(RULES.filter((t) => !closedRules.includes(t))); console.log(`\nfalse-alarm threshold ${THRESHOLD}: closed rules ${closedRules.join(", ")} ` + `(${closedRules.length}/${RULES.length}); after closing, caught ` + `${T.caught} -> ${after.caught}, missed ${T.missed} -> ${after.missed}, ` + `false alarm ${T.falseAlarm} -> ${after.falseAlarm}`); const open = RULES.filter((t) => !closedRules.includes(t)); const newlyMissed = alert(RULES).filter((x) => x.o.malicious && x.rule.length > 0 && !x.rule.some((t) => open.includes(t))).map((x) => x.o.name); console.log(`added to missed after closing: ${newlyMissed.join(", ")}; ` + `the cost of the sources the closed rules watch (${[...new Set(closedRules.map((t) => RULE[t][0]))].join(", ")}) keeps being paid`); // Limit and remaining (fictional): daily collection cap const LIMIT = 60; console.log(`collection cap ${LIMIT} GB/day; used ${totalGb}, remaining ${LIMIT - totalGb} -> ` + `${LIMIT - totalGb} GB/day of room for a fourth source, network flow doubling would exceed the cap`); ``` ``` 30 known events (11 malicious-tagged), 9 detection rules, 3 signal sources rule signal source alerts caught false alarm T1 audit log 1 1 0 T2 audit log 4 1 3 T3 audit log 2 1 1 T4 network flow 3 1 2 T5 network flow 5 2 3 T6 network flow 2 1 1 T7 identity behavior 5 3 2 T8 identity behavior 2 1 1 T9 identity behavior 3 2 1 total 22 9/11 13 signal source alone false alarm removed only this source audit log 3/11 4 7/11 2 network flow 3/11 6 6/11 3 identity behavior 4/11 3 6/11 3 "audit log" only: bulk-archive-delete "audit log" only: policy-expansion "network flow" only: store-outbound-copy "network flow" only: leak-to-new-destination "identity behavior" only: key-exfiltration "identity behavior" only: brute-force-login "network flow" only: quiet-side-channel "identity behavior" only: shared-account-takeover no source catches: logging-shutdown no source catches: slow-exfiltration signal source GB/day unit/GB unit/mo per only-this audit log 12 0.5 180 90 network flow 40 0.3 360 120 identity behavior 3 0.8 72 24 total 55 612 false-alarm threshold 3: closed rules T2, T5 (2/9); after closing, caught 9 -> 7, missed 2 -> 4, false alarm 13 -> 7 added to missed after closing: bulk-archive-delete, leak-to-new-destination; the cost of the sources the closed rules watch (audit log, network flow) keeps being paid collection cap 60 GB/day; used 55, remaining 5 -> 5 GB/day of room for a fourth source, network flow doubling would exceed the cap ``` The numbers are of the measurement class; their inputs are the assumptions above. Nine rules produce twenty-two alerts over thirty events: nine of eleven malicious events are caught, two get away, thirteen benign events get flagged. The false-alarm count being larger than the caught count comes from how this set was chosen — the benign events were chosen to share surfaces with the malicious ones — but the direction of the ratio is real: detection rules do not run without flagging legitimate work. Rule by rule, the distribution is not even. T1 produces a single alert and has no false ones; it is a narrowly written rule and sees only one event in exchange. T7 gets three of five alerts right. T2 and T5 are the noisiest pair with three false alarms each, and these two numbers come back in the lesson's last section. #### Signal Sources, Separately The real distinction sits in the source table, and it separates two different questions. The first column says what the source catches alone: audit log three, network flow three, identity behavior four. The three add up to ten, but the combined catch is nine; there is overlap. The third column is more useful — what remains once the source is removed. If the audit log is removed, caught drops from nine to seven; if either network flow or identity behavior is removed, it drops to six. The last column is the number that carries the decision: **the event caught only by this source.** Two in the audit log, three in network flow, three in identity behavior. Eight malicious events rest on a single source's shoulders, and they get named. Only the audit log sees the bulk archive delete and the policy expansion; neither leaves a trace on the network, because no data goes out. Only network flow sees the outbound copy to the store, the leak flowing to a new destination, and the quiet side channel; all three are permitted reads by an authorized subject, and they look ordinary in the audit log. Only identity behavior sees the key exfiltration, the brute-force login, and the shared-account takeover. Of the nine caught events, only one — the nightly role chain — shows up in two sources at once. Both missed events sit under the tables, named, and the reasons they got away resemble each other. The logging shutdown is a single policy change made during business hours; T1 could have caught it, but its predicate also checks the hour and it does not fire outside the night. The slow exfiltration is a 45 MB read; well under T4's threshold, and the destination is not new for T5. Both sit on the other side of the rule set's **threshold**, not its **scope** — meaning they can be caught by loosening it. Removing the hour condition from T1 catches the logging shutdown and turns the release rollout into a false alarm; dropping T4's threshold to 40 MB catches the slow exfiltration and raises that rule's false alarms from two to eight. Loosening the threshold moves a missed event to caught, but for every event it moves, it writes more than one item into the false-alarm column; the next section counts what those items do. The sentence this leads to is this: **the three sources are not backups for each other.** Cutting a source because of its cost turns the events that source alone saw directly into missed events. Which source can be cut is decided by looking at this column, not at its cost. #### Detection's Two Costs The first cost is the charge table. The three sources produce 55 GB a day and cost 612 fictional units a month. The line items are not equal: network flow takes three quarters of the volume and 360 units of the charge, identity behavior takes three GB for 72 units. The last column makes the two comparable — the monthly charge per event caught only by that source: identity behavior 24 units, audit log 90, network flow 120. Network flow is the most expensive source and the only thing that alone sees three events; both are true at once, and the decision is made between these two numbers. The second cost is not paid in currency. When the false-alarm threshold is set to three, T2 and T5 close; both had produced three false alarms each. After closing, caught goes from nine to seven, missed goes from two to four, false alarm drops from thirteen to seven. The two events added to missed are named: the bulk archive delete and the leak to a new destination. Both were events only the closed rule saw. This measure has the same shape as the one M20/K04 used for fitness functions — every setting that raises catch rate also raises false alarms, and the behavior a false alarm produces is the rule getting closed — the relationship is stated in one sentence and moved past. This second cost needs underlining: when a rule closes, collection does not stop. The audit log and network flow charge keep being paid exactly as before; what gets cut is not the charge, it is the visibility the charge bought. The decision to close is also made by the team reading the alerts, so detection's real scope is not the written rule set — it is the rule set **left open**, and the gap between the two shows up on no bill. Little remains on the limit side: the daily collection cap is 60 GB, used is 55, remaining is 5. There is room for 5 GB for a fourth signal source, and if network flow doubles, the cap is exceeded. Adding a source and an existing source's volume growing both run into the same limit. #### Summary - Nine rules produce twenty-two alerts over thirty events: nine of eleven malicious events are caught, two leave no trace in any source, thirteen benign events get flagged. - Eight of the nine caught events are caught by only a single signal source; only one shows up in two sources at once. The three sources are not backups for each other. - The cost of removing a source is in the table: if the audit log is removed, caught drops from nine to seven; if either network flow or identity behavior is removed, it drops to six. - The collection charge is 612 fictional units a month; per event caught only by that source, identity behavior costs 24, audit log 90, network flow 120 units. - At a false-alarm threshold of three, two rules close: caught goes from 9 to 7, missed goes from 2 to 4, false alarm drops from 13 to 7; the charge for the closed rules' sources keeps being paid. - The collection cap is 60 GB/day, used is 55, remaining is 5; a fourth source or network flow's growth runs into the same limit. #### Next Step Detection rules ran, caught and missed were counted, and the closing of two rules was recorded too. All of these numbers look inward: the team's own measure, its own threshold, its own decision. When someone outside says "show that access is restricted and unusual activity is monitored," what is being asked for is not a measure but **evidence**, and part of that evidence is not even in the tenant's hands. The next lesson takes compliance requirements by type and separates, for every requirement, where the evidence comes from — the provider's document or the tenant's own record — and counts in which requirements the evidence stays in the gap. ### Mapping Compliance Requirements to the Cloud Source: https://academia.sh/en/courses/cloud-governance/mapping-compliance-requirements-to-the-cloud Turning compliance requirements into evidence in the rented layer: across 12 requirement types and 26 evidence items, 7 items come from the provider's document and 16 from the tenant's own record, 3 requirements' evidence stays in the gap, 7 of the 14 items sitting outside the tenant's layer have evidence that does not come from the provider, and producing evidence costs 139 fictional units and 160 person-hours a year. The previous lesson ran detection rules and produced three numbers: caught, missed, false alarm. All three were the team's own measure — the same side set the threshold and made the decision to close a rule. When someone outside says "show that access is restricted and unusual activity is monitored," what is being asked for is not these numbers. What is being asked for is **evidence**: a record or document that shows, in a form presentable to someone who did not set that obligation, that the obligation is met. Turning external constraints into runnable checks was measured in M20/K04; there, evidence took the place of loosening, and all of the evidence was the architecture's own output. In the rented layer, this splits. Part of the requirements are met on the provider's machine, facility, or personnel; the tenant has no record it can produce there, and the **provider's document** stands in as evidence. This lesson's measure is that split. Requirements here are referred to **by type** — data residency constraint, retention obligation, right to deletion, access-logging obligation, privileged-access evidence — and no regulation, standard, certification, or framework name is written. #### Requirements and Evidence Items The example course is the fictional **regional measurement network**. **CS43 — requirement catalog:** twelve requirement types. **CS44 — evidence-item table:** twenty-six items; each item carries which layer it sits in (provider, shared, tenant), who produces its evidence (the provider's document, the tenant's own record, none), its freshness in days, its daily line count, and its retention period. **CS45 — requirement-item mapping:** thirty-two links; one item can serve more than one requirement. **CS46 — record units:** 200 bytes per line, 1.5 fictional units a month per GB. **CS47 — report generation:** every requirement is compiled once a quarter, 2 person-hours fixed per requirement plus 0.5 person-hours per item. **CS48 — evidence store cap:** 12 GB (fictional unit). The file below builds the tables and separates evidence by its producer. It is a process-internal model; no real document, record, or charge exists. ```js // compliance/evidence.mjs — turning compliance requirements into evidence in the rented layer. // Process-internal model: the regional measurement network (fictional); no real document, record, or charge exists. const parseRows = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); const row = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); // Evidence item: name|layer|producer (provider|own|none)|freshness (days)|daily lines|retention (days) const ITEM = Object.fromEntries(parseRows(` resource-inventory|tenant|own|1|400|1095 region-tag|tenant|own|1|400|1095 data-flow-map|tenant|none|0|0|0 facility-region-attestation|provider|provider|365|0|0 retention-policy-record|tenant|own|1|150|1095 lifecycle-run-record|tenant|own|1|900|1095 backup-copy-inventory|shared|own|7|60|1095 deletion-request-record|tenant|own|1|30|2555 deletion-propagation-record|tenant|own|1|120|2555 backup-deletion-confirmation|shared|none|0|0|0 audit-log|shared|own|1|9000|2555 data-plane-access-record|provider|own|1|4000|2555 provider-personnel-access-attestation|provider|provider|365|0|0 privileged-session-record|tenant|own|1|200|2555 role-assumption-record|tenant|own|1|600|2555 encryption-configuration-scan|shared|own|1|300|1095 key-ownership-attestation|provider|provider|365|0|0 transit-endpoint-scan|tenant|own|1|250|1095 client-verification-record|tenant|none|0|0|0 key-rotation-record|shared|own|1|20|2555 key-material-destruction-attestation|provider|provider|365|0|0 facility-access-control-attestation|provider|provider|365|0|0 hardware-destruction-attestation|provider|provider|365|0|0 restore-drill-record|tenant|own|90|10|1095 incident-timeline|shared|own|1|80|2555 provider-incident-notification|provider|provider|30|0|0`) .map(([name, layer, producer, fresh, lines, retention]) => [name, { layer, producer, fresh: +fresh, lines: +lines, retention: +retention }])); // Requirement type -> evidence items. No regulation, standard, or framework name. const REQUIREMENT = parseRows(` G01 data residency constraint|resource-inventory region-tag data-flow-map facility-region-attestation G02 retention lower bound|retention-policy-record lifecycle-run-record backup-copy-inventory G03 retention upper bound|retention-policy-record lifecycle-run-record G04 right to deletion|deletion-request-record deletion-propagation-record backup-deletion-confirmation G05 access-logging obligation|audit-log data-plane-access-record G06 privileged-access evidence|privileged-session-record role-assumption-record provider-personnel-access-attestation audit-log G07 encryption-at-rest evidence|encryption-configuration-scan key-ownership-attestation G08 encryption-in-transit evidence|transit-endpoint-scan client-verification-record G09 key-rotation evidence|key-rotation-record key-ownership-attestation key-material-destruction-attestation G10 physical-access-and-destruction evidence|facility-access-control-attestation hardware-destruction-attestation G11 recoverability evidence|restore-drill-record backup-copy-inventory G12 incident-notification evidence|incident-timeline provider-incident-notification audit-log`) .map(([name, k]) => ({ name, item: k.split(" ") })); const LAYER = ["provider", "shared", "tenant"]; const PRODUCER = ["provider", "own", "none"]; console.log(`${REQUIREMENT.length} requirement types, ${Object.keys(ITEM).length} evidence items, ` + `${REQUIREMENT.reduce((t, g) => t + g.item.length, 0)} requirement-item links`); // Per requirement: who the evidence comes from, whether a gap exists, how old the oldest evidence is const G = [-47, 8, 16, 8, 8, 15]; console.log(); row(G, "requirement type", "items", "from provider", "own", "gap", "oldest (days)"); const status = {}; for (const g of REQUIREMENT) { const count = (u) => g.item.filter((k) => ITEM[k].producer === u).length; const produced = g.item.filter((k) => ITEM[k].producer !== "none"); const fresh = Math.max(...produced.map((k) => ITEM[k].fresh)); status[g.name] = { gap: count("none"), fresh }; row(G, g.name, g.item.length, count("provider"), count("own"), count("none"), fresh); } const gapped = REQUIREMENT.filter((g) => status[g.name].gap > 0); console.log(`\nrequirements whose evidence stays in the gap: ${gapped.length}/${REQUIREMENT.length} -> ` + gapped.map((g) => `${g.name.split(" ")[0]} (${g.item.filter((k) => ITEM[k].producer === "none").join(", ")})`).join("; ")); // The shared-responsibility boundary redrawn on the evidence side: layer x producer const C = [-18, 12, 10, 8, 9]; console.log(); row(C, "layer \\ producer", "provider", "own", "none", "total"); for (const lt of LAYER) { const set = Object.values(ITEM).filter((k) => k.layer === lt); row(C, lt, ...PRODUCER.map((u) => set.filter((k) => k.producer === u).length), set.length); } row(C, "total", ...PRODUCER.map((u) => Object.values(ITEM).filter((k) => k.producer === u).length), Object.keys(ITEM).length); const outsideTenant = Object.entries(ITEM).filter(([, k]) => k.layer !== "tenant"); const outsideNotProvider = outsideTenant.filter(([, k]) => k.producer !== "provider"); console.log(`of the ${outsideTenant.length} items that sit outside the tenant's layer, ` + `${outsideNotProvider.length} have evidence that does not come from the provider: ` + `${outsideNotProvider.map(([a]) => a).join(", ")}`); // The ongoing cost of producing evidence (fictional unit) const BYTES_PER_LINE = 200, GB = 1e9, UNIT_PER_GB = 1.5; const own = Object.entries(ITEM).filter(([, k]) => k.producer === "own"); const volume = own.reduce((t, [, k]) => t + (k.lines * k.retention * BYTES_PER_LINE) / GB, 0); const dailyLines = own.reduce((t, [, k]) => t + k.lines, 0); console.log(`\n${own.length} items coming from the tenant's own record: ` + `${dailyLines.toLocaleString("en-US")} lines a day; by the end of their retention period ` + `${volume.toFixed(2)} GB, ${(volume * UNIT_PER_GB).toFixed(2)} units a month, ` + `${(volume * UNIT_PER_GB * 12).toFixed(0)} units a year of storage`); // Report generation: every requirement is compiled once a quarter const [QUARTER, FIXED, ITEM_HOURS] = [4, 2, 0.5]; const personHours = REQUIREMENT.reduce((t, g) => t + QUARTER * (FIXED + g.item.length * ITEM_HOURS), 0); console.log(`report generation: ${REQUIREMENT.length} requirements x ${QUARTER} quarters x ` + `(${FIXED} + items x ${ITEM_HOURS}) = ${personHours} person-hours/year`); const attestation = Object.values(ITEM).filter((k) => k.producer === "provider"); console.log(`freshness of the ${attestation.length} documents coming from the provider: ` + `${Math.min(...attestation.map((k) => k.fresh))}-${Math.max(...attestation.map((k) => k.fresh))} days; ` + `freshness of the tenant's own record: 1-90 days`); // Limit and remaining (fictional): the evidence store's cap const LIMIT = 12; console.log(`evidence store cap ${LIMIT} GB; used ${volume.toFixed(2)}, remaining ` + `${(LIMIT - volume).toFixed(2)} -> the audit log's daily line count can still grow by ` + `${Math.floor(((LIMIT - volume) * GB) / (BYTES_PER_LINE * ITEM["audit-log"].retention)) .toLocaleString("en-US")} lines`); ``` ``` 12 requirement types, 26 evidence items, 32 requirement-item links requirement type items from provider own gap oldest (days) G01 data residency constraint 4 1 2 1 365 G02 retention lower bound 3 0 3 0 7 G03 retention upper bound 2 0 2 0 1 G04 right to deletion 3 0 2 1 1 G05 access-logging obligation 2 0 2 0 1 G06 privileged-access evidence 4 1 3 0 365 G07 encryption-at-rest evidence 2 1 1 0 365 G08 encryption-in-transit evidence 2 0 1 1 1 G09 key-rotation evidence 3 2 1 0 365 G10 physical-access-and-destruction evidence 2 2 0 0 365 G11 recoverability evidence 2 0 2 0 90 G12 incident-notification evidence 3 1 2 0 30 requirements whose evidence stays in the gap: 3/12 -> G01 (data-flow-map); G04 (backup-deletion-confirmation); G08 (client-verification-record) layer \ producer provider own none total provider 7 1 0 8 shared 0 5 1 6 tenant 0 10 2 12 total 7 16 3 26 of the 14 items that sit outside the tenant's layer, 7 have evidence that does not come from the provider: backup-copy-inventory, backup-deletion-confirmation, audit-log, data-plane-access-record, encryption-configuration-scan, key-rotation-record, incident-timeline 16 items coming from the tenant's own record: 16,520 lines a day; by the end of their retention period 7.72 GB, 11.58 units a month, 139 units a year of storage report generation: 12 requirements x 4 quarters x (2 + items x 0.5) = 160 person-hours/year freshness of the 7 documents coming from the provider: 30-365 days; freshness of the tenant's own record: 1-90 days evidence store cap 12 GB; used 7.72, remaining 4.28 -> the audit log's daily line count can still grow by 8,374 lines ``` The numbers are of the measurement class; their inputs are the assumptions above. Twelve requirements resolve into twenty-six evidence items and thirty-two links. Of the twenty-six items, seven come from the provider's document, sixteen from the tenant's own record, and three have no producer. Item count is larger than requirement count, and link count is larger than item count; a single item serves more than one obligation — the audit log appears in three separate requirements, the key ownership attestation in two. #### Evidence's Two Producers The first place worth reading in the requirement table is the second and third columns. G10 sits entirely on the provider's side: both of its two items are documents, and the tenant has no record it could produce there. G03 and G05 sit entirely on the tenant's side. The remaining nine requirements are mixed, and their evidence file contains pieces of two different types. This decides the practical shape of compliance work: meeting a requirement is not one owner's job — it is assembling pieces coming from two producers. The last column shows this assembly's weak link. **A requirement's evidence is as fresh as its oldest piece.** For G05, freshness is one day, because both of its items are records kept daily. For G01, G06, G07, G09, and G10, freshness is 365 days; each of these five requirements holds a provider document renewed once a year, and the tenant's daily records do not bring the file's age down. A gap of up to a year can remain between an event and the renewal of the document that covers it; the provider also sets the renewal calendar, so this is a devolved decision, and it is an item the tenant cannot speed up. What the two producers produce is not the same kind of thing, and this distinction comes after the numbers. The tenant's own record can be queried: give it a date range, filter to a subject, pull an answer to a different question out of the same record, and recompile it retroactively if needed. The provider's document is a closed output — it is accepted in the form it is given, no query can be written into it, its scope cannot be extended, and a second version cannot be requested for a resource it does not cover. Sixteen items are of the first type, seven of the second. The practical consequence is this: when an auditor asks a question that is not in the table, the tenant has a chance of producing an answer for sixteen items and none for seven; for those seven, the answer that can be given is limited to what is written in the document. Evidence stays in the gap for three requirements, and all three get named. In G01, the data flow map is missing: there is no record showing which data flows to which region, so the link between the tag in the inventory and the facility attestation cannot be established. In G04, the backup deletion confirmation is missing: the deletion request gets recorded and propagated, but no record is produced showing the deletion actually happened in the backup copies. In G08, the client verification record is missing: the endpoint scan shows the server side, and there is no evidence the client actually performed verification. What the three have in common is this: what is missing is not a document, it is a **record flow that was never set up**. No one refused it, no one withheld it — no one asked for it. #### The Boundary Redrawn on the Evidence Side The cross table is this lesson's main measure. The layer columns say where responsibility sits, the rows say who the evidence comes from, and the two do not line up. Of the eight items sitting in the provider's layer, seven are covered by a document, one is not: the data-plane access record happens entirely inside the provider, but the one who produces its evidence is the tenant, by keeping that logging turned on. Of the six items in the shared layer, **none** come from the provider's document; five come from the tenant's record, one comes from neither. The total collapses this to a single line: of the fourteen items sitting outside the tenant's layer, seven have evidence that does not come from the provider. In other words, the responsibility split does not carry over to the evidence side as is; **the evidence boundary has shifted toward the tenant relative to the layer boundary.** The backup copy inventory, the audit log, the encryption configuration scan, the key rotation record, and the incident timeline are work that happens in the shared layer, but it is the tenant who will do the showing when showing is required. This distinction is not visible in the contract; it is visible in who compiles the evidence file — and this is exactly where the compilation burden is born. #### The Ongoing Cost of Producing Evidence Evidence is not produced once and done; it is an ongoing line item, and it is counted under two headings. The sixteen items coming from the tenant's own record produce 16,520 lines a day; by the end of their retention periods they hold 7.72 GB, costing 11.58 fictional units a month and 139 a year in storage. The second heading is heavier: each of the twelve requirements is compiled once a quarter, and that runs 160 person-hours a year. Putting the two line items side by side is this lesson's result — evidence's ongoing cost is not mostly storage, it is **compilation labor**, and it grows per item, not per requirement. M20/K04's closed loop applies here too, and it adds one word: evidence is also a data class, subject to its own retention period and its own residency constraint. The word it adds is this: **the tenant does not produce half of this data class.** The freshness of the seven documents coming from the provider runs 30 to 365 days, and that range is not the tenant's choice; the freshness of the tenant's own record runs 1 to 90 days, and that range is the tenant's choice. How fresh the evidence file can be is decided by both sides at once. The limit side is counted too. The evidence store's cap is 12 GB, used is 7.72, remaining is 4.28. That is enough for the audit log's daily line count to grow by 8,374 more lines; past that, either the retention period has to be shortened — and retention period is itself a requirement — or the cap has to be grown, which gets written onto the bill. #### Summary - Twelve requirement types resolve into twenty-six evidence items and thirty-two links; seven of the items come from the provider's document, sixteen from the tenant's own record, three have no producer. - Three requirements' evidence stays in the gap (data flow map, backup deletion confirmation, client verification record); for all three, what is missing is not a document, it is a record flow that was never set up. - A requirement's evidence is as fresh as its oldest piece: five requirements have a freshness of 365 days, because each holds a provider document renewed once a year, and the renewal calendar is not the tenant's. - The shared-responsibility boundary does not stand in the same place on the evidence side: seven of the fourteen items sitting outside the tenant's layer have evidence that does not come from the provider, and none of the six items in the shared layer are covered by a document. - The ongoing cost of producing evidence is two line items: 16,520 lines/day and 7.72 GB, costing 139 fictional units a year in storage, plus 160 person-hours a year of report compilation. The second is the heavy one. - 7.72 of the evidence store's 12 GB cap is full; the remaining 4.28 GB leaves room for the audit log to grow by 8,374 lines a day. #### Next Step The topic closes here, and the table it leaves behind fits into a single sentence: in the rented layer, security is narrowing who can do what, changing what the data shows when it is read, and being able to show that both were done. All three were given a number — how many legitimate jobs narrowing stops, how many access records isolation breaks, how many detection misses, and in which requirement the evidence stays in the gap. The one thing that was not given a number is their sum. Compliance auditing consumes a quota, private endpoints consume a monthly rent per service and per network, signal collection consumes a per-GB charge, evidence retention consumes a store limit; each was written in fictional units in its own section, but none was set beside another. Access was narrowed, data was encrypted, and evidence was produced — all of it has a bill, and that bill was never read across this topic. The next topic opens the bill line by line: which item belongs to what, which team it falls on, and which share cannot be assigned to any team at all. ## Cost and Limits Reading the bill and planning the ceiling: a cost model made of compute, storage, transfer, and request fees; tagging with ownership-based cost allocation; a budget and alerts that catch unexpected spending early; resource sizing to reduce overprovisioning; quotas and service limits in scale planning; and evaluating vendor lock-in through portability and exit cost. ### Cost Model Source: https://academia.sh/en/courses/cloud-governance/cost-model The bill for the rented layer splits into four items: of the monthly 1,542 fictional units, 37.7% is compute, 31.8% is transfer, 20.2% is request, and only 10.3% is storage; the request fee is 1.95 times the storage fee. When the same bill is mapped to eleven design decisions, more than half of the decisions (50.8%) belong to the platform, and four spread across more than one item. In the raw measurement stream, the point where the request fee overtakes storage comes out at 693,370 items/month (11,364 bytes); the network writes 152 bytes per item, sitting 74.8 times past that point, and pays 299.43 units of request fee against 6.48 units of storage. The previous topic narrowed access, encrypted data, and produced evidence. All three consumed a resource: the key management service was called, an audit log was written, private endpoints were opened, a secret was rotated. None of this was free, and none of it was counted. This topic reads the bill. The first question is the plainest one: what items make up the monthly total, which design decision does each item come from, and does the bill name that decision? #### Four Items and the Decision Behind Them The example is the fictional **regional measurement network**: five teams, three environments, a rented machine, network, store, function, and edge layer. Items, quantities, and unit prices are fictional; **the currency is fictional too** and is called `unit`, corresponding to no real price list. **CC1:** the bill carries four item types — compute, storage, transfer, request — and gives fourteen lines. **CC2:** every line has a measure, a monthly quantity, and a price per measure. **CC3:** every line comes from a **design decision** with an owner; the bill carries neither field, both are kept in the model. **CC4:** the raw measurement stream's body volume is fixed — the number of **items** it is split into does not change the volume, only the item metadata and request count. **CC5:** data written in a given month is paid for across its retention period. **CC6:** the bill comes at month and line-item granularity; it carries no split by resource, team, or environment. The bill, items, and stream are a process-internal **model**; there is no real bill. ```js // cost/bill.mjs — monthly bill model: no real bill. Items, quantities, and unit prices are a // data structure; the currency is fictional and is called `unit`, corresponding to no real // price list. Each row is tied to a *bill item*, and also to the *design decision* that caused // it and the party who made that decision; the bill never carries this second link at all. // [item type, line name, measure, monthly quantity, price per measure (unit), decision, decision owner] export const BILL = [ ["compute", "rented machine", "machine-hour", 8640, 0.062, "all three environments are always on", "platform"], ["compute", "function call duration", "thousand GB-seconds", 41.6, 0.850, "a function per event", "application"], ["compute", "edge layer processing", "thousand calls", 930, 0.011, "pre-processing at the edge", "application"], ["storage", "raw measurement body", "GB-month", 189.1, 0.021, "raw measurement kept 24 months", "governance"], ["storage", "item metadata", "GB-month", 119.4, 0.021, "each read writes a separate item", "application"], ["storage", "processed series", "GB-month", 1420, 0.019, "every total is precomputed", "reporting"], ["storage", "snapshot and backup", "GB-month", 2140, 0.048, "daily snapshot, 35 days", "governance"], ["storage", "record store", "GB-month", 1360, 0.017, "every request is logged", "governance"], ["transfer", "out-of-region egress", "GB", 2960, 0.082, "reports are pulled from outside the region", "reporting"], ["transfer", "cross-region copy", "GB", 4400, 0.019, "continuous copy to a second region", "platform"], ["transfer", "within-region cross-zone", "GB", 18200, 0.009, "every write fans out to three regions", "platform"], ["request", "object write", "million requests", 51.84, 4.300, "each read writes a separate item", "application"], ["request", "object read", "million requests", 212.544, 0.360, "reports are pulled from outside the region", "reporting"], ["request", "function call", "million requests", 62.1, 0.190, "a function per event", "application"], ]; export const TYPE = ["compute", "storage", "transfer", "request"]; export const unit = (r) => r[3] * r[4]; export const sum = (v) => v.reduce((a, b) => a + b, 0); // raw measurement stream: volume fixed, the only thing that changes is how many *items* it is split into (CC5) export const STREAM = { counter: 18000, readsPerDay: 96, days: 30, // monthly read count derives from this bodyBytes: 152, metadataBytes: 96, // every item also carries metadata retentionMonths: 24, storageUnit: 0.021, // this month's data is paid for 24 months writeUnit: 4.30, readUnit: 0.36, readMultiple: 4.1, }; export const bodyGB = () => STREAM.counter * STREAM.readsPerDay * STREAM.days * STREAM.bodyBytes / 1e9; // M: monthly item count. Body volume is independent of M; metadata grows with M. export const storageUnits = (M) => (bodyGB() + M * STREAM.metadataBytes / 1e9) * STREAM.retentionMonths * STREAM.storageUnit; export const requestUnits = (M) => M / 1e6 * (STREAM.writeUnit + STREAM.readMultiple * STREAM.readUnit); // both lines are linear in M: the crossover comes out in closed form export const crossover = () => { const a = (STREAM.writeUnit + STREAM.readMultiple * STREAM.readUnit) / 1e6; const b = STREAM.metadataBytes / 1e9 * STREAM.retentionMonths * STREAM.storageUnit; return bodyGB() * STREAM.retentionMonths * STREAM.storageUnit / (a - b); }; ``` ```js // cost/measure.mjs — split into four items, each item mapped to the decision that caused it, // and the point where the request fee overtakes the storage fee. What is measured is not the // bill itself, it is the bill's *resolution*. import { BILL, TYPE, STREAM, unit, sum, bodyGB, storageUnits, requestUnits, crossover } from "./bill.mjs"; const write = (w, ...s) => console.log(s.map((v, i) => (w[i] < 0 ? String(v).padEnd(-w[i]) : String(v).padStart(w[i]))).join("")); const t0 = process.hrtime.bigint(); const T = sum(BILL.map(unit)), share = (x) => `${(100 * x / T).toFixed(1)}%`; console.log(`${BILL.length} bill lines, monthly ${T.toFixed(0)} units (fictional currency)`); console.log("\n1. four items: what the bill says"); const A = [-26, -21, 11, 12, 11, 8]; write(A, "line", "measure", "qty", "price/unit", "monthly", "share"); for (const t of TYPE) { const s = BILL.filter((r) => r[0] === t); for (const r of s) write(A, r[1], r[2], r[3], r[4].toFixed(3), unit(r).toFixed(1), share(unit(r))); const subtotal = sum(s.map(unit)); write(A, ` ${t} total`, "-", "-", "-", subtotal.toFixed(1), share(subtotal)); } const T4 = TYPE.map((t) => sum(BILL.filter((r) => r[0] === t).map(unit))); console.log(` request ${T4[3].toFixed(1)} units, storage ${T4[1].toFixed(1)} units: ` + `ratio ${(T4[3] / T4[1]).toFixed(2)}`); console.log("\n2. the same bill mapped to decisions: a line item does not name a decision"); const K = new Map(); for (const r of BILL) { const k = K.get(r[5]) ?? { unit: 0, types: new Set(), owner: r[6] }; k.unit += unit(r); k.types.add(r[0]); K.set(r[5], k); } const B = [-44, -13, 22, 11, 7]; write(B, "decision", "owner", "which items", "monthly", "share"); const sorted = [...K.entries()].sort((a, b) => b[1].unit - a[1].unit); for (const [name, k] of sorted) write(B, name, k.owner, [...k.types].join("+"), k.unit.toFixed(1), share(k.unit)); const multi = sorted.filter(([, k]) => k.types.size > 1); console.log(` ${K.size} decisions, ${BILL.length} bill lines; ${multi.length} decisions spread ` + `across more than one item type`); const OWNER = new Map(); for (const [, k] of K) OWNER.set(k.owner, (OWNER.get(k.owner) ?? 0) + k.unit); const ownerSorted = [...OWNER.entries()].sort((a, b) => b[1] - a[1]); const ownerLines = ownerSorted.map(([o, v]) => `${o} ${v.toFixed(1)} units (${share(v)})`); console.log(` by decision owner: ${ownerLines.slice(0, 2).join(", ")},`); console.log(` ${ownerLines.slice(2).join(", ")}`); console.log("\n3. where the request fee overtakes the storage fee (raw measurement stream)"); const actualReads = STREAM.counter * STREAM.readsPerDay * STREAM.days; console.log(` volume fixed: ${bodyGB().toFixed(2)} GB/month body, ` + `${(actualReads / 1e6).toFixed(2)} million reads/month`); const C = [15, 12, 12, 12, 10, 11]; write(C, "items/month", "bytes/item", "reads/item", "storage", "request", "dominant"); for (const M of [1e4, 1e5, 3e5, 693372, 1e6, 5e6, 51.84e6, 2e8]) { const d = storageUnits(M), i = requestUnits(M); write(C, M.toLocaleString("en-US"), (bodyGB() * 1e9 / M).toFixed(0), (actualReads / M).toFixed(2), d.toFixed(2), i.toFixed(2), i > d ? "request" : "storage"); } const M0 = crossover(); console.log(` crossover ${Math.round(M0).toLocaleString("en-US")} items/month: ` + `${(bodyGB() * 1e9 / M0).toFixed(0)} bytes/item, ` + `${(actualReads / M0).toFixed(1)} reads/item`); const Mv = actualReads; console.log(` the network's choice, a separate item per read: ${(Mv / M0).toFixed(1)}x past the crossover;`); console.log(` storage ${storageUnits(Mv).toFixed(2)} units, request ${requestUnits(Mv).toFixed(2)} units, ` + `ratio ${(requestUnits(Mv) / storageUnits(Mv)).toFixed(1)}`); console.log("\n4. the bill's resolution: how many splits it gives, how many are needed"); const D = [-36, 8, 18]; const TEAM = 5, ENV = 3, RESOURCE = 6; write(D, "split", "count", "source"); write(D, "bill lines (monthly)", BILL.length, "bill"); write(D, "item types", TYPE.length, "bill"); write(D, "design decisions", K.size, "not in the bill"); write(D, "deciding party", OWNER.size, "not in the bill"); write(D, "team x environment x resource type", TEAM * ENV * RESOURCE, "not in the bill"); const remaining = TEAM * ENV * RESOURCE - BILL.length; console.log(` the bill gives ${BILL.length} splits, the network requires ${TEAM * ENV * RESOURCE} units ` + `of separation: ${remaining} units cannot be read from the bill`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, only the band is checked console.log(`\nfour tables within the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`); ``` ``` 14 bill lines, monthly 1542 units (fictional currency) 1. four items: what the bill says line measure qty price/unit monthly share rented machine machine-hour 8640 0.062 535.7 34.7% function call duration thousand GB-seconds 41.6 0.850 35.4 2.3% edge layer processing thousand calls 930 0.011 10.2 0.7% compute total - - - 581.3 37.7% raw measurement body GB-month 189.1 0.021 4.0 0.3% item metadata GB-month 119.4 0.021 2.5 0.2% processed series GB-month 1420 0.019 27.0 1.7% snapshot and backup GB-month 2140 0.048 102.7 6.7% record store GB-month 1360 0.017 23.1 1.5% storage total - - - 159.3 10.3% out-of-region egress GB 2960 0.082 242.7 15.7% cross-region copy GB 4400 0.019 83.6 5.4% within-region cross-zone GB 18200 0.009 163.8 10.6% transfer total - - - 490.1 31.8% object write million requests 51.84 4.300 222.9 14.5% object read million requests 212.544 0.360 76.5 5.0% function call million requests 62.1 0.190 11.8 0.8% request total - - - 311.2 20.2% request 311.2 units, storage 159.3 units: ratio 1.95 2. the same bill mapped to decisions: a line item does not name a decision decision owner which items monthly share all three environments are always on platform compute 535.7 34.7% reports are pulled from outside the region reporting transfer+request 319.2 20.7% each read writes a separate item application storage+request 225.4 14.6% every write fans out to three regions platform transfer 163.8 10.6% daily snapshot, 35 days governance storage 102.7 6.7% continuous copy to a second region platform transfer 83.6 5.4% a function per event application compute+request 47.2 3.1% every total is precomputed reporting storage 27.0 1.7% every request is logged governance storage 23.1 1.5% pre-processing at the edge application compute 10.2 0.7% raw measurement kept 24 months governance storage 4.0 0.3% 11 decisions, 14 bill lines; 3 decisions spread across more than one item type by decision owner: platform 783.1 units (50.8%), reporting 346.2 units (22.5%), application 282.8 units (18.3%), governance 129.8 units (8.4%) 3. where the request fee overtakes the storage fee (raw measurement stream) volume fixed: 7.88 GB/month body, 51.84 million reads/month items/month bytes/item reads/item storage request dominant 10,000 787968 5184.00 3.97 0.06 storage 100,000 78797 518.40 3.98 0.58 storage 300,000 26266 172.80 3.99 1.73 storage 693,372 11364 74.77 4.00 4.00 request 1,000,000 7880 51.84 4.02 5.78 request 5,000,000 1576 10.37 4.21 28.88 request 51,840,000 152 1.00 6.48 299.43 request 200,000,000 39 0.26 13.65 1155.20 request crossover 693,370 items/month: 11364 bytes/item, 74.8 reads/item the network's choice, a separate item per read: 74.8x past the crossover; storage 6.48 units, request 299.43 units, ratio 46.2 4. the bill's resolution: how many splits it gives, how many are needed split count source bill lines (monthly) 14 bill item types 4 bill design decisions 11 not in the bill deciding party 4 not in the bill team x environment x resource type 90 not in the bill the bill gives 14 splits, the network requires 90 units of separation: 76 units cannot be read from the bill four tables within the 2000 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### The Bill's Shape The first table splits the monthly 1,542 units into four items. Compute is the largest at **37.7%**, and almost all of it comes from a single line: the three environments' rented machines, 535.7 units, **34.7% of the bill**. Transfer is second at **31.8%**. Request is **20.2%**, and storage is only **10.3%**. The ranking flips an expectation: in a network that stores data, the smallest item should be storage, and instead **the request fee is 1.95 times storage**. The object-write line alone runs 222.9 units; storage's five lines together total 159.3 units. A small unit price does not mean a small item either. Within-region cross-zone transfer, at 0.009 units per GB, is the cheapest unit price on the bill; at 18,200 GB it holds **10.6% of the bill** — more than the whole of storage. What is cheap gets used heavily, and heavy use grows. #### A Bill Item Does Not Name a Decision The second table re-totals the same amount by the decision that caused each line. Fourteen bill lines collapse to **eleven decisions**, and three of them do not fit inside a single item type: "reports are pulled from outside the region" shows up in both transfer and request (319.2 units), "each read writes a separate item" in both storage and request (225.4 units), "a function per event" in both compute and request (47.2 units). This is why reading the bill does not produce a decision. Read as **20.2%**, the request item is not one thing: it holds the residue of three decisions, each of which also marks another item. There is nothing inside the request item itself that shrinks it; the decision has to change. By decision owner the total sharpens further. **50.8% of the bill** comes from three decisions made by the platform: machines staying continuously on, every write spreading across three regions, the continuous copy to a second region. Reporting is **22.5%**, application **18.3%**, governance **8.4%**. Showing the bill to the teams and expecting cuts starts from a place where half the total is no team's decision. #### The Point Where the Small Fee Turns Dominant The third table isolates a single stream. The raw measurement stream produces 51.84 million reads and **7.88 GB** of body a month; that volume is fixed. The only thing that changes is how many items it is split into. The scan lays two curves side by side. Storage is nearly flat: 3.97 units at ten thousand items, 13.65 units at two hundred million — the growth is all item metadata. Request, by contrast, is **linear and starts at zero**: 0.06 units at ten thousand items, 1,155.20 at two hundred million. The two curves cross at **693,370 items/month**. That is **11,364 bytes** per item, roughly **74.8 reads**. Below that point storage dominates and the request fee is noise; above it, the reverse holds. The network writes every read as a separate item: 152 bytes per item, **74.8 times** past the crossover. There, storage is 6.48 units, request is **299.43 units** — forty-six times as much. The bill writes the per-request fee as 4.30 units/million, too small to measure for one operation; it becomes dominant not from price but from **count**. The design decision behind this fee does not look like a storage decision, it is a **splitting** decision, and the bill never uses that name. #### The Bill's Resolution The fourth table counts the limit. The bill gives fourteen lines a month and four item types. The network, with five teams, three environments, and six resource types, requires separating **90 units**. No mapping takes fourteen lines to ninety units: **76 units cannot be read from the bill**. Neither the deciding party nor the decision itself is on the bill; both are known only in the model. As it stands, the bill is not an accounting document, it is a **total**; if ownership is not on it, ownership must come from elsewhere. #### Summary - The fictional network's monthly bill of 1,542 units splits into four items: compute 37.7%, transfer 31.8%, request 20.2%, storage 10.3%. In a network that stores data, **the request fee is 1.95 times storage**. - A small unit price does not mean a small item: the cheapest transfer line, at 0.009 units per GB, holds 10.6% of the bill at 18,200 GB. - Fourteen bill lines come from eleven design decisions, and three spread across more than one item; shrinking an item takes work on the decision, not the item. - **50.8% of the bill** comes from three platform decisions; application teams' decisions are 18.3%. Expecting the cut from teams leaves out half the total. - In the fixed-volume raw measurement stream, the request fee overtakes storage at **693,370 items/month** — 11,364 bytes per item. The network writes 152-byte items, sitting 74.8 times past the crossover, and pays **299.43 units** of request fee against 6.48 units of storage. - The bill gives fourteen splits, the network requires 90 units of separation: **76 units cannot be read from the bill**. #### Next Step The bill's total has been read and its items split, but no item yet has an owner. The next lesson brings ownership in from outside the bill: resources get tagged, and cost is distributed to teams from that tag. What gets measured is not the distribution itself, it is where the distribution **does not hold** — the share held by an untagged resource, which rule splits a shared resource, and what happens to the past bill once tagging becomes mandatory. ### Tagging and Cost Allocation Source: https://academia.sh/en/courses/cloud-governance/tagging-and-cost-allocation Ownership information comes from outside the bill, and it does not come in full: in a fictional inventory of 148 resources, visible tag coverage is 77.7% while valid coverage is 75.0%, and 25.1% of the monthly 4,214.3 fictional units stays unallocatable, entering no rule at all. When the shared 1,584.9 units is split across three rules, the same team's bill swings between 8.7% and 18.6%. A tag mandate zeroes the unallocatable share but rejects 37 resources and stalls 14 legitimate jobs worth 511.9 units; even once tags reach 95.9% coverage by month's end, that month's 568.6 units (13.5%) never correct retroactively. The previous lesson measured that the bill gives fourteen splits while the network requires ninety units of separation. That gap does not close from the bill; ownership information has to come from somewhere else. On the cloud side, that place is **the resource itself**: a tag is attached to a resource, and cost is distributed from that tag. This lesson's question is not how the distribution should be done, it is **where it does not hold**. Team-based accounting and the unsplittable item were measured in M22/K08; what is measured here is the tag, its input. #### Tagged Resource Inventory The example is the fictional **regional measurement network**: five teams, three environments, a rented machine, network, store, function, and edge layer. Resources, tags, and amounts are fictional; **the currency is fictional too** and is called `unit`, corresponding to no real price list. **CC7:** the inventory holds 148 resources; eight types, three of which are used by more than one team and carry a `shared` tag. **CC8:** whether a resource has an owner tag depends on **how it was created**; there are four creation paths. **CC9:** a misspelled owner tag shows up **tagged** in the coverage report but has no counterpart in the team roster; it is invalid in allocation. **CC10:** a shared resource's real driver is **unknown** — there is no true share to compare against in this lesson, what is measured is not deviation but rule-driven swing. **CC11:** a resource can carry at most ten tag keys; the network uses seven. **CC12:** a tag is completed on some day; bill lines cut before that day are untagged and never correct. The inventory is a process-internal **model** derived from a generator; the seed is visible. ```js // cost/inventory.mjs — tagged resource inventory model: no real inventory. Resources are // derived with a generator, the seed is visible, and the run repeats. The currency is fictional // and is called `unit`. Whether a tag *exists* is decided by how the resource was created; // a misspelled tag is a separate case: tagged in the coverage report, invalid in allocation. export const SEED = 6108; export const COUNT = 148; export const TEAM = ["meter-reading", "validation", "billing", "field", "reporting"]; export const ENV = ["production", "staging", "development"]; export const TAG_LIMIT = 10; // tag keys per resource, at most export const USED_KEY = ["owner", "environment", "service", "version", "temporary", "responsible", "cost-center"]; // type name, monthly unit lo-hi, shared (used by more than one team) export const TYPE = [ ["rented machine", 18, 96, false], ["block storage", 4, 34, false], ["object store", 6, 52, false], ["function", 1, 12, false], ["edge node", 3, 21, false], ["network gateway", 12, 40, true], ["shared message bus", 20, 64, true], ["monitoring collector", 9, 30, true], ]; // creation path: name, share, probability of an owner tag, probability the tag is spelled right export const PATH = [ ["template", 0.40, 1.00, 1.00], ["auto-scaling", 0.26, 0.71, 0.94], ["manual", 0.22, 0.55, 0.78], ["legacy migration", 0.12, 0.08, 0.60], ]; const pick = (table, u) => { let a = 0; for (const s of table) { a += s[1]; if (u < a) return s; } return table[table.length - 1]; }; // misspelling shapes: tagged in the coverage report, no counterpart in the team roster const VARIANT = [(e) => e.toUpperCase(), (e) => e.replace(/-/g, ""), (e) => e + "-team"]; export const inventory = () => { let s = SEED >>> 0; const r = () => { s = (Math.imul(s, 1664525) + 1013904223) >>> 0; return s / 4294967296; }; const list = []; for (let i = 0; i < COUNT; i++) { const [typeName, lo, hi, shared] = TYPE[Math.floor(r() * TYPE.length)]; const [pathName, , pOwned, pCorrect] = pick(PATH, r()); const env = ENV[Math.floor(r() * ENV.length)]; const unit = Math.round((lo + r() * (hi - lo)) * 10) / 10; const team = TEAM[Math.floor(r() * TEAM.length)]; let owner = null, valid = false; if (shared) { owner = "shared"; valid = true; } else if (r() < pOwned) { if (r() < pCorrect) { owner = team; valid = true; } else owner = VARIANT[Math.floor(r() * VARIANT.length)](team); } const day = 1 + Math.floor(r() * 34); // day the tag is completed; >30 = never const urgent = pathName === "manual" && r() < 0.18; // urgent repair job list.push({ id: i, type: typeName, shared, path: pathName, env, unit, owner, valid, fixDay: owner && valid ? 0 : day, urgent }); } return list; }; export const sum = (v) => v.reduce((a, b) => a + b, 0); export const sumUnits = (v) => sum(v.map((k) => k.unit)); ``` ```js // cost/allocate.mjs — tag coverage, cost allocation to teams (three rules), the resources a tag // mandate rejects, and tagging's failure to reach back into the past. What is measured is not // whether the tag is correct, it is *where the allocation does not hold*. import { inventory, TEAM, TYPE, PATH, TAG_LIMIT, USED_KEY, SEED, sum, sumUnits } from "./inventory.mjs"; const write = (w, ...s) => console.log(s.map((v, i) => (w[i] < 0 ? String(v).padEnd(-w[i]) : String(v).padStart(w[i]))).join("")); const t0 = process.hrtime.bigint(); const R = inventory(), T = sumUnits(R), share = (x) => `${(100 * x / T).toFixed(1)}%`; console.log(`seed ${SEED}: ${R.length} resources, monthly ${T.toFixed(1)} units (fictional currency)`); console.log("\n1. tag coverage by creation path"); const A = [-20, 11, 10, 10, 10, 11, 12]; write(A, "creation path", "resources", "tagged", "valid", "invalid", "untagged", "unit share"); for (const [name] of PATH) { const g = R.filter((k) => k.path === name), e = g.filter((k) => k.owner); write(A, name, g.length, e.length, g.filter((k) => k.valid).length, e.filter((k) => !k.valid).length, g.filter((k) => !k.owner).length, share(sumUnits(g))); } const tagged = R.filter((k) => k.owner), valid = R.filter((k) => k.valid); const invalid = R.filter((k) => k.owner && !k.valid), blank = R.filter((k) => !k.owner); write(A, "total", R.length, tagged.length, valid.length, invalid.length, blank.length, "100%"); console.log(` visible coverage ${(100 * tagged.length / R.length).toFixed(1)}%, ` + `valid coverage ${(100 * valid.length / R.length).toFixed(1)}%`); console.log(` invalid tag ${invalid.length} resources ${sumUnits(invalid).toFixed(1)} units ` + `(${share(sumUnits(invalid))}): shows up tagged in the coverage report`); console.log(` tag key limit ${TAG_LIMIT}, used ` + `${USED_KEY.length}, remaining ${TAG_LIMIT - USED_KEY.length}`); console.log("\n2. allocation: direct, shared, unallocated"); const direct = R.filter((k) => k.valid && k.owner !== "shared"); const shared = R.filter((k) => k.owner === "shared"); const unallocated = R.filter((k) => !k.valid); const D = sumUnits(direct), P = sumUnits(shared), X = sumUnits(unallocated); const B = [-14, 11, 15, 9]; write(B, "bucket", "resources", "monthly units", "share"); write(B, "direct", direct.length, D.toFixed(1), share(D)); write(B, "shared", shared.length, P.toFixed(1), share(P)); write(B, "unallocated", unallocated.length, X.toFixed(1), share(X)); const teamUnit = TEAM.map((e) => sumUnits(direct.filter((k) => k.owner === e))); const teamCount = TEAM.map((e) => direct.filter((k) => k.owner === e).length); const ratio = (v) => { const t = sum(v); return v.map((x) => x / t); }; const RULE = { equal: TEAM.map(() => 1 / TEAM.length), cost: ratio(teamUnit), "resource count": ratio(teamCount) }; console.log(` shared ${P.toFixed(1)} units split by a rule;`); console.log(` unallocated ${X.toFixed(1)} units (${share(X)}) enters no rule`); console.log("\n3. three split rules: how much the same team's bill swings by rule"); const C = [-16, ...TEAM.map(() => 15)]; write(C, "rule", ...TEAM); const result = {}; for (const [name, p] of Object.entries(RULE)) result[name] = TEAM.map((_, i) => teamUnit[i] + P * p[i]); write(C, "direct", ...teamUnit.map((x) => x.toFixed(1))); for (const [name, v] of Object.entries(result)) write(C, name, ...v.map((x) => x.toFixed(1))); const swing = TEAM.map((_, i) => { const v = Object.values(result).map((s) => s[i]); return [Math.min(...v), Math.max(...v)]; }); write(C, "widest gap", ...swing.map(([a, b]) => (b - a).toFixed(1))); write(C, "as % of bill", ...swing.map(([a, b]) => `${(100 * (b - a) / a).toFixed(1)}%`)); const en = swing.map(([a, b]) => (b - a) / a); const i0 = en.indexOf(Math.max(...en)); console.log(` no real share exists to compare against: what is measured is not deviation, it is rule-driven swing`); console.log(` most swing ${TEAM[i0]}: ${(100 * en[i0]).toFixed(1)}%; ` + `unallocated ${share(X)} is off the bill under all three rules`); console.log("\n4. tag mandate: how many resources are rejected, how many are legitimate work"); const rejected = R.filter((k) => !k.valid); const E = [-20, 11, 15, 12, 15]; write(E, "creation path", "rejected", "monthly units", "legitimate", "stalled units"); let legitCount = 0, legitUnit = 0; for (const [name] of PATH) { const g = rejected.filter((k) => k.path === name); const m = g.filter((k) => k.path === "auto-scaling" || k.urgent); legitCount += m.length; legitUnit += sumUnits(m); write(E, name, g.length, sumUnits(g).toFixed(1), m.length, sumUnits(m).toFixed(1)); } write(E, "total", rejected.length, sumUnits(rejected).toFixed(1), legitCount, legitUnit.toFixed(1)); console.log(` ${rejected.length} resources rejected (${(100 * rejected.length / R.length).toFixed(1)}%); ` + `${legitCount} of these are capacity or urgent repair work`); console.log(` the mandate zeroes the unallocated ${share(X)} share, in exchange ` + `${legitUnit.toFixed(1)} units' worth of work, ${legitCount} items, stalls`); console.log("\n5. tagging added later: the past bill does not correct itself"); const DAYS = 30, MILESTONE = [1, 5, 10, 15, 20, 25, 30]; const F = [6, 14, 10, 16, 19]; write(F, "day", "tagged", "coverage", "units/day", "accumulated loss"); const dailyLoss = (g) => sumUnits(R.filter((k) => !k.valid && k.fixDay > g)) / DAYS; let accumulated = 0; for (let g = 1; g <= DAYS; g++) { accumulated += dailyLoss(g); if (!MILESTONE.includes(g)) continue; const closed = R.filter((k) => k.valid || k.fixDay <= g); write(F, g, closed.length, `${(100 * closed.length / R.length).toFixed(1)}%`, (sumUnits(closed) / DAYS).toFixed(2), accumulated.toFixed(1)); } const never = R.filter((k) => !k.valid && k.fixDay > DAYS); console.log(` end-of-month coverage ${(100 * (R.length - never.length) / R.length).toFixed(1)}%, ` + `but the month's unallocated share is ${accumulated.toFixed(1)} units (${(100 * accumulated / T).toFixed(1)}%)`); console.log(` ${never.length} resources are never tagged: ${sumUnits(never).toFixed(1)} units/month ` + `stay unallocated every month`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, only the band is checked console.log(`\nfive tables within the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`); ``` ``` seed 6108: 148 resources, monthly 4214.3 units (fictional currency) 1. tag coverage by creation path creation path resources tagged valid invalid untagged unit share template 58 58 58 0 0 40.8% auto-scaling 40 31 29 2 9 30.6% manual 27 16 14 2 11 14.5% legacy migration 23 10 10 0 13 14.1% total 148 115 111 4 33 100% visible coverage 77.7%, valid coverage 75.0% invalid tag 4 resources 123.6 units (2.9%): shows up tagged in the coverage report tag key limit 10, used 7, remaining 3 2. allocation: direct, shared, unallocated bucket resources monthly units share direct 58 1572.2 37.3% shared 53 1584.9 37.6% unallocated 37 1057.2 25.1% shared 1584.9 units split by a rule; unallocated 1057.2 units (25.1%) enters no rule 3. three split rules: how much the same team's bill swings by rule rule meter-reading validation billing field reporting direct 286.6 250.3 267.8 309.8 457.7 equal 603.6 567.3 584.8 626.8 774.7 cost 575.5 502.6 537.8 622.1 919.1 resource count 641.8 550.9 568.4 555.7 840.3 widest gap 66.3 64.7 47.0 71.0 144.4 as % of bill 11.5% 12.9% 8.7% 12.8% 18.6% no real share exists to compare against: what is measured is not deviation, it is rule-driven swing most swing reporting: 18.6%; unallocated 25.1% is off the bill under all three rules 4. tag mandate: how many resources are rejected, how many are legitimate work creation path rejected monthly units legitimate stalled units template 0 0.0 0 0.0 auto-scaling 11 444.3 11 444.3 manual 13 222.8 3 67.6 legacy migration 13 390.1 0 0.0 total 37 1057.2 14 511.9 37 resources rejected (25.0%); 14 of these are capacity or urgent repair work the mandate zeroes the unallocated 25.1% share, in exchange 511.9 units' worth of work, 14 items, stalls 5. tagging added later: the past bill does not correct itself day tagged coverage units/day accumulated loss 1 112 75.7% 106.85 33.6 5 115 77.7% 108.90 161.3 10 119 80.4% 112.15 306.2 15 123 83.1% 115.44 434.3 20 133 89.9% 129.34 513.4 25 139 93.9% 136.04 552.4 30 142 95.9% 137.50 568.6 end-of-month coverage 95.9%, but the month's unallocated share is 568.6 units (13.5%) 6 resources are never tagged: 89.3 units/month stay unallocated every month five tables within the 2000 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Coverage Is Not One Number The first table splits tag coverage by creation path and produces two different numbers. **Visible coverage is 77.7%**: 115 of 148 resources carry an owner tag. **Valid coverage is 75.0%**: on four of those, the tag value matches no team at all. Four resources, 123.6 units, **2.9% of the bill** — clean in the coverage report, lost in allocation. Where coverage comes from is written into the path itself. All 58 resources created from a template are tagged **correctly**; the tag is written by the template, not a person. Thirteen of the 23 resources from legacy migration carry no tag at all, and 27 manually created resources show both a missing tag (11) and a misspelling (2). Nine of the 40 resources from auto-scaling carry no tag: scaling that does not inherit the resource does not inherit the tag either. The limit is also counted: a resource can carry ten tag keys, the network uses seven, **three remain**; adding one more split spends one of those three slots. #### Three Rules, Three Different Bills The second table splits allocation three ways: 1,572.2 units directly assigned (37.3%), 1,584.9 units shared (37.6%), and **1,057.2 units unallocated (25.1%)**. The unallocated share is not a splitting problem; the input is missing, and no rule accepts it. The third table splits the shared amount three ways: equal share, the direct-cost ratio, and the resource-count ratio. All three read only from the tag. Unlike M22/K08, there is no **real share** here; what is measured is not deviation, it is **the same team's bill swinging by rule**. Reporting swings the most: 774.7 at equal share, 840.3 at resource count, 919.1 at cost ratio — a gap of 144.4 units, **18.6% of its bill**. Billing swings least (8.7%). A team can watch its share swing by a sixth without changing anything about its own resources, purely because one rule changed centrally. The unallocated 25.1% shows up in none of the three rules; the teams' totals never add up to the total bill. #### The Work the Mandate Stalls The fourth table applies the obvious fix: a resource with no valid owner tag cannot be created. The mandate **zeroes the unallocated 25.1%**, because those resources never exist. Its cost sits in the same table. **37 resources are rejected**, and 14 of them are legitimate work: 11 resources opened by auto-scaling are capacity, three of the manually opened ones are urgent repairs. The stalled work is 511.9 units. **All** of auto-scaling's rejections are legitimate — the rule bites hardest exactly where no one is at a keyboard. The 13 legacy-migration rejections (390.1 units) stall no legitimate work at all: those resources already exist, and the mandate only reaches **new** ones. #### Tags Do Not Reach Backward The fifth table completes tags over the course of the month. Coverage climbs from 75.7% on day 1 to **95.9%** on day 30; by month's end the inventory report reads nearly clean. The month's bill is not clean: the accumulated unallocated amount is **568.6 units**, **13.5%** of the month. The reason is one sentence: bill lines are cut day by day, and a cut line carries whatever tag the resource wore that day. A resource tagged on day 20 leaves its first nineteen days unallocated. Some providers let a tag the resource already wore be activated retroactively for cost allocation; none writes a tag that was never attached back into the past. Coverage is **instantaneous**, the bill is **continuous**; putting both in the same report produces the picture of a month that has already been fixed. Six resources are never tagged at all: 89.3 units stay unallocated again every month. #### Summary - In the fictional inventory of 148 resources, **visible coverage is 77.7%, valid coverage is 75.0%**; the four-resource gap between them (123.6 units, 2.9%) shows up tagged in the report and gets lost in allocation. Coverage depends on the creation path: all 58 template resources are tagged correctly, 13 of 23 legacy-migration resources carry no tag. - Of the monthly 4,214.3 units, 37.3% is directly assigned, 37.6% is shared, and **25.1% enters no rule at all**; the teams' totals never add up to the total bill. - Split across three rules, the shared share makes the same team's bill swing between 8.7% and 18.6%; reporting's bill sits in a 144.4-unit band, between 774.7 and 919.1. - A tag mandate zeroes the unallocated share but **rejects 37 resources**; 14 of them are legitimate work, and 511.9 units' worth stalls. All 11 of auto-scaling's rejections are capacity work. - Even once tags reach 95.9% coverage by month's end, that month's **568.6 units (13.5%)** stay unallocated: coverage is instantaneous, the bill is continuous. #### Next Step Items were split and one attempt at ownership was made; both read the bill **after** it arrives. The next lesson moves the question onto the time axis: two alert types run over a spending curve, the day each is noticed is counted, and the bill's own delay enters the table as a constraint — by the time an alert fires, how much of the spending has already happened. ### Budget and Alerts Source: https://academia.sh/en/courses/cloud-governance/budget-and-alerts Measures when unexpected spending gets noticed: across five rises placed on a fictional spending curve, the threshold alert misses two of them entirely and, by the time it first fires, 86.4% of the monthly budget is already spent (204 units remain); the forecast alert catches all five but stays open on 63 of ninety days. The threshold family's best setting misses 821 units while the forecast family stays at 12.5; the false-alarm gap is 5 versus 6 across 21 event-free days. The bill's delay adds directly to the day noticed: a three-day delay adds 1.8 days to the average delay and 210 units to the missed amount, and an eight-day delay lets one rise go entirely unnoticed. The previous two lessons split the bill into items and made one attempt at ownership; both happen after the bill arrives. When the month closes, which team spent how much is known, and the spending is already done. This lesson's question sits inside the month: when spending rises unexpectedly, **which day** it gets noticed, and whether a rise ever ends unnoticed. #### The Spending Curve and Two Alerts The example is the fictional **regional measurement network**: five teams, three environments, a rented machine, network, store, function, and edge layer. The curve, budget, and events are fictional; **the currency is fictional too** and is called `unit`, corresponding to no real price list. **CC13:** the monthly budget is 1,500 units and three months are watched. **CC14:** the curve is built from a baseline, a weekly pattern, slow growth, and noise; growth alone drifts **the event-free baseline** toward the budget too. **CC15:** five rises are placed on the curve; four of them end in the month they start. **CC16:** the threshold alert watches the month's cumulative spend against the budget ratio; the forecast alert predicts month end from the last *k* days' average. **CC17:** every day the condition holds is an **alert day**; one that arrives during a rise is a notice, one on an event-free day is a **false alarm**. **CC18:** the bill arrives delayed — day *g*'s decision is made with *g*−delay's data, so the condition shifts by exactly the delay. The curve, budget, and alerts are a process-internal **model**; there is no real spending record. ```js // cost/spend.mjs — daily spend curve model: no real spend record. The curve is made of a baseline, // a weekly pattern, slow growth, and hand-placed *events*; noise comes from a generator, the seed // is visible, the currency is fictional and is called `unit`. The event-free baseline is also // computed separately: an event's *excess* spend is the difference between the two. export const SEED = 3391; export const MONTHS = 3, MONTH_DAYS = 30, DAYS = MONTHS * MONTH_DAYS; export const BUDGET = 1500; // monthly budget (unit, fictional) export const BASE = 43, GROWTH = 0.12; // daily baseline and daily slow growth export const WEEK = [1.08, 1.10, 1.06, 1.04, 0.98, 0.72, 0.68]; // event: name, start day (1..90), duration (days), daily excess unit export const EVENT = [ ["dev environment left running", 9, 22, 12.5], ["temporary data copy", 38, 6, 41.0], ["wrong size choice", 47, 40, 9.0], ["edge node duplication", 63, 25, 7.5], ["reprocessing loop", 71, 4, 63.0], ]; export const active = (d) => EVENT.filter(([, b, s]) => d >= b && d < b + s); export const series = (withEvents = true) => { let s = SEED >>> 0; const r = () => { s = (Math.imul(s, 1664525) + 1013904223) >>> 0; return s / 4294967296; }; const v = [0]; for (let d = 1; d <= DAYS; d++) { const noise = 0.94 + r() * 0.12; let x = (BASE + GROWTH * (d - 1)) * WEEK[(d - 1) % 7] * noise; if (withEvents) for (const [, , , f] of active(d)) x += f; v.push(Math.round(x * 100) / 100); } return v; }; export const month = (d) => Math.floor((d - 1) / MONTH_DAYS); export const dayOfMonth = (d) => ((d - 1) % MONTH_DAYS) + 1; export const sum = (v) => v.reduce((a, b) => a + b, 0); ``` ```js // cost/alert.mjs — two alert kinds run on the same curve: threshold watches budget percentage, // forecast predicts month end from the last *k* days' average. What is counted is not correctness // but *the day noticed* — how many days after an event starts the alert fires, whether the rise // ever goes unnoticed. The bill's delay is a constraint: day g's decision uses g-DELAY's data, so // the condition shifts by exactly DELAY days. Every day the condition holds is an *alert day*. import { EVENT, BUDGET, MONTHS, MONTH_DAYS, DAYS, SEED, active, series, month, dayOfMonth, sum } from "./spend.mjs"; const write = (w, ...s) => console.log(s.map((v, i) => (w[i] < 0 ? String(v).padEnd(-w[i]) : String(v).padStart(w[i]))).join("")); const t0 = process.hrtime.bigint(); const ACTUAL = series(), BASELINE = series(false); const excess = (d) => ACTUAL[d] - BASELINE[d]; const monthTotal = (v, m) => sum(v.slice(m * MONTH_DAYS + 1, (m + 1) * MONTH_DAYS + 1)); const cumulative = (m, d) => sum(ACTUAL.slice(m * MONTH_DAYS + 1, m * MONTH_DAYS + d + 1)); console.log(`seed ${SEED}: ${DAYS} days, monthly budget ${BUDGET} units (fictional currency)`); console.log("\n1. spending curve: budget, actual, overrun"); const A = [7, 10, 14, 11, 11, 14]; write(A, "month", "budget", "actual", "overrun", "baseline", "event excess"); for (let m = 0; m < MONTHS; m++) { const act = monthTotal(ACTUAL, m), base = monthTotal(BASELINE, m); write(A, m + 1, BUDGET, act.toFixed(1), (act - BUDGET).toFixed(1), base.toFixed(1), (act - base).toFixed(1)); } const withinMonth = EVENT.filter(([, b, s2]) => month(b) === month(b + s2 - 1)).length; console.log(` ${EVENT.length} events placed (in the model above); ${withinMonth} of them ` + `end in the month they start`); // run one alert kind over the curve; returns the list of triggered days const simulate = (kind, p, delay) => { const days = []; for (let g = 1; g <= DAYS; g++) { const v = g - delay; // last day readable from the bill if (v < 1) continue; const m = month(v), d = dayOfMonth(v); if (month(g) !== m) continue; // month boundary: decide with the month's own data const acc = cumulative(m, d); let trigger = false; if (kind === "threshold") trigger = acc / BUDGET >= p; else { if (d < p) continue; const avg = sum(ACTUAL.slice(m * MONTH_DAYS + d - p + 1, m * MONTH_DAYS + d + 1)) / p; trigger = acc + avg * (MONTH_DAYS - d) > BUDGET; } if (trigger) days.push(g); } return days; }; // map alert days onto events: day noticed, delay, excess spent during the delay const measure = (days) => { const found = EVENT.map(([, b, s]) => { const g = days.find((x) => x >= b && x < b + s); return g === undefined ? null : g; }); const falseAlarms = days.filter((g) => active(g).length === 0).length; const missed = EVENT.filter((_, i) => found[i] === null).length; const delays = found.map((g, i) => (g === null ? null : g - EVENT[i][1])); const missedUnits = sum(EVENT.map(([, b, s], i) => { const end = found[i] === null ? b + s : found[i]; let x = 0; for (let g = b; g < end; g++) x += excess(g); return x; })); const delayVals = delays.filter((x) => x !== null); return { found, delays, falseAlarms, missed, missedUnits, total: days.length, avgDelay: delayVals.length ? sum(delayVals) / delayVals.length : NaN }; }; console.log("\n2. two alert kinds, same curve (threshold 85%, forecast window 7 days, delay 0)"); const es = simulate("threshold", 0.85, 0), ta = simulate("forecast", 7, 0); const measuredThreshold = measure(es), measuredForecast = measure(ta); const C = [-32, 9, 15, 10, 14, 10]; write(C, "event", "starts", "threshold day", "delay", "forecast day", "delay"); for (let i = 0; i < EVENT.length; i++) { const g = (x) => (x === null ? "missed" : x); write(C, EVENT[i][0], EVENT[i][1], g(measuredThreshold.found[i]), g(measuredThreshold.delays[i]), g(measuredForecast.found[i]), g(measuredForecast.delays[i])); } // how much of the budget is spent by the day the alert arrives: each month's first alert day const spentShare = (days) => { const first = []; for (let m = 0; m < MONTHS; m++) { const g = days.find((x) => month(x) === m); if (g !== undefined) first.push(cumulative(m, dayOfMonth(g)) / BUDGET); } return sum(first) / first.length; }; for (const [name, o, g] of [["threshold", measuredThreshold, es], ["forecast", measuredForecast, ta]]) { console.log(` ${name.padEnd(9)} ${o.total} alert days, ${EVENT.length - o.missed} events ` + `caught, ${o.missed} missed, ${o.falseAlarms} false-alarm days`); console.log(` ${" ".repeat(9)} ${o.missedUnits.toFixed(1)} units excess during the delay; first alert ` + `day ${(100 * spentShare(g)).toFixed(1)}% of the budget is spent, ` + `${(BUDGET * (1 - spentShare(g))).toFixed(0)} units remain`); } console.log("\n3. parameter scan: how early the alert fires, how many false alarms"); const D = [-11, 9, 12, 10, 8, 14, 8, 14]; write(D, "kind", "setting", "alert days", "caught", "missed", "avg delay", "false", "missed units"); const settings = [["threshold", 0.60], ["threshold", 0.85], ["threshold", 1.00], ["forecast", 3], ["forecast", 7], ["forecast", 14]]; const scored = []; for (const [kind, p] of settings) { const o = measure(simulate(kind, p, 0)); scored.push([kind, p, o]); write(D, kind, kind === "threshold" ? `${(100 * p).toFixed(0)}%` : `${p} days`, o.total, EVENT.length - o.missed, o.missed, isNaN(o.avgDelay) ? "-" : o.avgDelay.toFixed(1), o.falseAlarms, o.missedUnits.toFixed(1)); } const best = scored.reduce((a, b) => (b[2].missedUnits < a[2].missedUnits ? b : a)); console.log(` least-missing setting: ${best[0]} ${best[1]}, ` + `${best[2].missedUnits.toFixed(1)} units, ${best[2].falseAlarms} false-alarm days`); const eventFree = [...Array(DAYS)].map((_, i) => i + 1).filter((g) => active(g).length === 0).length; console.log(` event-free days ${eventFree}/${DAYS}: a false alarm can only happen on these days`); console.log(` ${best[0]} ${best[1]} keeps the alert open on ${best[2].total} of ${DAYS} days ` + `(${(100 * best[2].total / DAYS).toFixed(0)}%)`); console.log("\n4. the bill's delay is a constraint: the decision is made with g-DELAY's data"); const F = [9, 12, 8, 14, 14, 14]; write(F, "delay", "alert days", "missed", "avg delay", "missed units", "budget share"); for (const gk of [0, 1, 3, 5, 8]) { const o = measure(simulate(best[0], best[1], gk)); write(F, `${gk} day${gk === 1 ? "" : "s"}`, o.total, o.missed, isNaN(o.avgDelay) ? "-" : o.avgDelay.toFixed(1), o.missedUnits.toFixed(1), `${(100 * o.missedUnits / (MONTHS * BUDGET)).toFixed(1)}%`); } const g0 = measure(simulate(best[0], best[1], 0)), g3 = measure(simulate(best[0], best[1], 3)); console.log(` a 3-day delay adds ${(g3.avgDelay - g0.avgDelay).toFixed(1)} days to the day ` + `noticed and grows the missed amount by ${(g3.missedUnits - g0.missedUnits).toFixed(1)} units`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, only the band is checked console.log(`\nfour tables within the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`); ``` ``` seed 3391: 90 days, monthly budget 1500 units (fictional currency) 1. spending curve: budget, actual, overrun month budget actual overrun baseline event excess 1 1500 1553.7 53.7 1278.7 275.0 2 1500 1752.8 252.8 1380.8 372.0 3 1500 2137.0 637.0 1463.5 673.5 5 events placed (in the model above); 4 of them end in the month they start 2. two alert kinds, same curve (threshold 85%, forecast window 7 days, delay 0) event starts threshold day delay forecast day delay dev environment left running 9 25 16 15 6 temporary data copy 38 missed missed 38 0 wrong size choice 47 52 5 47 0 edge node duplication 63 77 14 67 4 reprocessing loop 71 missed missed 71 0 threshold 29 alert days, 3 events caught, 2 missed, 3 false-alarm days 1292.0 units excess during the delay; first alert day 86.4% of the budget is spent, 204 units remain forecast 63 alert days, 5 events caught, 0 missed, 6 false-alarm days 141.0 units excess during the delay; first alert day 34.6% of the budget is spent, 982 units remain 3. parameter scan: how early the alert fires, how many false alarms kind setting alert days caught missed avg delay false missed units threshold 60% 46 4 1 5.5 5 821.0 threshold 85% 29 3 2 11.7 3 1292.0 threshold 100% 17 3 2 15.7 3 1440.0 forecast 3 days 68 5 0 0.2 6 12.5 forecast 7 days 63 5 0 2.0 6 141.0 forecast 14 days 47 4 1 5.8 6 967.5 least-missing setting: forecast 3, 12.5 units, 6 false-alarm days event-free days 21/90: a false alarm can only happen on these days forecast 3 keeps the alert open on 68 of 90 days (76%) 4. the bill's delay is a constraint: the decision is made with g-DELAY's data delay alert days missed avg delay missed units budget share 0 days 68 0 0.2 12.5 0.3% 1 day 65 0 0.8 82.5 1.8% 3 days 59 0 2.0 222.5 4.9% 5 days 53 0 3.2 362.5 8.1% 8 days 44 1 4.3 490.5 10.9% a 3-day delay adds 1.8 days to the day noticed and grows the missed amount by 210.0 units four tables within the 2000 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### The Threshold Alert Waits on the Budget The first table shows all three months overrun: 53.7 — 252.8 — 637.0 units. The source is not only the rises; the event-free baseline climbs from 1,278.7 to 1,463.5, filling **97.6% of the budget** in the third month. The second table runs the threshold alert at 85%. Two rises **go entirely unnoticed**: the temporary data copy (6 days, 41 units a day) and the reprocessing loop (4 days, 63 units a day). Both short and large, they end before their cumulative total clears the threshold. The long, small ones are noticed 16 and 14 days later; the total missed is **1,292 units**. The reason is the measure itself: the threshold alert watches **cumulative spend**, not the rise. By the day it first fires, an average **86.4%** of the budget is already spent, leaving **204 units** — a notice of what remains, not a warning. Even at 60% the missed amount is 821 units, and one short rise still goes unnoticed. #### The Forecast Alert Is Early but Constant At a seven-day window, the forecast alert catches **all five** rises, noticing two short ones the day they start (38 and 71), because the last seven days' average jumps in a day and the month-end forecast clears the budget. The missed amount drops from 1,292 to **141 units**, and only **34.6%** of the budget is spent by the first alert — 982 units remain. The third table shows the price. Shrinking the window to three days drops the missed amount to **12.5 units** but keeps the alert open **68 of 90 days (76%)**. False alarms do not explain it: 6 of 21 event-free days carry one, against 5 at a 60% threshold. What separates the two families is not false alarms but **how long the true alarm rings**: the forecast alert sounds for as long as the rise lasts and stops being distinctive, while the threshold stays quiet but speaks late. Widening the window to fourteen days makes the forecast alert behave like the threshold: one rise goes unnoticed, and the missed amount climbs to 967.5 units. #### The Bill's Delay Sets a Ceiling The fourth table adds one constraint: the bill's data arrives delayed. Because the alert condition shifts by exactly the delay, the day noticed shifts the same amount. Average delay climbs from 0.2 days to 0.8 at one day, 2.0 at three, 4.3 at eight. The missed amount tracks this linearly: 12.5 — 82.5 — 222.5 — 362.5 — 490.5 units. At an eight-day delay, the four-day reprocessing loop **ends before its data ever appears** and goes unnoticed; **10.9%** of the three-month budget is spent unnoticed. This is a ceiling: however much the alert setting improves, the day noticed cannot precede the bill's delay. The delay adds no separate cost item either — the amount leaving the budget is the same, only spent before the alert arrives. #### Summary - All three months overrun (53.7 — 252.8 — 637.0 units); part of it comes not from a rise but from baseline growth — the event-free baseline fills 97.6% of the budget in the third month. - **The threshold alert watches cumulative spend, not the rise**: at an 85% threshold, two short rises go entirely unnoticed, and by its first firing 86.4% of the budget is already spent — 204 units remain. Total missed: 1,292 units. - The forecast alert catches all five rises at a seven-day window and drops the missed amount to 141 units; **982 units** remain by the first alert. - The cost of noticing early is not false alarms but duration: a three-day window drops the missed amount to 12.5 units but keeps the alert open 68 of 90 days (76%); false alarms sit at 6 of 21 event-free days, against 5 at 60%. - **The bill's delay adds directly to the day noticed**: a 3-day delay adds 1.8 days to the average and 210 units to the missed amount; at an 8-day delay a four-day rise goes entirely unnoticed and 10.9% of the three-month budget is spent unnoticed. #### Next Step An alert says spending rose unexpectedly; it says nothing about expected spending. As the baseline edges closer to the budget every month, no alert fires — there is no event, only resources built larger than needed. The next lesson looks at that baseline: downsizing candidates are drawn from usage history, and each candidate's savings and risk are counted together. ### Resource Sizing Source: https://academia.sh/en/courses/cloud-governance/resource-sizing Measures the bill side of overprovisioning: across the fictional network's ten resources, average p95 occupancy is 31.4%, and downsizing that leaves a 20% margin over p95 promises 401 of the monthly 875 fictional units (45.8%). 353 units of that savings (88.0%) require a disruption, seven candidates have a past peak that exceeds the new tier, and the worst moment falls 168.4% short; the share that is both non-disruptive and risk-free is only 10 units. Raising the safety margin from 0% to 100% drops savings from 458 to 64 units and short hours from 47 to 3. An idle 147.2 units, invisible in any usage report, is deleted with no disruption and hands over 37% of the downsizing savings on its own. The previous lesson measured that alerts catch unexpected rises but never question expected spending: as the baseline edges toward the budget, no alert fires — there is no event, only resources built larger than needed. The example sizing was measured in M23/K02 and is not repeated here. This lesson's axis is **the bill side**: which resource can be downsized is drawn from usage history, each candidate's **savings and risk counted on the same line**. #### Usage History and the Tier Ladder The example is the fictional **regional measurement network**: five teams, three environments, a rented machine, network, store, function, and edge layer. Resources, tiers, and amounts are fictional; **the currency is fictional too** and is called `unit`, corresponding to no real price list. **CC19:** there are ten resources and five types; every type has a **tier ladder**, each tier a capacity unit paired with a monthly price. **CC20:** the usage history is 30 days × 24 hours, built from a daily pattern, a weekend dip, noise, and sparse peaks. **CC21:** a downsizing candidate is the smallest tier that still suffices once a **safety margin** is added on top of p95 demand. **CC22:** risk is two numbers — how many hours the new tier fell short in the past, and **how far short at the worst moment**. **CC23:** whether stepping a tier down requires a disruption depends on the type. **CC24:** an idle resource's usage ratio is undefined; it never shows up in any occupancy report. The ladder, history, and candidates are a process-internal **model**; there is no real measurement record. ```js // cost/capacity.mjs — tier ladder and usage history model: no real measurement record. Tiers are // [capacity unit, monthly unit] pairs; a 30-day x 24-hour history is derived from a generator, // the seed is visible, the currency is fictional and is called `unit`. The example sizing was // measured in M23/K02; what is measured here is the savings and risk of stepping a tier down. export const SEED = 8814; export const DAYS = 30, HOURS = DAYS * 24; export const LADDER = { "rented machine": [[1, 14], [2, 27], [4, 52], [8, 101], [16, 198]], "block storage": [[100, 7], [250, 17], [500, 33], [1000, 64], [2000, 126]], "function memory": [[1, 3], [2, 6], [4, 11], [8, 21]], "network gateway": [[1, 22], [2, 41], [4, 79]], "edge node": [[1, 9], [2, 17], [4, 32]], }; // does stepping the tier down require a disruption, and why export const DISRUPTION = { "rented machine": ["yes", "restart"], "block storage": ["yes", "data move"], "function memory": ["no", "next call"], "network gateway": ["no", "tier switch"], "edge node": ["yes", "node swap"], }; // name, type, current tier index, team, base demand, peak multiplier, peak probability export const RESOURCE = [ ["meter-receiver-1", "rented machine", 3, "meter-reading", 2.10, 2.1, 0.012, 13], ["meter-receiver-2", "rented machine", 3, "meter-reading", 3.40, 1.6, 0.020, 13], ["validator", "rented machine", 2, "validation", 1.15, 2.4, 0.008, 11], ["bill-builder", "rented machine", 4, "billing", 4.20, 2.8, 0.006, 14], ["report-engine", "rented machine", 3, "reporting", 1.05, 5.2, 0.004, 12], ["raw-measurement-disk", "block storage", 3, "meter-reading", 340, 1.1, 0.010, 15], ["archive-disk", "block storage", 4, "reporting", 610, 1.0, 0.000, 11], ["event-function", "function memory", 3, "validation", 1.80, 2.0, 0.015, 11], ["north-gateway", "network gateway", 2, "meter-reading", 1.30, 2.2, 0.011, 12], ["field-node", "edge node", 2, "field", 0.75, 2.6, 0.007, 11], ]; // resources invisible to any usage measurement: usage ratio is undefined // name, count, unit monthly price, why idle, does deleting it need a disruption export const IDLE = [ ["unattached block storage", 4, 17, "machine deleted, storage remained", "no"], ["unused static address", 6, 4, "migration finished, address never released", "no"], ["old snapshot", 23, 2.4, "no retention rule", "no"], ]; const PATTERN = [0.42, 0.38, 0.36, 0.35, 0.38, 0.48, 0.66, 0.84, 0.98, 1.06, 1.10, 1.12, 1.08, 1.09, 1.11, 1.07, 1.00, 0.92, 0.84, 0.76, 0.68, 0.60, 0.52, 0.46]; export const history = (k) => { const [, , , , base, peak, p] = k; let s = (SEED + k[7] * 7919) >>> 0; // seed tied to a fixed field, not the name const r = () => { s = (Math.imul(s, 1664525) + 1013904223) >>> 0; return s / 4294967296; }; const v = []; for (let t = 0; t < HOURS; t++) { const weekend = (Math.floor(t / 24) % 7) >= 5 ? 0.74 : 1.0; let x = base * PATTERN[t % 24] * weekend * (0.92 + r() * 0.16); if (r() < p) x *= peak; v.push(x); } return v; }; export const percentile = (v, q) => { const s = [...v].sort((a, b) => a - b); return s[Math.min(s.length - 1, Math.floor(q * s.length))]; }; export const sum = (v) => v.reduce((a, b) => a + b, 0); ``` ```js // cost/scale.mjs — downsizing candidates are drawn from usage history and each candidate's *savings and risk* // are counted together: how many hours the proposed tier fell short in the past, how much it fell short in the // worst moment. Those that require a disruption are separated out, and idle resources are counted on their own. import { RESOURCE, LADDER, DISRUPTION, IDLE, HOURS, SEED, history, percentile, sum } from "./capacity.mjs"; const write = (w, ...s) => console.log(s.map((v, i) => (w[i] < 0 ? String(v).padEnd(-w[i]) : String(v).padStart(w[i]))).join("")); const t0 = process.hrtime.bigint(); const H = RESOURCE.map(history); const tierCapacity = (t, i) => LADDER[t][i][0], tierUnit = (t, i) => LADDER[t][i][1]; const T = sum(RESOURCE.map((k) => tierUnit(k[1], k[2]))); const IDLE_TOTAL = sum(IDLE.map(([, n, b]) => n * b)); console.log(`seed ${SEED}: ${RESOURCE.length} resources, ${HOURS} hours history, ` + `monthly ${T} units + idle ${IDLE_TOTAL.toFixed(1)} units (fictional currency)`); console.log("\n1. usage history: how full is the current tier"); const A = [-22, -19, 10, 15, 9, 9, 10]; write(A, "resource", "type", "capacity", "monthly units", "p95", "peak", "p95 full"); for (let i = 0; i < RESOURCE.length; i++) { const k = RESOURCE[i], c = tierCapacity(k[1], k[2]), v = H[i]; write(A, k[0], k[1], c, tierUnit(k[1], k[2]), percentile(v, 0.95).toFixed(2), Math.max(...v).toFixed(2), `${(100 * percentile(v, 0.95) / c).toFixed(0)}%`); } const occupancy = RESOURCE.map((k, i) => percentile(H[i], 0.95) / tierCapacity(k[1], k[2])); console.log(` p95 occupancy average ${(100 * sum(occupancy) / occupancy.length).toFixed(1)}%; ` + `below 50% ${occupancy.filter((x) => x < 0.5).length} resources`); // margin: safety margin added on top of p95. Proposed tier = smallest tier that still covers it. const candidate = (margin) => RESOURCE.map((k, i) => { const [name, type, cur] = k, v = H[i], target = percentile(v, 0.95) * (1 + margin); let proposed = cur; for (let j = 0; j < LADDER[type].length; j++) if (LADDER[type][j][0] >= target) { proposed = Math.min(cur, j); break; } const c = tierCapacity(type, proposed); const short = v.filter((x) => x > c).length; const shortfall = Math.max(0, (Math.max(...v) - c) / c); return { name, type, cur, proposed, savings: tierUnit(type, cur) - tierUnit(type, proposed), short, shortfall, drop: cur - proposed, remaining: proposed }; }); console.log("\n2. downsizing candidates (p95 + 20% safety margin): savings and risk together"); const B = [-22, 9, 8, 7, 9, 11, 10, 22]; write(B, "resource", "tier", "new", "drop", "savings", "short hrs.", "worst", "disruption"); const CAND = candidate(0.20); let disrupt = 0, noDisrupt = 0; for (const a of CAND) { if (a.drop === 0) continue; const [needs, reason] = DISRUPTION[a.type]; if (needs === "yes") disrupt += a.savings; else noDisrupt += a.savings; write(B, a.name, tierCapacity(a.type, a.cur), tierCapacity(a.type, a.proposed), a.drop, a.savings, a.short, `${(100 * a.shortfall).toFixed(1)}%`, `${needs} (${reason})`); } const down = CAND.filter((a) => a.drop > 0); const tt = sum(CAND.map((a) => a.savings)); console.log(` ${down.length}/${RESOURCE.length} resources are candidates, savings ${tt} units/month ` + `(${(100 * tt / T).toFixed(1)}%)`); console.log(` disruptive ${disrupt} units (${(100 * disrupt / tt).toFixed(1)}%), ` + `non-disruptive ${noDisrupt} units`); const risky = CAND.filter((a) => a.drop > 0 && a.short > 0); console.log(` ${risky.length} candidates have a past peak that exceeds the new tier: total ` + `${sum(risky.map((a) => a.short))} hours, worst shortfall ` + `${(100 * Math.max(...CAND.map((a) => a.shortfall))).toFixed(1)}%`); console.log(` tier steps remaining on the ladder: ${sum(down.map((a) => a.remaining))}, ` + `${down.filter((a) => a.remaining === 0).length} resources at the bottom tier`); console.log("\n3. safety-margin scan: savings and short hours move together"); const C = [8, 12, 11, 11, 12, 12, 11]; write(C, "margin", "candidates", "savings", "% of bill", "short hrs.", "% of hours", "worst"); for (const p of [0, 0.20, 0.35, 0.60, 1.00]) { const a = candidate(p), i2 = a.filter((x) => x.drop > 0), s = sum(a.map((x) => x.savings)); const y = sum(a.map((x) => x.short)), e = Math.max(...a.map((x) => x.shortfall)); write(C, `${(100 * p).toFixed(0)}%`, i2.length, s, `${(100 * s / T).toFixed(1)}%`, y, `${(100 * y / (HOURS * RESOURCE.length)).toFixed(2)}%`, `${(100 * e).toFixed(1)}%`); } console.log(` short hours stay under 1% of ${HOURS} x ${RESOURCE.length} hours across the whole scan:`); console.log(` what decides it is not the count, it is the worst-moment shortfall`); console.log("\n4. idle resources: invisible in any usage-ratio report"); const D = [-26, 8, 12, 15, 45, 12]; write(D, "item", "count", "unit price", "monthly units", "why idle", "disruption"); for (const [name, n, b, why, needs] of IDLE) write(D, name, n, b, (n * b).toFixed(1), why, needs); write(D, "total", sum(IDLE.map((x) => x[1])), "-", IDLE_TOTAL.toFixed(1), "-", "no"); console.log(` idle ${IDLE_TOTAL.toFixed(1)} units/month, ${(100 * IDLE_TOTAL / T).toFixed(1)}% ` + `of the used bill; all of it is deleted with no disruption`); console.log(` and it hands over ${(100 * IDLE_TOTAL / tt).toFixed(0)}% of the downsizing ` + `savings on its own`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, only the band is checked console.log(`\nfour tables within the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`); ``` ``` seed 8814: 10 resources, 720 hours history, monthly 875 units + idle 147.2 units (fictional currency) 1. usage history: how full is the current tier resource type capacity monthly units p95 peak p95 full meter-receiver-1 rented machine 8 101 2.42 5.13 30% meter-receiver-2 rented machine 8 101 3.92 6.33 49% validator rented machine 4 52 1.31 3.15 33% bill-builder rented machine 16 198 4.78 13.85 30% report-engine rented machine 8 101 1.19 5.37 15% raw-measurement-disk block storage 1000 64 387.62 438.71 39% archive-disk block storage 2000 126 694.29 737.55 35% event-function function memory 8 21 2.06 4.11 26% north-gateway network gateway 4 79 1.48 2.81 37% field-node edge node 4 32 0.86 2.23 21% p95 occupancy average 31.4%; below 50% 10 resources 2. downsizing candidates (p95 + 20% safety margin): savings and risk together resource tier new drop savings short hrs. worst disruption meter-receiver-1 8 4 1 49 6 28.3% yes (restart) validator 4 2 1 25 3 57.5% yes (restart) bill-builder 16 8 1 97 5 73.1% yes (restart) report-engine 8 2 2 74 2 168.4% yes (restart) raw-measurement-disk 1000 500 1 31 0 0.0% yes (data move) archive-disk 2000 1000 1 62 0 0.0% yes (data move) event-function 8 4 1 10 1 2.7% no (next call) north-gateway 4 2 1 38 1 40.6% no (tier switch) field-node 4 2 1 15 2 11.3% yes (node swap) 9/10 resources are candidates, savings 401 units/month (45.8%) disruptive 353 units (88.0%), non-disruptive 48 units 7 candidates have a past peak that exceeds the new tier: total 20 hours, worst shortfall 168.4% tier steps remaining on the ladder: 16, 0 resources at the bottom tier 3. safety-margin scan: savings and short hours move together margin candidates savings % of bill short hrs. % of hours worst 0% 10 458 52.3% 47 0.65% 168.4% 20% 9 401 45.8% 20 0.28% 168.4% 35% 8 370 42.3% 20 0.28% 168.4% 60% 5 245 28.0% 16 0.22% 168.4% 100% 2 64 7.3% 3 0.04% 34.2% short hours stay under 1% of 720 x 10 hours across the whole scan: what decides it is not the count, it is the worst-moment shortfall 4. idle resources: invisible in any usage-ratio report item count unit price monthly units why idle disruption unattached block storage 4 17 68.0 machine deleted, storage remained no unused static address 6 4 24.0 migration finished, address never released no old snapshot 23 2.4 55.2 no retention rule no total 33 - 147.2 - no idle 147.2 units/month, 16.8% of the used bill; all of it is deleted with no disruption and it hands over 37% of the downsizing savings on its own four tables within the 2000 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Occupancy Is Low, Savings Look Large The first table divides ten resources' p95 demand by current capacity. Average occupancy is **31.4%**, and **all ten resources** sit under half. The extreme case, the report engine, sits on an eight-unit tier with p95 demand of 1.19 — 15% of capacity. The second table turns this into money. With a 20% safety margin over p95, **nine resources become candidates** and **401 units** drop off the monthly 875-unit bill: **45.8%**. The one exception is meter-receiver-2: its p95 fills half its capacity, so one tier down falls short — the two columns to the right change this figure. #### Two Numbers Sit Beside the Savings The first column counts **risk**: how many hours the new tier fell short in the past. On seven candidates this is above zero, totaling 20 hours — three in a thousand of the 7,200-hour history. The second number keeps this from being negligible. **The capacity missing at the worst moment** is **168.4%** for the report engine: a two-unit new tier runs more than half short against a 5.37-unit peak. Bill-builder falls 73.1% short, validator 57.5%. At the peak, service queues or gets rejected; the measure saying "two hours" does not soften that. The second column writes out **disruption** item by item. The machine tier needs a restart, block storage needs a data move, an edge node needs a node swap; only function memory and the network gateway step down without one. **353 units (88.0%)** of the savings require a disruption, the non-disruptive share is **48 units**. Together, the two columns narrow the table: 38 of the non-disruptive 48 units is the network gateway, worst shortfall **40.6%**. What is left — non-disruptive, no notable shortfall — is the event function, **10 units**, **2.5%** of the 401-unit promise, free of cost. The disks show the reverse: the two disk candidates carry zero risk, but their 93-unit savings require a data-move disruption. #### What the Safety Margin Buys The third table scans the margin. At zero, savings are 458 units (52.3%), short hours are 47. At 20%, short hours drop to 20 and savings lose only 57 units: the curve's most efficient point. At 60%, savings fall to 245 units and short hours drop only to 16 — 156 units paid for 4 hours gained. The worst shortfall, by contrast, **does not change at all up to a 100% margin** (168.4%), because the margin is added to p95, not the peak. A rare, large peak's risk cannot be bought with a safety margin. #### The Item Nothing Touches The fourth table counts the item the occupancy report never shows. Four unattached block storage volumes, six unused static addresses, and twenty-three old snapshots with no retention rule run **147.2 units** a month — **16.8%** of the used bill. They never enter the candidate list: their usage ratios are undefined, there is no demand to divide. The comparison is the payoff: the whole 147.2 units is deleted **without disruption** and carries no peak risk, while downsizing's free-of-cost share is only 10 units. The idle resource hands over **37%** of the savings on its own. #### Summary - **All ten resources** sit under half full at p95 (average 31.4%, lowest 15%); a p95 + 20% margin makes nine resources candidates, and **401 of the monthly 875 units (45.8%)** shows up as savings. - **353 units (88.0%) of the savings require a disruption**; 38 of the non-disruptive 48 units is a candidate with a worst-moment shortfall of 40.6%. The share that is both non-disruptive and risk-free is **10 units, 2.5% of the savings**. Risk and disruption are independent: the two disk candidates produce no shortfall, yet both need a data-move disruption. - The safety margin buys short hours, **it cannot buy the worst shortfall**: as the margin climbs from 0% to 100%, savings drop from 458 to 64 units, but the worst shortfall stays at 168.4% — the margin is added to p95, not the peak. - An idle resource runs **147.2 units (16.8%)** and never shows up in any occupancy report; it is deleted without disruption and hands over 37% of the savings on its own. #### Next Step Every number in this lesson points downward: what can be downsized, what can be deleted. A number points toward growth too, and it never shows up on the bill. The next lesson applies the scaling plan to a limit table: how many units remain before each limit, and which limit fills first. ### Quotas and Service Limits Source: https://academia.sh/en/courses/cloud-governance/quotas-and-service-limits The scaling plan is applied to a fictional limit table: eight limits average 66.5% full, six are shared by five teams, and two can never be increased. Once the growth plan runs, the first limit to fill is not the fullest one but the one tied to the fastest driver — name resolution at month 3.1, function concurrency at month 4.3; by month twelve, seven of the eight limits are exceeded. Scanning request growth from 2% to 25% changes the identity of the limit that fills first. The cost of hitting a limit depends on its behavior and increase time: a total of 2,195 fictional units and 116 limit-days, 51.7% of which comes from a single silent-throttling limit. The previous lesson measured the downward direction: what can be downsized, what can be deleted. There is a number pointing upward too, and it never shows up on the bill. Every item in the rented layer comes with a ceiling; that ceiling is not set by the amount paid, it is set by the **quota** the provider places. The quota mechanism itself was measured in M22/K03 and is not repeated here. This lesson's question belongs to the scaling plan: **how many units remain before each limit**, which one fills first as growth continues, and what happens once it does. #### The Limit Table and the Growth Plan The example is the fictional **regional measurement network**: five teams, three environments, a rented machine, network, store, function, and edge layer. Limits, growth rates, and amounts are fictional; **the currency is fictional too** and is called `unit`, corresponding to no real price list. **CC25:** there are eight limits; every limit has a **scope** (account, region, network, partition) and a **driver** that consumes it. **CC26:** the scope decides how many teams share the limit. **CC27:** the growth plan is per driver; every driver has its own monthly growth rate. **CC28:** every limit has an increasable ceiling and an increase-request duration; if the ceiling equals the limit, the limit **cannot be increased**. **CC29:** hitting a limit has four behaviors — creation-denial, request-denial, queue, silent throttling — and **the day it is noticed depends on the behavior**. **CC30:** a daily loss accrues while a limit stays full; on a limit that cannot be increased, the loss runs for thirty days, that is, until the plan changes. The limit table and growth plan are a process-internal **model**; there is no real limit list. ```js // cost/limit.mjs — quota and service limit table model: no real limit list. Every row is a // limit, tied to a *driver* that consumes it and to what happens on impact. The quota // mechanism itself was measured in M22/K03; what is measured here is the growth plan applied // to this table. The currency is fictional and is called `unit`. // name, scope, driver, current usage, limit, increasable ceiling, increase-request days, // behavior, daily loss while the limit stays full (unit) export const LIMIT = [ ["machine cores per region", "region", "counter", 412, 640, 2000, 6, "creation-denial", 34], ["object writes per second", "partition", "request", 1850, 3500, 12000, 4, "request-denial", 61], ["concurrent function executions", "account", "request", 640, 1000, 4000, 3, "queue", 18], ["name resolutions per second", "network", "request", 7200, 10000, 10000, 0, "silent throttling", 27], ["static addresses per account", "account", "edge", 34, 50, 200, 5, "creation-denial", 9], ["virtual networks per account", "account", "environment", 9, 12, 40, 7, "creation-denial", 4], ["retention rules per partition", "partition", "data", 62, 100, 100, 0, "creation-denial", 6], ["rules per rule set", "network", "service", 148, 200, 400, 9, "creation-denial", 12], ]; // growth plan: each driver's monthly growth rate (fictional) export const GROWTH = { counter: 0.06, request: 0.11, data: 0.09, edge: 0.05, environment: 0.02, service: 0.04 }; // scope decides how many teams share the limit: an account-, region-, or network-scoped limit // has one team's growth consume the others' headroom export const SHARED = { account: 5, region: 5, network: 5, partition: 1 }; // behavior on hitting the limit: day noticed, and why it takes that long export const BEHAVIOR = { "creation-denial": [1, "request fails on the spot"], "request-denial": [1, "the caller sees the error"], "queue": [4, "leaks out as latency"], "silent throttling": [12, "produces no error at all"], }; export const fillMonth = (usage, limit, g) => g <= 0 ? Infinity : Math.log(limit / usage) / Math.log(1 + g); export const sum = (v) => v.reduce((a, b) => a + b, 0); ``` ```js // cost/plan.mjs — the growth plan applied to the limit table: how many units remain before each // limit, which limit fills first, whether the first-filling limit's identity changes when request // growth is scanned, and the daily cost of hitting a limit. Numbers come from the model alone. import { LIMIT, GROWTH, SHARED, BEHAVIOR, fillMonth, sum } from "./limit.mjs"; const write = (w, ...s) => console.log(s.map((v, i) => (w[i] < 0 ? String(v).padEnd(-w[i]) : String(v).padStart(w[i]))).join("")); const t0 = process.hrtime.bigint(); console.log(`${LIMIT.length} limits, ${Object.keys(GROWTH).length} drivers ` + `(fictional limit table and fictional growth plan)`); console.log("\n1. how many units remain before each limit"); const A = [-33, -11, 7, 10, 9, 11, 10, 13]; write(A, "limit", "scope", "teams", "usage", "limit", "remaining", "fill", "driver"); for (const [name, scope, driver, use, lim] of LIMIT) write(A, name, scope, SHARED[scope], use, lim, lim - use, `${(100 * use / lim).toFixed(1)}%`, driver); const fillAvg = sum(LIMIT.map(([, , , u, l]) => u / l)) / LIMIT.length; const isShared = LIMIT.filter(([, scope]) => SHARED[scope] > 1); console.log(` average fill ${(100 * fillAvg).toFixed(1)}%; fullest ` + `${(100 * Math.max(...LIMIT.map(([, , , u, l]) => u / l))).toFixed(1)}%`); console.log(` ${isShared.length}/${LIMIT.length} limits are shared by five teams: one team's ` + `growth consumes another's headroom`); const fixed = LIMIT.filter(([, , , , l, ceiling]) => ceiling <= l); console.log(` ${fixed.length} limits cannot be increased (ceiling = limit): ` + fixed.map((x) => x[0]).join(", ")); console.log("\n2. applying the growth plan: which limit fills first"); const B = [-33, 9, 12, 11, 11, 11]; write(B, "limit", "growth", "fill month", "month 12", "ceiling x", "increase"); const order = LIMIT.map((x) => [x, fillMonth(x[3], x[4], GROWTH[x[2]])]) .sort((a, b) => a[1] - b[1]); for (const [[name, scope, driver, use, lim, ceiling, days], n] of order) { const twelve = use * Math.pow(1 + GROWTH[driver], 12); write(B, name, `${(100 * GROWTH[driver]).toFixed(0)}%`, n === Infinity ? "-" : n.toFixed(1), `${(100 * twelve / lim).toFixed(0)}%`, (ceiling / lim).toFixed(2) + "x", ceiling <= lim ? "none" : `${days} days`); } const twelveMonth = LIMIT.filter(([, , driver, use, lim]) => use * Math.pow(1 + GROWTH[driver], 12) > lim); console.log(` fills first: ${order[0][0][0]} (month ${order[0][1].toFixed(1)}); ` + `second ${order[1][0][0]} (month ${order[1][1].toFixed(1)})`); console.log(` the first to fill is not the fullest limit: the fullest is ` + `${LIMIT.reduce((a, b) => (a[3] / a[4] >= b[3] / b[4] ? a : b))[0]}`); console.log(` ${twelveMonth.length}/${LIMIT.length} limits are exceeded by month 12`); console.log("\n3. scanning request growth: does the first-filling limit change"); const C = [9, 33, 7, 33, 7]; write(C, "request", "first to fill", "month", "second to fill", "month"); for (const g of [0.02, 0.05, 0.08, 0.11, 0.16, 0.25]) { const grown = { ...GROWTH, request: g }; const s = LIMIT.map((x) => [x, fillMonth(x[3], x[4], grown[x[2]])]).sort((a, b2) => a[1] - b2[1]); write(C, `${(100 * g).toFixed(0)}%`, s[0][0][0], s[0][1].toFixed(1), s[1][0][0], s[1][1].toFixed(1)); } console.log(` the request driver feeds three limits; changing its rate changes not just the`); console.log(` date but the *identity* of the limit that fills first`); console.log("\n4. hitting a limit: what happens, and what is lost until it is increased"); const D = [-33, 19, 9, 10, 8, 12]; write(D, "limit", "behavior", "noticed", "increase", "days", "lost units"); let totalLoss = 0, totalDays = 0; for (const [name, , , , lim, ceiling, days, behavior, gk] of LIMIT) { const [notice] = BEHAVIOR[behavior]; const span = ceiling <= lim ? notice + 30 : notice + days; // if it cannot be increased, the loss continues totalLoss += gk * span; totalDays += span; write(D, name, behavior, notice, ceiling <= lim ? "none" : days, span, (gk * span).toFixed(0)); } console.log(` total ${totalLoss.toFixed(0)} units lost, ${totalDays} limit-days; ` + `average ${(totalDays / LIMIT.length).toFixed(1)} days/limit`); const silent = LIMIT.filter((x) => x[7] === "silent throttling"); const instant = LIMIT.filter((x) => BEHAVIOR[x[7]][0] === 1); console.log(` when it is noticed depends on behavior: ${instant.length} limits are seen the ` + `same day, silent throttling on day ${BEHAVIOR["silent throttling"][0]}`); const sk = sum(silent.map((x) => x[8] * (12 + 30))); console.log(` silent throttling's loss is ${sk.toFixed(0)} units, ` + `${(100 * sk / totalLoss).toFixed(1)}% of the total; that limit cannot be increased, the plan has to change`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, only the band is checked console.log(`\nfour tables within the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`); ``` ``` 8 limits, 6 drivers (fictional limit table and fictional growth plan) 1. how many units remain before each limit limit scope teams usage limit remaining fill driver machine cores per region region 5 412 640 228 64.4% counter object writes per second partition 1 1850 3500 1650 52.9% request concurrent function executions account 5 640 1000 360 64.0% request name resolutions per second network 5 7200 10000 2800 72.0% request static addresses per account account 5 34 50 16 68.0% edge virtual networks per account account 5 9 12 3 75.0% environment retention rules per partition partition 1 62 100 38 62.0% data rules per rule set network 5 148 200 52 74.0% service average fill 66.5%; fullest 75.0% 6/8 limits are shared by five teams: one team's growth consumes another's headroom 2 limits cannot be increased (ceiling = limit): name resolutions per second, retention rules per partition 2. applying the growth plan: which limit fills first limit growth fill month month 12 ceiling x increase name resolutions per second 11% 3.1 252% 1.00x none concurrent function executions 11% 4.3 224% 4.00x 3 days retention rules per partition 9% 5.5 174% 1.00x none object writes per second 11% 6.1 185% 3.43x 4 days machine cores per region 6% 7.6 130% 3.13x 6 days rules per rule set 4% 7.7 118% 2.00x 9 days static addresses per account 5% 7.9 122% 4.00x 5 days virtual networks per account 2% 14.5 95% 3.33x 7 days fills first: name resolutions per second (month 3.1); second concurrent function executions (month 4.3) the first to fill is not the fullest limit: the fullest is virtual networks per account 7/8 limits are exceeded by month 12 3. scanning request growth: does the first-filling limit change request first to fill month second to fill month 2% retention rules per partition 5.5 machine cores per region 7.6 5% retention rules per partition 5.5 name resolutions per second 6.7 8% name resolutions per second 4.3 retention rules per partition 5.5 11% name resolutions per second 3.1 concurrent function executions 4.3 16% name resolutions per second 2.2 concurrent function executions 3.0 25% name resolutions per second 1.5 concurrent function executions 2.0 the request driver feeds three limits; changing its rate changes not just the date but the *identity* of the limit that fills first 4. hitting a limit: what happens, and what is lost until it is increased limit behavior noticed increase days lost units machine cores per region creation-denial 1 6 7 238 object writes per second request-denial 1 4 5 305 concurrent function executions queue 4 3 7 126 name resolutions per second silent throttling 12 none 42 1134 static addresses per account creation-denial 1 5 6 54 virtual networks per account creation-denial 1 7 8 32 retention rules per partition creation-denial 1 none 31 186 rules per rule set creation-denial 1 9 10 120 total 2195 units lost, 116 limit-days; average 14.5 days/limit when it is noticed depends on behavior: 6 limits are seen the same day, silent throttling on day 12 silent throttling's loss is 1134 units, 51.7% of the total; that limit cannot be increased, the plan has to change four tables within the 2000 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Whose Name Is on the Remaining Headroom The first table counts the headroom left on eight limits. Average fill is **66.5%**, the fullest is virtual networks per account (75.0%), with **three units** of headroom remaining — a team wanting a fourth environment instead of three spends those three units. Who the headroom belongs to is written into scope. **Six of the eight limits** are scoped to an account, region, or network, and shared by five teams. When one team claims a new static address, everyone else's headroom shrinks; no one sees anyone else's consumption. Only two limits are partition-scoped and belong to a single team. This is the reverse of the earlier lessons' allocation question: there, cost could not be split across teams; here, **the limit is already unsplit**, and whoever arrives first consumes it. Two limits' ceiling equals their own value: name resolutions per second and retention rules per partition **cannot be increased**. For these, "how many units remain" has only one meaning, because there is no way for the number to grow. #### The First Limit to Fill Is Not the Fullest The second table runs the growth plan. The order comes out not by fill level but by **the combination of fill level and growth rate**: the first to fill is name resolutions per second, 72.0% full but 11% monthly growth, **month 3.1**; second is concurrent function executions (64.0% full, 11% growth, month 4.3). The fullest limit, virtual networks per account, does not fill until **month 14.5**, because the environment count that consumes it grows only 2% a month. A quota report sorted by fill percentage produces exactly the reverse priority list on this network. By month twelve, **seven of the eight limits** are exceeded. The overrun's size also differs: name resolutions reaches **252%** of its limit, function concurrency 224%. The ceiling-multiplier column gives the counterpart: function concurrency can be raised fourfold and survives twelve months, name resolutions cannot be increased at all. #### Growth Rate Changes the Limit's Identity The third table scans a single driver. The request rate feeds three limits at once, making it the plan's most uncertain variable. At 2% and 5%, the first to fill is **retention rules per partition** (month 5.5), which has nothing to do with requests; it consumes data volume. From 8% on, name resolutions takes first place, dropping to month 1.5 at 25%. What changes is not just the date but **the limit's identity**: slow growth means preparing for retention rules, fast growth means name resolutions, and the two preparations share nothing — one wants a retention policy, the other an architecture change. The growth rate is an estimate. What this table says is that the cost of a wrong estimate is **not preparing for the wrong date, it is preparing for the wrong limit**. #### The Behavior of Hitting the Limit and the Time to Increase It The fourth table counts what happens after a limit fills. The day it is noticed depends on the behavior: creation-denial and request-denial produce an error the caller sees and are noticed **the same day**; a queue leaks out as latency (4 days); silent throttling produces no error at all and runs **12 days**. The increase request's duration adds on top: between 3 and 9 days. The total comes to **116 limit-days** and **2,195 units** lost, averaging 14.5 days per limit. Object write requests close out fastest (5 days, 305 units): noticed immediately, increased within four days. **51.7% of the number comes from a single row**: name resolutions per second. Silent throttling goes unnoticed for 12 days, and since the limit cannot be increased, the loss is not cut short by a single request; it runs 42 days and comes to **1,134 units**. A limit that cannot be increased has no "increase duration" — instead there is a **plan-change duration**, not measured here. #### Summary - Eight limits average **66.5%** full; the fullest has three units of headroom remaining. **Six are shared by five teams**: the limit is unsplit, and whoever grows first consumes the headroom. - Two limits can never be increased (ceiling = limit); for these, the remaining-units number has only one meaning. - **The first limit to fill is not the fullest**: the first is name resolutions, 72.0% full but growing 11% (month 3.1); the fullest limit, at 75.0%, does not fill until month 14.5 because it grows only 2%. - By the end of twelve months, **seven of the eight limits** are exceeded; name resolutions reaches 252% of its limit. - Scanning request growth from 2% to 25% changes the **identity** of the first limit to fill — from retention rules to name resolutions. A wrong estimate's cost is not preparing for the wrong date, it is preparing for **the wrong limit**. - The cost of hitting a limit is behavior plus increase duration: 116 limit-days, 2,195 units. **51.7% of that** comes from a single silent-throttling limit — unnoticed for 12 days, unable to be increased, running 42 days and totaling 1,134 units. #### Next Step The same thing sits in every row: the rented side sets the limit's value, whether it can be increased, and how many days that takes. On the two limits that cannot be increased, the only way out is changing the plan, and that change has stopped being the network's own decision. The next lesson reads this dependency's total: how many items can be moved, what replaces the ones that cannot, and what moving costs on the bill. ### Vendor Lock-In Source: https://academia.sh/en/courses/cloud-governance/vendor-lock-in The twelve rented items split into portability degrees, and it turns out bill weight never measures portability: 48.3% of the monthly bill comes from two items portable through a standard interface, 9.6% from three items that cannot be moved at all. The exit bill is 879 fictional units, 52% of it request fee rather than bytes; yet that is only 9.1% of the move cost, the rest is 257 person-days of work. The three paths to reducing lock-in are measured by prep, move, and capability lost: the lowest break-even point is 48.9%, and one path's break-even is 466% — it does not pay off even if the move is certain. The previous lesson folded limits into the scaling plan, and the same thing stood in every row: the rented side sets both the limit's value and whether it can be increased. This lesson's question sits at the end of that sentence — how possible is **going somewhere else**, and what it costs. #### Items, Degrees, and the Exit Bill The example is the fictional **regional measurement network**: five teams, three environments, a rented machine, network, store, function, and edge layer. Items and amounts are fictional; **the currency is fictional too** and is called `unit`. **CC31:** twelve items split into four **portability degrees** — moved through a standard interface, moved with adaptation, requiring a rewrite, non-portable. **CC32:** every item has a move workload; on non-portable items this number is the workload of **what gets set up in its place**. **CC33:** every item uses a set of provider-specific capabilities, and prep costs some of them. **CC34:** the exit fee is a one-time charge made of two parts — bytes and requests. **CC35:** the horizon is 36 months; prep is paid across that span, the move only if the move actually happens. The items and paths are a process-internal **model**. ```js // cost/portability.mjs — portability model of the rented items: no real inventory. Every item // is tied to a *portability degree*, a move workload, and a count of provider-specific // capabilities. The currency is fictional and is called `unit`; unit prices are taken from the // fictional list in this topic's first lesson. export const PERSON_DAY = 34; // fictional cost of one person-day (unit) export const EGRESS_GB = 0.082, READ = 0.36; // per-GB egress, per-million-requests read export const HORIZON = 36; // evaluation horizon (months) // name, degree, work (person-days), monthly unit, GB, objects (millions), special capability, replaced by export const ITEM = [ ["raw measurement store", "standard interface", 4, 6.5, 308.6, 1244.16, 0, "-"], ["rented machine", "standard interface", 6, 535.7, 0, 0, 0, "-"], ["snapshot", "adaptation", 9, 102.7, 2140, 0.4, 2, "-"], ["record store", "adaptation", 7, 23.1, 1360, 8, 1, "-"], ["private connectivity", "adaptation", 9, 46.0, 0, 0, 3, "-"], ["managed array store", "adaptation", 18, 196.0, 640, 3, 4, "-"], ["event queue", "rewrite", 22, 58.0, 90, 6, 5, "-"], ["function execution", "rewrite", 34, 47.2, 0, 0, 6, "-"], ["identity and policy", "rewrite", 41, 0, 0, 0, 7, "-"], ["key management", "non-portable", 38, 34.0, 0, 0, 3, "own key store"], ["audit log pipeline", "non-portable", 24, 23.1, 620, 5, 2, "own log pipeline"], ["threat detection", "non-portable", 45, 51.0, 0, 0, 4, "rule writing and maintenance"], ]; export const DEGREE = ["standard interface", "adaptation", "rewrite", "non-portable"]; // paths to reduce lock-in. Multipliers apply to the four degrees in order: with the prep done, how much of // the move workload remains. The exit multiplier drops if data no longer sits with the other provider. // name, one-time person-days, monthly person-days, monthly extra units, multipliers, exit, capability lost export const PATH = [ ["nothing", 0, 0.0, 0.0, [1.00, 1.00, 1.00, 1.00], 1.00, 0.00], ["abstraction layer", 25, 0.6, 0.0, [1.00, 0.25, 0.35, 1.00], 1.00, 0.45], ["run on two providers", 45, 0.0, 512.3, [0.00, 0.00, 0.50, 1.00], 0.05, 0.75], ["common-feature only", 0, 3.2, 0.0, [0.60, 0.30, 0.35, 0.00], 1.00, 0.85], ]; export const sum = (v) => v.reduce((a, b) => a + b, 0); export const bucket = (d) => ITEM.filter((k) => k[1] === d); ``` ```js // cost/exit.mjs — items are split by portability degree, the exit bill is totaled, and the three // paths to reduce lock-in are measured by prep, move, and the *capability lost*; move probability // is scanned to work out each path's break-even point. Numbers come from the model alone. import { ITEM, DEGREE, PATH, PERSON_DAY, EGRESS_GB, READ, HORIZON, sum, bucket } from "./portability.mjs"; const write = (w, ...s) => console.log(s.map((v, i) => (w[i] < 0 ? String(v).padEnd(-w[i]) : String(v).padStart(w[i]))).join("")); const t0 = process.hrtime.bigint(); const MONTHLY = sum(ITEM.map((k) => k[3])), WORK = sum(ITEM.map((k) => k[2])); const GB = sum(ITEM.map((k) => k[4])), OBJ = sum(ITEM.map((k) => k[5])); const CAP = sum(ITEM.map((k) => k[6])), EXIT = GB * EGRESS_GB + OBJ * READ; console.log(`${ITEM.length} rented items, monthly ${MONTHLY.toFixed(1)} units (fictional currency)`); console.log("\n1. portability degree, bill weight, and exit bill"); const A = [-20, 7, 6, 15, 12, 12, 8, 14, 12]; write(A, "degree", "items", "work", "monthly units", "bill share", "capability", "GB", "objects (mn)", "exit units"); for (const d of DEGREE) { const g = bucket(d), br = sum(g.map((k) => k[3])); const gb = sum(g.map((k) => k[4])), n = sum(g.map((k) => k[5])); write(A, d, g.length, sum(g.map((k) => k[2])), br.toFixed(1), `${(100 * br / MONTHLY).toFixed(1)}%`, sum(g.map((k) => k[6])), gb.toFixed(0), n.toFixed(2), (gb * EGRESS_GB + n * READ).toFixed(1)); } write(A, "total", ITEM.length, WORK, MONTHLY.toFixed(1), "100%", CAP, GB.toFixed(0), OBJ.toFixed(2), EXIT.toFixed(1)); console.log(" where the non-portable items are replaced: " + bucket("non-portable").map((k) => k[7]).join(", ")); const raw = ITEM[0]; console.log(` ${(100 * OBJ * READ / EXIT).toFixed(0)}% of the exit bill is request fee, not bytes: raw measurement`); console.log(` store drops ${(raw[4] * EGRESS_GB).toFixed(1)} units of bytes against ` + `${(raw[5] * READ).toFixed(1)} units of requests`); console.log(` exit ${EXIT.toFixed(0)} units, ${(100 * EXIT / MONTHLY).toFixed(0)}% of the monthly bill; ` + `work ${(WORK * PERSON_DAY).toFixed(0)} units`); console.log(` exit is ${(100 * EXIT / (EXIT + WORK * PERSON_DAY)).toFixed(1)}% of the ` + `total move cost`); const measure = (y) => { const [, oneTime, monthlyDays, monthlyUnit, mult, exitMult, lost] = y; const work = sum(DEGREE.map((d, i) => sum(bucket(d).map((k) => k[2])) * mult[i])); return { prep: oneTime * PERSON_DAY + HORIZON * (monthlyDays * PERSON_DAY + monthlyUnit), work, move: work * PERSON_DAY + EXIT * exitMult, lost: CAP * lost }; }; console.log("\n2. three paths to reduce lock-in (36-month horizon)"); const C = [-23, 13, 16, 12, 12, 12]; write(C, "path", "prep units", "remaining work", "move units", "capability", "break-even"); const base = measure(PATH[0]); for (const y of PATH) { const o = measure(y), p = y === PATH[0] ? null : o.prep / (base.move - o.move); write(C, y[0], o.prep.toFixed(0), o.work.toFixed(1), o.move.toFixed(0), o.lost ? `-${o.lost.toFixed(0)}` : "0", p === null ? "-" : `${(100 * p).toFixed(1)}%`); } console.log(` ${CAP} special capabilities exist; every path buys its move cost with capability`); console.log("\n3. scanning move probability: expected cost and the cheapest path"); const D = [13, 12, 20, 23, 22, 23]; write(D, "probability", ...PATH.map((y) => y[0]), "cheapest"); for (const p of [0.05, 0.25, 0.50, 0.70, 1.00]) { const v = PATH.map((y) => { const o = measure(y); return o.prep + p * o.move; }); write(D, `${(100 * p).toFixed(0)}%`, ...v.map((x) => x.toFixed(0)), PATH[v.indexOf(Math.min(...v))][0]); } const ps = PATH.slice(1).map((y) => [y[0], measure(y).prep / (base.move - measure(y).move)]); const best = ps.reduce((a, b) => (a[1] <= b[1] ? a : b)); console.log(` lowest break-even ${best[0]}: ${(100 * best[1]).toFixed(1)}%; below it no ` + `prep pays off`); console.log(` ${ps.filter((x) => x[1] > 1).length} path's break-even is above 100%: it does not ` + `pay off even if the move is certain`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, only the band is checked console.log(`\nthree tables within the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`); ``` ``` 12 rented items, monthly 1123.3 units (fictional currency) 1. portability degree, bill weight, and exit bill degree items work monthly units bill share capability GB objects (mn) exit units standard interface 2 10 542.2 48.3% 0 309 1244.16 473.2 adaptation 4 43 367.8 32.7% 10 4140 11.40 343.6 rewrite 3 97 105.2 9.4% 18 90 6.00 9.5 non-portable 3 107 108.1 9.6% 9 620 5.00 52.6 total 12 257 1123.3 100% 37 5159 1266.56 879.0 where the non-portable items are replaced: own key store, own log pipeline, rule writing and maintenance 52% of the exit bill is request fee, not bytes: raw measurement store drops 25.3 units of bytes against 447.9 units of requests exit 879 units, 78% of the monthly bill; work 8738 units exit is 9.1% of the total move cost 2. three paths to reduce lock-in (36-month horizon) path prep units remaining work move units capability break-even nothing 0 257.0 9617 0 - abstraction layer 1584 161.7 6377 -17 48.9% run on two providers 19973 155.5 5331 -28 466.0% common-feature only 3917 52.8 2676 -31 56.4% 37 special capabilities exist; every path buys its move cost with capability 3. scanning move probability: expected cost and the cheapest path probability nothing abstraction layer run on two providers common-feature only cheapest 5% 481 1903 20239 4051 nothing 25% 2404 3179 21306 4586 nothing 50% 4808 4773 22638 5255 abstraction layer 70% 6732 6048 23704 5790 common-feature only 100% 9617 7961 25304 6593 common-feature only lowest break-even abstraction layer: 48.9%; below it no prep pays off 1 path's break-even is above 100%: it does not pay off even if the move is certain three tables within the 2000 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Bill Weight Does Not Measure Portability The first table lines up two rankings, and they run opposite. **48.3% of the bill** comes from the two standard-interface items, whose workload is **10 person-days**. The three items requiring a rewrite hold 9.4% of the bill but 97 person-days; the three non-portable ones hold 9.6% and demand 107 person-days. The same inverse relationship shows up in the capability column: the two items holding half the bill use **zero** provider-specific capabilities, the seven items holding 19% use all 27. Lock-in accumulates not with money spent but with **the feature used**. What replaces the three non-portable items is written into their names: an own key store, an own log pipeline, rule writing — not a service in any of the three, but **ongoing work**. #### Where the Exit Bill Accumulates The exit bill is **879 units** — 78% of the monthly bill, paid once. **52% of it is request fee, not bytes**, and one item alone produces the difference: raw measurement store carries 25.3 units of bytes against **447.9 units** of requests. The splitting decision from this topic's first lesson is paid a second time here. But exit is not the move's biggest item: 257 person-days of work run **8,738 units**, and exit is only **9.1% of the total**. Exit shows up on a single line; the real item never shows up on any bill at all. #### The Price of Prep Is Capability The second table measures three ways to reduce lock-in. The **abstraction layer** costs 1,584 units over 36 months, drops the move from 9,617 to 6,377 units, and costs **17 of 37** capabilities. **Running on two providers at once** drops the move to 5,331 units and all but zeroes the exit, but its prep is **19,973 units** — twice the price of the move itself. **Using only common features** gives the cheapest move (2,676 units), prep of 3,917, and takes **31 of 37** capabilities. What the three paths share sits in that last column: none of them buys portability with money, **all of them trade it for capability**. None touches the three non-portable items. #### Where Lock-In Is Acceptable The third table scans move probability. **At 5% and 25%, the cheapest path is doing nothing**; at 50% the abstraction layer takes over, at 70% and above the common-feature path leads. The lowest break-even is **48.9%**: if the three-year move probability sits below that, no prep pays for itself. Running on two providers breaks even at **466%**; being meaningless as a probability is the result itself. Because a second three-year bill costs more than a one-time move, that path does not pay off even if the move is certain. In this table, lock-in is not a mistake, it is a **price**; the measure is not "are we locked in" but **"is our move probability above 48.9%."** #### Summary - Bill weight does not measure portability: **48.3% of the bill** comes from the two items that move through a standard interface, and those two use zero special capabilities; seven items holding 19% use 27 capabilities. - Three items never move at all; what replaces them is not a service but **ongoing work** (107 person-days). - The exit bill is **879 units**, and 52% of it is request fee — seventeen times the bytes in raw measurement store. Still, it is only **9.1% of the move**; 257 person-days of work at 8,738 units is the real item, and it never shows up on any bill. - All three paths **trade portability for capability** (17, 28, 31 capabilities), and none touches the three non-portable items. The lowest break-even is **48.9%**, the highest **466%**. #### Course Wrap-Up | Lesson | Who can do / see it | Limit and remaining | The cost of narrowing or measuring | |---|---|---|---| | identity/01 | group grants 231 permissions, 157 needed | 3 of 4 departed cannot be closed | narrowing drops 142 legitimate permissions | | identity/02 | a single wildcard opens 132x the scope | explicit deny wins on 88 requests | the condition blocks 130 legitimate reads | | identity/03 | 131 of 840 requests are permitted | 555 requests exceed the account limit | removing the documents stalls 15 jobs | | identity/04 | 17 of 30 pairs are on the trust list | 480 remain before the call limit | closing the role stalls 90 operations a day | | identity/05 | 79 key copies, 49 unregistered | 3 of 30 instances cannot get an identity | host-scoped grants 54 excess permissions | | identity/06 | 46 items open, 27 needed | safe threshold 275 days, observed 90 | a 1-day window stalls 8 paths | | identity/07 | blast radius 24 → 8 → 3 | quota overrun 1,142, 580 remaining | 24 documents grow to 57 | | security/01 | decrypting pairs drop from 45 to 20 | 43 days remain in the key period | 132 legitimate reads a day stop | | security/02 | 4 of 16 legs are unknown | 29 days remain in the transition window | 34 clients cannot be upgraded | | security/03 | a human read 34 secrets in 90 days | 17 secrets were never rotated | the zero-miss window is 601,584 min | | security/04 | 273 of 480 events, 88 by default | 11 subjects can delete the record | a separate account stalls 6 legitimate jobs | | security/05 | 32 of 46 firings are real | scan quota 240, 108 remaining | remediation mode breaks 8 configurations | | security/06 | 14 of 25 accesses come from the open internet | 12 of 18 endpoints full, 6 idle | closing it breaks 12 legitimate accesses | | security/07 | 9 of 11 malicious, 8 from a single source | aggregation 60 GB/day, 5 remaining | at threshold 3, caught drops to 7 | | security/08 | 7 of 26 evidence items come from the provider | evidence store 12 GB, 4.28 remaining | 139 units a year, 160 person-hours | | cost/01 | 50.8% of the bill is a platform decision | 14 splits exist, 90 needed | the request crossover is 693,370 items | | cost/02 | valid coverage 75.0% | 10 tag keys, 3 remaining | the mandate stalls 511.9 units of work | | cost/03 | 86.4% of the budget is gone by the first alert | 204 units remain | the alert is open 68 of 90 days | | cost/04 | p95 occupancy 31.4% | 16 tier steps remain on the ladder | 88.0% of savings require a disruption | | cost/05 | 6/8 limits shared by five teams | the fullest has 3 units remaining | 116 limit-days, 2,195 units | | cost/06 | 48.3% of the bill is standard-interface portable | 3 items cannot move, 37 special capabilities | break-even does not pay off below 48.9% | | Course | Measure axis | Unit of measure | |---|---|---| | K01 Cloud Fundamentals | the cost of renting | delegated decision | | K02 Compute and Networking in the Cloud | the delegated decision in a concrete item | irreversible item | | K03 Storage and Data in the Cloud | when data is rented | egress and transfer | | K04 Serverless and Edge Computing | the delegated decision comes back as a constraint | constraint item | | K05 Cloud Governance, Security and Cost | the rented thing's owner and limit | authority, visibility, quota | The one question asked throughout M23 was **what you delegate and what you give up in return**. Five courses measured the same rented network five times, and every measurement took the same shape: a decision got delegated, an item became irreversible. In this course, that item gained an owner and a limit. The **exit** column carried through all five courses, and its total is read here: three non-portable items, 257 person-days of work, an 879-unit exit bill, a 48.9% break-even. Lock-in is not a flaw in renting, it is its **price**.