--- title: 'Platform Metrics' source: 'https://academia.sh/en/courses/platform-engineering/platform-metrics' course: 'Platform Engineering' language: en updated: '2026-09-13T05:08:29+00:00' license: 'CC BY-SA 4.0' objectives: - 'Ties adoption to three numbers: how many teams, how many capabilities, how much usage per capability' - 'Decomposes the gap in delivery metrics and names the confounder: selection bias' - "Counts the platform's own metric and shows two cases where the metric points the wrong way" --- # Platform Metrics Measuring whether the platform actually works: only twenty-five of forty team-capability pairs are filled, two capabilities costing fifty-four work-days serve at most one team, 48%-77% of the raw gap in four delivery metrics comes from selection bias and the difference-in-differences stays smaller than the within-group spread in all four, the same 550-minute outage turns into 2,750 team-minutes under central adoption, and forcing adoption improves the metric by 60% while making work per usage 18% worse. The previous lesson moved five teams in waves and counted who was left behind, by name. Throughout this topic, every lesson counted **what the platform did**; none of them asked whether any of it actually made teams' work faster. The four delivery metrics were defined in M22/K01 and are not repeated here; they are the input. What is measured is not the metrics themselves, but **where the gap in the metric comes from**. The through-line is again the fictional **regional measurement network** and its five fictional teams. ## Tying Adoption to a Single Number **PC37:** each of the eight capabilities has its setup work, annual maintenance, number of teams using it, and usage count written down. **PC38:** the five teams' four delivery metrics are read before and after adoption. **PC39:** three teams have adopted; the order comes from the migration wave, meaning the teams with no obstacle migrated first. **PC40:** the platform is a dependency; over twelve months, outages, their durations, the number of teams on the platform at that moment, and the source are written down. **PC41:** if adoption is forced, every team-capability pair brings one unit of compliance work, and only a portion of the forced ones produce usage. **PC42:** annual support requests per team and the platform team's capacity are written down. Capabilities, metrics, and outages are a **model** built within the lesson; there is no randomness. ```js // metric/data.mjs — platform metrics model: there is no real metrics pipeline; capabilities, // team metrics, and outages are a data structure, with no randomness. Metric *definitions* were // established in M22/K01 and are an input here. Work-days are fictional. export const TEAM = ["meter-reading", "validation", "billing", "field", "reporting"]; export const ADOPTED = [0, 1, 4]; // comes from the migration wave: teams with no obstacle went first export const NOT_ADOPTED = [2, 3]; // critical period and incompatible requirement // capability: name, setup work, annual maintenance (work-days), teams using it, usage in the period export const CAPABILITY = [ ["environment on demand", 34, 7, 4, 60], ["scaffold generation", 22, 5, 5, 5], ["service catalog", 18, 6, 3, 5], ["policy gate", 27, 8, 5, 72], ["showback", 15, 4, 3, 30], ["migration tool", 31, 3, 4, 4], ["secret rotation", 19, 5, 0, 0], ["shared dashboard library", 24, 6, 1, 3], ]; // metric: name, unit, direction (+1 higher is better, -1 lower is better), before adoption, after (team order) export const METRIC = [ ["deployment frequency", "week", +1, [4.0, 3.5, 1.6, 1.1, 2.8], [8.4, 4.6, 2.4, 1.5, 5.9]], ["lead time", "hours", -1, [52, 61, 96, 120, 47], [18, 52, 84, 112, 12]], ["failure rate", "%", -1, [14, 11, 18, 21, 9], [6, 9.5, 15.5, 19, 4]], ["time to restore", "min", -1, [190, 145, 240, 310, 160], [95, 130, 215, 300, 70]], ]; // outage: month, duration (min), number of teams on the platform at that time, source export const OUTAGE = [[2, 45, 1, "platform"], [4, 120, 2, "lower layer"], [6, 25, 3, "platform"], [8, 210, 3, "platform"], [10, 60, 3, "lower layer"], [12, 90, 3, "platform"]]; export const REQUEST_PER_TEAM = 34, CAPACITY = 90; // annual requests per team / platform capacity export const COMPLY_WORK = 4, FORCED_USERS = 4, FORCED_USAGE = 3; // the cost of forcing adoption export const sum = (v) => v.reduce((a, b) => a + b, 0); const avg = (v) => sum(v) / v.length; const group = (g, d, y) => avg(g.map((i) => d[i])) * y; // for one metric: raw gap, selection bias, difference-in-differences, and the adopting group's own internal spread export function decompose([, , direction, before, after]) { const d = (i) => (after[i] - before[i]) * direction; // improvement, sign-corrected const adoptedDiff = ADOPTED.map(d); return { afterGap: group(ADOPTED, after, direction) - group(NOT_ADOPTED, after, direction), beforeGap: group(ADOPTED, before, direction) - group(NOT_ADOPTED, before, direction), diffInDiff: avg(adoptedDiff) - avg(NOT_ADOPTED.map(d)), spread: Math.max(...adoptedDiff) - Math.min(...adoptedDiff) }; } ``` ```js // metric/measure.mjs — adoption breakdown, decomposition of the four delivery metrics, the // platform's own metric, and what forcing adoption does to the metric. import { TEAM, ADOPTED, CAPABILITY, METRIC, OUTAGE, REQUEST_PER_TEAM, CAPACITY, COMPLY_WORK, FORCED_USERS, FORCED_USAGE, sum, decompose } from "./data.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const t0 = process.hrtime.bigint(); const WORK = sum(CAPABILITY.map((y) => y[1] + y[2])), USAGE = sum(CAPABILITY.map((y) => y[4])); const pairs = sum(CAPABILITY.map((y) => y[3])), totalPairs = CAPABILITY.length * TEAM.length; console.log(`${TEAM.length} teams, ${CAPABILITY.length} capabilities, ${ADOPTED.length} adopters`); console.log("\n1. adoption: teams and usage per capability"); const A = [-26, 8, 8, 9, 11, 14]; write(A, "capability", "setup", "maint.", "teams", "usage", "usage/work"); for (const [name, k, b, e, u] of CAPABILITY) write(A, name, k, b, `${e}/${TEAM.length}`, u, u ? ((k + b) / u).toFixed(2) : "-"); write(A, "total", sum(CAPABILITY.map((y) => y[1])), sum(CAPABILITY.map((y) => y[2])), `${pairs}/${totalPairs}`, USAGE, (WORK / USAGE).toFixed(2)); const idle = CAPABILITY.filter((y) => y[3] <= 1), idleWork = sum(idle.map((y) => y[1] + y[2])); console.log(` adoption ${(100 * pairs / totalPairs).toFixed(1)}%, ` + `${(pairs / CAPABILITY.length).toFixed(2)} teams/capability; least-used ` + `${idle.length} capabilities ${idleWork} work-days (${(100 * idleWork / WORK).toFixed(0)}%)`); console.log("\n2. four delivery metrics: does the gap come from the platform"); const B = [-29, 11, 17, 14, 10, 10]; write(B, "metric", "raw gap", "selection bias", "diff-in-diff", "spread", "separable"); for (const o of METRIC) { const r = decompose(o); write(B, `${o[0]} (${o[1]})`, r.afterGap.toFixed(2), r.beforeGap.toFixed(2), r.diffInDiff.toFixed(2), r.spread.toFixed(2), Math.abs(r.diffInDiff) > r.spread ? "yes" : "no"); } const p = METRIC.map((o) => { const r = decompose(o); return 100 * r.beforeGap / r.afterGap; }); console.log(` selection bias explains ${Math.min(...p).toFixed(0)}%-` + `${Math.max(...p).toFixed(0)}% of the raw gap`); console.log(" diff-in-diff is smaller than the spread in all four: the remaining share cannot be separated"); console.log("\n3. the platform is a dependency: how many teams does the same outage stop"); const duration = sum(OUTAGE.map((k) => k[1])); const actual = sum(OUTAGE.map((k) => k[1] * k[2])); const platformOutages = OUTAGE.filter((k) => k[3] === "platform"); console.log(` ${OUTAGE.length} outages ${duration} min; platform-caused ${platformOutages.length} outages ` + `(${sum(platformOutages.map((k) => k[1]))} min), lower layer ${OUTAGE.length - platformOutages.length} outages`); const C = [-11, 14, 18, 15, 16]; write(C, "adopters", "distributed", "central team-min", "support req.", "over capacity"); for (const n of [1, 3, 5]) write(C, n, `${duration} team-min`, n * duration, n * REQUEST_PER_TEAM, n * REQUEST_PER_TEAM > CAPACITY ? `${(100 * (n * REQUEST_PER_TEAM / CAPACITY - 1)).toFixed(0)}%` : "-"); console.log(` actual ${actual} team-min (adoption grew within the period), ` + `${(actual / duration).toFixed(1)}x the distributed figure`); console.log("\n4. what forcing adoption does to the metric"); const forced = totalPairs - pairs, extraWork = forced * COMPLY_WORK; const extraUsage = FORCED_USERS * FORCED_USAGE; const D = [-22, 14, 13, 20]; write(D, "arrangement", "adoption", "usage", "work per usage"); write(D, "voluntary", `${(100 * pairs / totalPairs).toFixed(1)}%`, USAGE, (WORK / USAGE).toFixed(2)); write(D, "forced", "100.0%", USAGE + extraUsage, ((WORK + extraWork) / (USAGE + extraUsage)).toFixed(2)); const better = (100 * (totalPairs / pairs - 1)).toFixed(0); const worse = (100 * ((WORK + extraWork) / (USAGE + extraUsage) / (WORK / USAGE) - 1)).toFixed(0); const noUsage = forced - FORCED_USERS; console.log(` of the ${forced} forced pairs, ${noUsage} produce no usage`); console.log(` adoption improves ${better}% while work per usage worsens ${worse}%`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, the band is audited console.log(`\nfour tables in the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`); ``` ``` 5 teams, 8 capabilities, 3 adopters 1. adoption: teams and usage per capability capability setup maint. teams usage usage/work environment on demand 34 7 4/5 60 0.68 scaffold generation 22 5 5/5 5 5.40 service catalog 18 6 3/5 5 4.80 policy gate 27 8 5/5 72 0.49 showback 15 4 3/5 30 0.63 migration tool 31 3 4/5 4 8.50 secret rotation 19 5 0/5 0 - shared dashboard library 24 6 1/5 3 10.00 total 190 44 25/40 179 1.31 adoption 62.5%, 3.13 teams/capability; least-used 2 capabilities 54 work-days (23%) 2. four delivery metrics: does the gap come from the platform metric raw gap selection bias diff-in-diff spread separable deployment frequency (week) 4.35 2.08 2.27 3.30 no lead time (hours) 70.67 54.67 16.00 26.00 no failure rate (%) 10.75 8.17 2.58 6.50 no time to restore (min) 159.17 110.00 49.17 80.00 no selection bias explains 48%-77% of the raw gap diff-in-diff is smaller than the spread in all four: the remaining share cannot be separated 3. the platform is a dependency: how many teams does the same outage stop 6 outages 550 min; platform-caused 4 outages (370 min), lower layer 2 outages adopters distributed central team-min support req. over capacity 1 550 team-min 550 34 - 3 550 team-min 1650 102 13% 5 550 team-min 2750 170 89% actual 1440 team-min (adoption grew within the period), 2.6x the distributed figure 4. what forcing adoption does to the metric arrangement adoption usage work per usage voluntary 62.5% 179 1.31 forced 100.0% 191 1.54 of the 15 forced pairs, 11 produce no usage adoption improves 60% while work per usage worsens 18% four tables in the 2000 ms band: yes ``` The numbers are of the **measurement** class; their inputs are the assumptions above. ## Adoption Is Three Numbers, Not One Twenty-five of forty team-capability pairs are filled (**62.5%**), coming to 3.13 teams per capability. The third column changes the table: work per usage is 0.49 work-days for the `policy gate` capability and **10.00 work-days** for the `shared dashboard library` capability — a twentyfold difference. Two capabilities go unrewarded. `secret rotation` is never used at all: 19 work-days of setup, 5 work-days of maintenance, zero usage. `shared dashboard library` serves a single team. Together they account for **54 work-days**, **23%** of total investment — the case where the platform does not pay off is readable here in a single column: the work was done, and nothing came back for it. ## Does the Gap Come From the Platform The raw gap is the difference between the two groups after adoption, and it is large across all four metrics: 4.35 deployments a week, 70.67 hours of lead time, 10.75 points of failure rate, 159.17 minutes of time to restore. The second column writes down how much of that gap **already existed before adoption**: of lead time's 70.67-hour gap, **54.67** of it; of failure rate's 10.75 points, 8.17. The confounder's name is **selection bias**, and its cause was measured in the previous lesson — the teams that migrated to the platform first were the ones with no obstacle, and they were already faster. It explains **48% to 77%** of the raw gap. The third column takes the difference between the two. The difference-in-differences also subtracts the non-adopting group's own improvement over the same period — clearing out the second confounder, the **common trend**. What is left is 2.27 for deployment frequency and 16.00 hours for lead time. The fourth column knocks down even what is left. The spread among the three adopting teams themselves is 3.30 for deployment frequency and 26.00 for lead time; it is **bigger** than the difference-in-differences on every metric, and the last column says `no` on all four: **the gap is real, but where it comes from cannot be separated out.** ## The Platform Is a Dependency The third table counts the platform's own metric: six outages and 550 minutes over twelve months, four of them platform-caused (370 minutes). The real number is not the duration — it is **team-minutes**. Under a distributed arrangement, one failure stops one team: 550 team-minutes. Under central adoption, that same 550 minutes gets multiplied by however many teams have adopted — **2,750 team-minutes** at five teams. What actually happened is 1,440, **2.6 times** the distributed figure. This is where the abstraction leaks: because a single layer is hidden, a single failure stops five teams at once. **The more successful the platform is, the more expensive one of its outages becomes.** Support requests also exceed capacity by **89%** at five teams. ## Where the Metric Points the Wrong Way The fourth table is the result of targeting the adoption rate. Once forced, the rate climbs from 62.5% to 100%; the fifteen forced pairs bring 60 work-days of compliance work, and **eleven of them produce no usage at all**. **The metric improves 60% while work per usage worsens from 1.31 to 1.54, an 18% decline** — the metric moves, the thing it was supposed to measure does not. The second wrong-way case is in the second table. Migrating the easy teams first is the right call for the migration program, but that same call produces 48% to 77% of the gap in the delivery metrics: an ordering that is efficiency in one place is bias in another. ## Summary - 25 of forty team-capability pairs are filled (62.5%); work per usage ranges from 0.49 to 10.00 work-days, and the two least-used capabilities hold 23% of the investment (54 work-days). - Across the four delivery metrics, 48%-77% of the raw gap comes from selection bias; the difference-in-differences also clears out the common trend, but stays smaller than the within-group spread on all four — where the gap comes from cannot be separated out. - The same 550-minute outage is 550 team-minutes under a distributed arrangement and 2,750 team-minutes at five teams; what actually happened is 1,440, 2.6 times the distributed figure, and support requests exceed capacity by 89%. - Forcing adoption pushes the rate to 100%, and 11 of the 15 forced pairs produce no usage at all; work per usage worsens by 18%. ## Course Wrap-Up | Lesson | Hidden layer | How many teams, how much gain | Leak or stepping outside | |---|---|---|---| | What Is Platform Engineering | 18 items, 666 work-days | threshold 7 teams; at 5, −121 days | leftover 30.6%; exemption in 9/18 | | Platform as a Product | request→need, 20→14 capabilities | 5 teams, 132 work-days | 128 days to product teams (97%) | | Golden Paths | 10 support items, three axes | 64 work-days/year; 1/5 fits | `field` loses all ten items | | Standardization and Autonomy Balance | mode choice, 80 work-days | hybrid arrangement +10 work-days | the wall blocks 10 legitimate requirements | | Self-Service | request queue, 1,551 requests | 6/12 types, 79 work-days | `region transfer` −32 work-days | | On-Demand Environment Provisioning | environment setup steps | 336 person-hours; threshold 5 teams | pool full 16/90 days; `field` outside | | Template and Scaffold Generation | 19 decisions collapse into 3 answers | 5 services, 95→15 answers | 4/5 services behind at version 2 | | Developer Portal and Service Catalog | reverse edge and ownership | 32→28.8 min; threshold 2 incidents | 6/12 records disagree | | Policy and Compliance Automation | the text of 6 rules | 5 teams, 830→152 min | 4 exemptions, lifespan 3.2x | | Cost Visibility | the bill and the accounting pipeline | 85 u/day; threshold 3 teams | 408 u/day outside audit | | Migration Strategies | 10 of migration's 14 steps | 4 teams at wave=1, 40 steps | 1–3 teams behind, legacy arrangement indefinite | | Platform Metrics | the measurement pipeline itself | 3 teams, 25/40 pairs (62.5%) | 1,440 team-min of outage | In none of the twelve rows is the third column ever without the fourth. In every lesson, the gain was tied to a number, and a cost stood right next to it: **the abstraction does not remove the layer it hides** — it multiplies its gain by the number of teams, and makes someone pay its cost through leaking or stepping outside. A threshold showed up explicitly in three lessons (7, 5, and 3 teams), and in all three, staying under it made the platform harmful. | Course | Measurement axis | Unit of measure | |---|---|---| | K01 DevOps Culture and Fundamentals | where the gap is hidden | gap item | | K02 Containers | isolation is a budget | byte, layer, resource | | K03 Container Orchestration | the cost of delegating a decision | information item | | K04 Continuous Integration and Delivery | what each gate catches, what it misses | marginal gate value | | K05 Infrastructure as Code | the gap between record and reality | record item | | K06 Observability and Operations | not data, a question | cost per question | | K07 Servers, Networking and Hardening | the default itself | open-surface item | | K08 Platform Engineering | the cost of abstraction | hidden layer | Eight courses told eight different stories, but measured in a single form. The gap item, the open-surface item, the hidden layer — all of them were asking the same three things: **a decision, a number, and the cost paid in exchange for that number.** In K02, the cost of isolation was measured in bytes; in K04, a gate's value was measured by the violations it caught; in K08, the value of abstraction was measured by the number of teams it saved. In none of them did "it is good" carry a decision on its own. Measuring itself was not free either: K06 counted the measuring side as its own line item on the bill; K08 showed that 23% of the measurement investment went unrewarded. **Not writing down the unmeasurable as if it were measured is itself a measurement decision** — and that is this curriculum's last line.