Adoption Measurement

The previous lesson’s model treated “local copy” as an assumption: it assumed that a certain portion of the work waiting in the queue would leave the system. In a real organization, this number is not assumed, it is measured.

Adoption measurement’s purpose is not to produce a success report. Finding where the system is not used tells you what it is missing; every local copy points either to a gap in the catalog, a findability problem, or a clogged contribution path.

What Gets Measured

The weakest question you can ask about adoption is “are teams using the system”; the answer is always “yes.” The measurable question is: what percentage of the component calls in the interface come from the system?

This ratio is coverage percentage, and it can be measured two different ways:

• Static scan. Imports and component calls in the source text are counted. Every call site counts as one unit; a call in a rarely opened admin screen carries the same weight as a call on the home page.
• Runtime telemetry. The component instances actually created are counted and weighted by how often the screen is viewed.

The two answer different questions, and both are needed. The static scan measures maintenance load: it tells you how many places would need code changed. The traffic-weighted measurement measures user impact: it tells you how much of the visible interface the system is producing.

Running the Scan

The program below scans seven files across three product teams. It resolves each file’s imports, counts component calls, classifies each call by whether it comes from the system package or a local file, and computes the two coverage percentages together. It then classifies the local components against the catalog.

// adoption.mjs — coverage percentage via repository scan, deviation list, catalog candidates

// Source files. The path's first segment gives the team. The number is that screen's
// views per period; runtime weighting comes from here. View descriptions are in the
// form el(component, props, ...children).
const REPO = [
  { path: "product-catalog/search-page.js", views: 48200, source: `
import { SearchField, Card, Badge, Button, Pagination, EmptyState } from "@system/components";
import { FacetPanel } from "../components/facet-panel";
export const SearchPage = () => el("div", {},
  el(SearchField), el(FacetPanel),
  el(Card, {}, el(Badge), el(Button)),
  el(Card, {}, el(Badge), el(Button)),
  el(EmptyState), el(Pagination));` },
  { path: "product-catalog/record-page.js", views: 31400, source: `
import { Card, Badge, Button, Breadcrumb, Tabs } from "@system/components";
import { RecordCard } from "../components/record-card";
export const RecordPage = () => el("div", {},
  el(Breadcrumb), el(RecordCard), el(Tabs),
  el(Card, {}, el(Badge), el(Button), el(Button)));` },
  { path: "product-catalog/borrow-page.js", views: 9800, source: `
import { Button, NotificationBanner, Skeleton } from "@system/components";
import { CustomButton } from "../components/custom-button";
import { LoadingBar } from "../components/loading-bar";
export const BorrowPage = () => el("div", {},
  el(NotificationBanner), el(Skeleton),
  el(CustomButton), el(CustomButton), el(Button), el(LoadingBar));` },
  { path: "product-membership/member-login.js", views: 22600, source: `
import { TextField, Button, Link, NotificationBanner } from "@system/components";
export const MemberLogin = () => el("form", {},
  el(TextField), el(TextField), el(Button), el(Link), el(NotificationBanner));` },
  { path: "product-membership/member-profile.js", views: 7300, source: `
import { Card, TextField, Checkbox, Button } from "@system/components";
import { StatusBadge } from "../components/status-badge";
import { CustomButton } from "../components/custom-button";
export const MemberProfile = () => el("div", {},
  el(Card, {}, el(StatusBadge), el(TextField), el(Checkbox)),
  el(CustomButton), el(Button));` },
  { path: "product-admin/shelf-management.js", views: 1900, source: `
import { Button, TextField, Modal } from "@system/components";
import { DataTable } from "../components/data-table";
import { FacetPanel } from "../components/facet-panel";
import { SimpleDropdown } from "../components/simple-dropdown";
export const ShelfManagement = () => el("div", {},
  el(FacetPanel), el(SimpleDropdown), el(DataTable),
  el(Modal, {}, el(TextField), el(Button)));` },
  { path: "product-admin/user-management.js", views: 1200, source: `
import { Button, TextField } from "@system/components";
import { DataTable } from "../components/data-table";
import { CustomModal } from "../components/custom-modal";
import { StatusBadge } from "../components/status-badge";
export const UserManagement = () => el("div", {},
  el(DataTable), el(StatusBadge), el(StatusBadge),
  el(CustomModal, {}, el(TextField), el(Button)));` },
];

const SYSTEM_PACKAGE = "@system/components";
// Catalog components' code-side names; local-copy detection looks these up.
const CATALOG = ["Button", "Link", "TextField", "Checkbox", "Badge", "Card", "Pagination",
  "SearchField", "EmptyState", "Modal", "Skeleton", "Tabs", "Dropdown", "NotificationBanner", "Breadcrumb"];

// Resolve a file's imports: name -> source.
function imports(source) {
  const map = new Map();
  for (const m of source.matchAll(/import\s*\{([^}]+)\}\s*from\s*"([^"]+)"/g)) {
    for (const name of m[1].split(",").map((s) => s.trim()).filter(Boolean)) map.set(name, m[2]);
  }
  return map;
}
// Usage count: an el() call is a component if its first argument starts with a capital letter.
function usages(source) {
  const count = new Map();
  for (const m of source.matchAll(/\bel\(\s*([A-Z][A-Za-z0-9]*)/g)) count.set(m[1], (count.get(m[1]) ?? 0) + 1);
  return count;
}

const teams = new Map();
const localTotal = new Map();
for (const file of REPO) {
  const team = file.path.split("/")[0];
  const imp = imports(file.source);
  const use = usages(file.source);
  const e = teams.get(team) ?? { system: 0, local: 0, weightedSystem: 0, weightedLocal: 0 };
  for (const [name, count] of use) {
    const source = imp.get(name);
    if (!source) continue;
    const isSystem = source === SYSTEM_PACKAGE;
    e[isSystem ? "system" : "local"] += count;
    e[isSystem ? "weightedSystem" : "weightedLocal"] += count * file.views;
    if (!isSystem) {
      const l = localTotal.get(name) ?? { count: 0, teams: new Set() };
      l.count += count;
      l.teams.add(team);
      localTotal.set(name, l);
    }
  }
  teams.set(team, e);
}

console.log("team                 system   local   coverage   traffic-weighted coverage");
let totalSystem = 0, totalLocal = 0, wSystem = 0, wLocal = 0;
for (const [team, e] of teams) {
  totalSystem += e.system; totalLocal += e.local;
  wSystem += e.weightedSystem; wLocal += e.weightedLocal;
  const coverage = (e.system / (e.system + e.local)) * 100;
  const weighted = (e.weightedSystem / (e.weightedSystem + e.weightedLocal)) * 100;
  console.log(
    `${team.padEnd(20)} ${String(e.system).padStart(6)} ${String(e.local).padStart(7)} ` +
    `${(coverage.toFixed(1) + "%").padStart(10)} ${(weighted.toFixed(1) + "%").padStart(26)}`
  );
}
console.log(
  `${"TOTAL".padEnd(20)} ${String(totalSystem).padStart(6)} ${String(totalLocal).padStart(7)} ` +
  `${(((totalSystem / (totalSystem + totalLocal)) * 100).toFixed(1) + "%").padStart(10)} ` +
  `${(((wSystem / (wSystem + wLocal)) * 100).toFixed(1) + "%").padStart(25)}`
);

// Deviation list: does the local component have a catalog counterpart?
const findMatch = (localName) => CATALOG.find((k) => localName.includes(k)) ?? null;
console.log("\ndeviation list");
console.log("local component      usage   teams   catalog match        class");
const candidates = [];
for (const [name, l] of [...localTotal].sort((a, b) => b[1].count - a[1].count)) {
  const match = findMatch(name);
  const cls = match ? "local copy" : (l.teams.size >= 2 ? "catalog candidate" : "product-specific");
  if (cls === "catalog candidate") candidates.push(name);
  console.log(
    `${name.padEnd(21)} ${String(l.count).padStart(6)} ${String(l.teams.size).padStart(7)}   ` +
    `${(match ?? "-").padEnd(20)} ${cls}`
  );
}

const copyCount = [...localTotal].filter(([name]) => findMatch(name)).reduce((t, [, l]) => t + l.count, 0);
console.log(`\nlocal-copy usage             : ${copyCount} (could move into the catalog)`);
console.log(`local components that are catalog candidates: ${JSON.stringify(candidates)}`);
console.log(`coverage if copies moved into the catalog: ${(((totalSystem + copyCount) / (totalSystem + totalLocal)) * 100).toFixed(1)}%`);

team                 system   local   coverage   traffic-weighted coverage
product-catalog          18       5      78.3%                      85.7%
product-membership        9       2      81.8%                      90.7%
product-admin             5       7      41.7%                      43.5%
TOTAL                    32      14      69.6%                     85.7%

deviation list
local component      usage   teams   catalog match        class
CustomButton               3       2   Button               local copy
StatusBadge                3       2   Badge                local copy
FacetPanel                 2       2   -                    catalog candidate
DataTable                  2       1   -                    product-specific
RecordCard                 1       1   Card                 local copy
LoadingBar                 1       1   -                    product-specific
SimpleDropdown             1       1   Dropdown             local copy
CustomModal                1       1   Modal                local copy

local-copy usage             : 9 (could move into the catalog)
local components that are catalog candidates: ["FacetPanel"]
coverage if copies moved into the catalog: 89.1%

Telling the Two Coverage Numbers Apart

The total row shows a 16.1-point gap between the two numbers: static coverage is 69.6%, traffic-weighted coverage is 85.7%. The gap comes from a single team — product-admin‘s coverage is 41.7%, the lowest on the list, but its screens’ view count is small next to the others’, so it nearly disappears in the weighted calculation.

Which number to use depends on the question. When it is how much of what the user sees comes from the system, the weighted number is correct. When it is the system’s maintenance load and how many places a breaking change would touch, the static number is correct; the versioning lesson’s migration-cost calculation also rests on this second number.

Reporting only the weighted number hides low-traffic products dropping out of the system entirely: product-admin sits at 41.7%, and if its situation does not improve, the system will never be used in this product at all. A governance decision that only looks at the weighted number never sees this team.

Classifying the Deviation List

The deviation list splits local components into three classes, and each class has a different job.

Local copy — a component with a catalog counterpart. CustomButton and StatusBadge are each used by two separate teams; both were written in place of a component already in the system. These rows raise a question: why was the copy written? Usually one of three reasons — the system component lacked a needed feature, the escape hatch was insufficient, or the contribution path was stuck in the queue seen in the previous lesson. Remove the copy without finding the answer and the same copy returns under another name.

Catalog candidate — a component with no counterpart that is used by two separate teams. FacetPanel shows up in this class. The first lesson’s scope criterion was looking for exactly this situation: a part duplicated across multiple teams is a candidate for entering the catalog.

Product-specific — a component with no counterpart, used by a single team. DataTable and LoadingBar fall in this class. These are not deviations; they are parts of the product itself, and bringing them into the catalog would only add review overhead.

The last line gives the coverage impact of closing the deviation: if the nine local copies moved into the system, coverage would rise from 69.6% to 89.1%. This number is not a target; it is a ceiling: it shows the best-case return on closing the copies and lets that work be compared against other work.

The Limit of Heuristic Matching

The report is the output of a program, and the matching rule the program uses is heuristic: if a catalog name appears inside a local name, it is counted as a copy. This rule is wrong in two places, and both are instructive.

The RecordCard row is flagged as a copy of the Card component. But as seen in the first lesson, a record card is a product-specific part that decides which fields of a bibliographic record are shown in what order — its name containing “Card” does not show it is a copy of the card component; it more likely just uses card. What the heuristic matcher produces is not a conclusion but a review queue.

The FacetPanel row’s error is more interesting: the program flags it as a catalog candidate, since nothing named “FacetPanel” exists in the catalog. But the first lesson’s catalog table has an entry called filter-panel — used by a single team in only four places, so it never met the scope criterion. Placed side by side, the real problem becomes visible: an unused filter panel sits in the catalog while two teams have each written their own facet panel.

This is not a scope problem but a findability problem, and its source is the synonym issue defined in the naming lesson. The fix is not adding a new component but writing the existing component’s synonymous names into the dictionary and feeding the search surface from it. Measurement reaches this conclusion only by reading two reports together; no single number says it alone.

The Cost of Measurement Itself

Adoption measurement is an ongoing job, and it carries two traps.

The first is turning coverage percentage into a target. If teams are judged by this number, product-specific components get forced into the catalog and the catalog loses the scope criterion from the first lesson. Coverage percentage is a diagnostic tool; the target is closing the causes of the deviation.

The second is the scan undercounting. The program above counts imports and calls; a component that wraps a system component and re-exports it, or a screen that uses a system component but overrides its styling entirely, escapes this count. Both are deviations and need separate checks: a list of wrapping components and a count of style rules written onto system components from outside.

Summary

• Adoption is not measured by “is the system used” but by what percentage of the interface’s component calls come from the system.
• Static scan gives maintenance load, traffic-weighted measurement gives user impact; reporting only the weighted number hides low-traffic products dropping out of the system.
• The deviation list splits into three classes: local copy points to a gap to close, catalog candidate to a new component, product-specific to no action at all. If a local copy’s cause is not found before removing it, the same copy returns under another name.
• Matching based on name similarity is heuristic and gets it wrong; the list it produces is not a conclusion but a review queue.
• When an unused component sitting in the catalog and the local counterpart teams wrote are seen side by side, the problem is not scope but findability; the fix is recording synonymous names.
• If coverage percentage is turned into a target, the catalog loses its scope criterion; measurement is a diagnostic tool.

Next Step

This lesson’s scan stayed on the code side: it checked whether the system component was used, but not whether the values it carries match the design library’s. A team may be using the system button while the button’s fill color differs from the one in the design file; coverage percentage counts this as one hundred percent. The next lesson builds the toolchain: it produces a diff report comparing the token export coming out of the design tool against the token definition in code, normalizes names and color formats to eliminate false diffs, and classifies the real deviation that remains.