Leak Response

The first four lessons of the topic measured the number, the life, and the change cost of secrets, and left the same question unanswered: what happens when a secret leaks. The exposure window was the time it takes for the secret to become invalid on its own; if the leak is detected and revoked, the window closes earlier. This lesson measures that “earlier.”

What is measured is the revocation flow, and the flow is on the defense side. Nothing is written about what can be done with a leaked secret or how such a secret would be found; the only things written are the minutes from the detection of the leak to the invalidation of the old value and the call volume that passes through the secret in those minutes. The measurement runs on the six static secrets left over from the previous lesson.

• SM23. The model of the previous four lessons continues. The environment is fiction, what is measured is a model written with node, and it carries no real secret value; the model holds only secret paths and flow durations.
• SM24. The leak channel of each secret is chosen in the model and determines the detection path: a leak that lands in the code repository is detected by continuous scanning, one that lands in a log file by usage anomaly, one at a third party by provider notice, and one in a client bundle by external report.
• SM25. The flow has five stages: detection, confirmation and impact assessment, finding the owner, generating the new value, propagation of the new value. The propagation time is the minimum overlap of the previous lesson.
• SM26. “Calls passing through the secret” is the total call volume that the secret authorized during the exposure window; it is an upper bound that shows the scale of the window, not the volume that was abused.
• SM27. There is no randomness; all numbers are derived from the model tables and the output is the same from run to run.

How Many Minutes from Detection to Revocation

// revocation.mjs — minutes from the leak to revocation and calls passing through the secret in that time. A MODEL.
// The environment is FICTION; THERE IS NO REAL SECRET VALUE, only secret paths and flow durations are modeled.
// How a secret would be used is not written; what is measured is the revocation flow on the defense side.

// Six static secrets left over from the previous lesson. channel: where the leak lands (chosen in the model).
// owner: whether the owner is written in the secret record. external: whether an external provider generates the new value.
// calls: daily calls the secret authorizes. spread: minimum overlap of the previous lesson (min).
export const SECRETS = {
  "infra/cache": { channel: "repository", owner: true, external: false, calls: 240000, spread: 60 },
  "signing/session-signing": { channel: "log", owner: true, external: false, calls: 180000,
    spread: 960 },
  "signing/document-signing": { channel: "repository", owner: true, external: false, calls: 6200,
    spread: 240 },
  "external/payment-gateway": { channel: "third-party", owner: true, external: true, calls: 3400,
    spread: 240 },
  "external/email-gateway": { channel: "client-bundle", owner: false, external: true, calls: 13400,
    spread: 1440 },
  "external/sms-gateway": { channel: "log", owner: false, external: true, calls: 8900,
    spread: 960 },
};
// the detection path depends on the channel: only a leak that lands in the code repository is caught by continuous scanning.
export const DETECTION = { repository: ["continuous scanning", 20], log: ["usage anomaly", 90],
  "third-party": ["provider notice", 240], "client-bundle": ["external report", 2880] };
export const CONFIRM = 30;                     // confirmation and impact assessment (min)
export const OWNED = 10, UNOWNED = 180;        // finding the owner (min)
export const PRODUCTION = 45, EXTERNAL_PRODUCTION = 120;  // generating the new value (min)
export const NAMES = Object.keys(SECRETS);
export const stages = (s, o = {}) => {
  const k = SECRETS[s];
  return { detection: o.scan && k.channel === "repository" ? 20 : DETECTION[k.channel][1],
    confirmation: CONFIRM,
    owner: k.owner || o.record ? OWNED : UNOWNED,
    production: o.staged ? 5 : k.external ? EXTERNAL_PRODUCTION : PRODUCTION,
    spread: o.noWait ? 0 : k.spread };
};
export const total = (s, o) => Object.values(stages(s, o)).reduce((t, x) => t + x, 0);
export const callsUntil = (s, o) => Math.round(SECRETS[s].calls * total(s, o) / 1440);
export const gen = (B) => (h) => h.map((v, j) => j ? String(v).padStart(B[j])
  : String(v).padEnd(B[0])).join("");

if (import.meta.url === `file://${process.argv[1]}`) {
  const row = gen([18, 22, 8, 9, 8, 10, 11, 8]);
  console.log(`revocation flow — ${NAMES.length} static secrets (fictional production environment); durations in minutes\n`);
  console.log(row(["secret", "detection path", "detect", "confirm", "owner", "generate",
    "propagate", "total"]));
  for (const s of NAMES) {
    const a = stages(s);
    console.log(row([s.split("/")[1], DETECTION[SECRETS[s].channel][0], a.detection,
      a.confirmation, a.owner, a.production, a.spread, total(s)]));
  }
  const T = NAMES.reduce((t, s) => t + total(s), 0), C = NAMES.reduce((t, s) => t + callsUntil(s), 0);
  console.log(row(["total", "", "", "", "", "", "", T]));

  console.log(`\n${gen([18, 12, 12, 14, 16])(["secret", "total min", "= hours",
    "daily calls", "until revoked"])}`);
  for (const s of [...NAMES].sort((x, y) => callsUntil(y) - callsUntil(x)))
    console.log(gen([18, 12, 12, 14, 16])([s.split("/")[1], total(s),
      (total(s) / 60).toFixed(1), SECRETS[s].calls, callsUntil(s)]));
  console.log(gen([18, 12, 12, 14, 16])(["total", T, (T / 60).toFixed(1), "", C]));
  console.log(`\naverage per secret ${Math.round(T / NAMES.length)} min ` +
    `(${(T / NAMES.length / 60).toFixed(1)} hours); longest ${Math.max(...NAMES.map((s) => total(s)))} min`);
  console.log(`if the same leak were in a short-lived credential the revocation flow would never run, ` +
    `the window would stay at 60 min: ${Math.round(Math.max(...NAMES.map((s) => total(s))) / 60)} times`);
}

revocation flow — 6 static secrets (fictional production environment); durations in minutes

secret                    detection path  detect  confirm   owner  generate  propagate   total
cache                continuous scanning      20       30      10        45         60     165
session-signing            usage anomaly      90       30      10        45        960    1135
document-signing     continuous scanning      20       30      10        45        240     345
payment-gateway          provider notice     240       30      10       120        240     640
email-gateway            external report    2880       30     180       120       1440    4650
sms-gateway                usage anomaly      90       30     180       120        960    1380
total                                                                                     8315

secret               total min     = hours   daily calls   until revoked
session-signing           1135        18.9        180000          141875
email-gateway             4650        77.5         13400           43271
cache                      165         2.8        240000           27500
sms-gateway               1380        23.0          8900            8529
payment-gateway            640        10.7          3400            1511
document-signing           345         5.8          6200            1485
total                     8315       138.6                        224171

average per secret 1386 min (23.1 hours); longest 4650 min
if the same leak were in a short-lived credential the revocation flow would never run, the window would stay at 60 min: 78 times

The upper table shows where the flow is spent. The average per secret is one thousand three hundred eighty-six minutes, that is twenty-three hours; but between the rows, from one hundred sixty-five to four thousand six hundred fifty minutes, there is a twenty-eight-fold difference.

Looking at the columns, the whole of the difference comes from two stages. Confirmation is thirty minutes in every row, finding the owner is ten or one hundred eighty, generating is forty-five or one hundred twenty. The detection column, though, goes from twenty to two thousand eight hundred eighty — a one-hundred-forty-four-fold range — and the propagation column from sixty to one thousand four hundred forty. What determines the length of the flow is not the speed of the response team but how the leak is detected and how slowly the new value propagates. Both are fixed before the leak.

The lower table converts the same rows into call volume and breaks the ordering. In duration, the worst item is the email gateway key, at four thousand six hundred fifty minutes; but the most calls pass through the session signing key by the time of revocation: one hundred forty-one thousand eight hundred seventy-five. That key authorizes one hundred eighty thousand calls a day, and one thousand one hundred thirty-five minutes is a long time at that volume. At the opposite end is the document signing key: in three hundred forty-five minutes only one thousand four hundred eighty-five calls. The same flow, the same procedure — and the result differs ninety-six-fold between two items, because the traffic behind them differs.

The last line ties back to the second lesson of this topic. If the same leak were in a short-lived credential, no step of this flow would run: it would not need to be detected, its owner would not need to be found, a new one would not need to be generated; the token would become invalid on its own in sixty minutes. The difference from the worst item is seventy-eight-fold. The revocation flow is the price paid for secrets whose life cannot be shortened.

What Shortening Which Stage Gains

// speedup.mjs — what shortening which stage of the flow gains, and what it cuts in return
import { NAMES, SECRETS, DETECTION, stages, total, callsUntil, gen } from "./revocation.mjs";

// if the old value is revoked without waiting for the switch to the new one, readers are cut for the switch duration.
// [app, switch (min), daily requests] — from the switch table of the previous lesson.
const READS = {
  "infra/cache": [["portal-api", 60, 96000]],
  "signing/session-signing": [["portal-frontend", 960, 84000], ["portal-api", 60, 96000]],
  "signing/document-signing": [["billing-service", 240, 6200]],
  "external/payment-gateway": [["billing-service", 240, 6200], ["payment-bridge", 0, 3400]],
  "external/email-gateway": [["subscriber-signup", 960, 1400], ["notification", 15, 12000],
    ["report-generator", 1440, 320]],
  "external/sms-gateway": [["subscriber-signup", 960, 1400], ["notification", 15, 12000]],
};
const cut = (o) => o.noWait ? Math.round(NAMES.reduce((t, s) => t + READS[s]
  .reduce((x, [, g, i]) => x + i * g / 1440, 0), 0)) : 0;

const SETTINGS = [["today", {}, "-"], ["+ ownership record", { record: 1 }, "deployment"],
  ["+ pre-staged replacement", { record: 1, staged: 1 }, "deployment"],
  ["+ continuous repository scanning", { record: 1, staged: 1, scan: 1 }, "build"],
  ["+ revoke without waiting for propagation", { record: 1, staged: 1, scan: 1, noWait: 1 },
    "runtime"]];
const row = gen([43, 11, 11, 15, 16]);
console.log(row(["measure", "total min", "gain", "calls passed", "cut requests"]));
const T0 = NAMES.reduce((t, s) => t + total(s), 0);
for (const [name, o] of SETTINGS) {
  const T = NAMES.reduce((t, s) => t + total(s, o), 0);
  console.log(row([name, T, T0 - T, NAMES.reduce((t, s) => t + callsUntil(s, o), 0), cut(o)]));
}
const last = SETTINGS[SETTINGS.length - 1][1], prev = SETTINGS[SETTINGS.length - 2][1];
const dC = NAMES.reduce((t, s) => t + callsUntil(s, prev) - callsUntil(s, last), 0);
console.log(`\nrevoke without waiting for propagation: saves ${dC} calls, cuts ${cut(last)} legitimate requests ` +
  `(${(dC / cut(last)).toFixed(1)} calls saved per 1 request cut)`);
console.log(`continuous repository scanning gains 0: two secrets already have the repository as their channel and detection is 20 min, ` +
  `the leak of the other four never lands in a repository`);

const row2 = gen([18, 16, 9, 11, 34]);
console.log(`\n${row2(["secret", "leak channel", "detect", "calls", "earlier closure"])}`);
const EARLIER = { repository: ["build", "pipeline secret scanning"],
  log: ["code", "masking in the log"], "client-bundle": ["build", "bundle secret scanning"],
  "third-party": ["none", "-"] };
for (const s of NAMES)
  console.log(row2([s.split("/")[1], SECRETS[s].channel, stages(s).detection, callsUntil(s),
    `${EARLIER[SECRETS[s].channel][0]}: ${EARLIER[SECRETS[s].channel][1]}`]));
const closable = NAMES.filter((s) => EARLIER[SECRETS[s].channel][0] !== "none");
console.log(`\nsecrets whose leak channel can be closed earlier: ${closable.length} / ${NAMES.length}; ` +
  `the detection latency of these channels totals ` +
  `${closable.reduce((t, s) => t + stages(s).detection, 0)} min`);
const top = NAMES.reduce((x, s) => stages(s).detection > stages(x).detection ? s : x);
console.log(`largest single share: ${top} — ${stages(top).detection} min detection latency and ` +
  `${Math.round(SECRETS[top].calls * stages(top).detection / 1440)} calls, all of it from the build stage`);

measure                                      total min       gain   calls passed    cut requests
today                                             8315          0         224171               0
+ ownership record                                7975        340         221538               0
+ pre-staged replacement                          7510        805         207648               0
+ continuous repository scanning                  7510        805         207648               0
+ revoke without waiting for propagation          3610       4705          56714           68503

revoke without waiting for propagation: saves 150934 calls, cuts 68503 legitimate requests (2.2 calls saved per 1 request cut)
continuous repository scanning gains 0: two secrets already have the repository as their channel and detection is 20 min, the leak of the other four never lands in a repository

secret                leak channel   detect      calls                   earlier closure
cache                   repository       20      27500   build: pipeline secret scanning
session-signing                log       90     141875          code: masking in the log
document-signing        repository       20       1485   build: pipeline secret scanning
payment-gateway        third-party      240       1511                           none: -
email-gateway        client-bundle     2880      43271     build: bundle secret scanning
sms-gateway                    log       90       8529          code: masking in the log

secrets whose leak channel can be closed earlier: 5 / 6; the detection latency of these channels totals 3100 min
largest single share: external/email-gateway — 2880 min detection latency and 26800 calls, all of it from the build stage

The first three rows give the counterpart of the two items written into the response procedure. Writing the owner into every secret record gains three hundred forty minutes; preparing and keeping ready a replacement value in advance for every secret gains four hundred sixty-five minutes more. Together they bring the eight thousand three hundred fifteen minute flow down to seven thousand five hundred ten — ten percent. Both are cheap and are written before a leak; but the total gain is a tenth of the flow.

The fourth row is the kind of result this course looks for: a measure whose gain is zero. Continuous repository scanning does not shorten the response time by even a minute. The reason is written in the lower table: of the six leaks, only two have the code repository as their channel, and in those two detection is already twenty minutes. The other four never land in a repository. Increasing the scanning power does not make visible a leak in places the scan never looks at; every improvement made without counting the set of items a defense covers produces this row.

The fifth row contains the real decision. Revoking the old value without waiting for the new value to propagate brings the flow down from seven thousand five hundred ten to three thousand six hundred ten, and cuts the calls that pass from two hundred seven thousand to fifty-six thousand — one hundred fifty thousand nine hundred thirty-four calls are saved. In return, sixty-eight thousand five hundred three legitimate requests are cut: one legitimate request for every two point two calls saved. This is the real decision of leak response and it cannot be settled by a procedure item, because the right answer changes with the secret: for the payment gateway key, not waiting for propagation may be reasonable; for the session signing key, it means cutting sixty thousand requests.

The last column of the lower table is the axis of the course. Five of the six leak channels can be closed at an earlier stage, and the detection latency of those five channels totals three thousand one hundred minutes. The largest single share is in the email gateway key: because it is embedded in the client bundle of the field app, the leak is noticed only by external report, two days later, and in those two days twenty-six thousand eight hundred calls pass through the key. There is nothing to be done on the response side for this item; the key entered the bundle at the build stage, and with a secret scan there it would never have entered. The same holds for the two keys that land in the log file, one stage earlier: masking in log writing is a code decision.

The only item left is the payment gateway key. The leak happened at a third party; neither a build scan nor code masking would have seen it. For this item the revocation flow really is the only way — one of six items.

Summary

• The revocation flow has five stages: detection, confirmation, finding the owner, generating the new value, propagation. Across the remaining six static secrets the total is 8,315 minutes, the average per secret is 1,386 minutes (23.1 hours), and the worst item takes 4,650 minutes.
• What determines the duration is not the response speed but the detection and propagation columns: confirmation is 30 minutes in every row, while detection varies between 20 and 2,880 and propagation between 60 and 1,440.
• Until revocation, 224,171 calls pass through the secrets, and the ordering is not the same as the duration ordering: the longest flow is the email gateway (4,650 min) but the most calls pass through session signing (141,875).
• An ownership record gains 340 minutes and a pre-staged replacement 465 — ten percent of the total. The gain of continuous repository scanning is zero: only two of the six leaks land in a repository and in those two detection is already 20 minutes.
• Revoking without waiting for propagation saves 150,934 calls and cuts 68,503 legitimate requests: one cut request for every 2.2 calls saved. The right answer changes with the secret and cannot be written into a procedure.
• Five of the six leak channels could have been closed at an earlier stage (3,100 minutes of detection latency in total); the largest single share is the key embedded in the client bundle — 2,880 minutes and 26,800 calls, all of it from the build stage.

Next Step

This topic measured the life of secrets and shortened it: the access matrix went from seventy-nine items to thirty-one, the exposure window shrank two-thousand-fold on sixteen items, twenty items disappeared entirely with workload identity, and the revocation flow was written minute by minute for the remaining six secrets. Secrets now have an owner, a life, a rotation schedule, and a leak procedure.

Under all of this there is an assumption, and it has not been touched at all. Everything measured in this topic was about a service proving itself: a call that carries the right credential passes, one that does not carry it does not. But what happens when a call does carry the credential. When the field app API calls the billing service, the billing service never asks whether that call is authorized to read billing data; it sees that the call came from somewhere and fulfills it. Even if the secret is managed correctly, if this stays so, the capture of a single service is a door that opens onto the others: every trust relation among the nine services is a path along which one can move from one service to another. The next topic counts those paths — how many service pairs have no authentication, how many of them mutual authentication closes, and at which stage the ones that remain open could have been closed.