Estimation and Planning

In the previous two lessons, the modules each work item would touch were known from the start. Handoffs, waiting, and rework were computed from that information. In reality this information does not exist when an item begins: how long the item will take is estimated, and the estimate changes as the work proceeds.

The impact of architectural decisions was estimated before; here what is estimated is the work item’s duration, and its unit is the round. The lesson compares a set of estimates against the durations that actually occurred and produces four numbers: the margin of error, the direction of the error, the stage-by-stage narrowing of the uncertainty cone, and how many items the plan built on the estimate held for. Then it is counted why a single-item estimate and a total estimate behave differently. The through-line is the regional library network and it is fictional.

Estimated Versus Actual

TD15 — the previous two lessons’ stream-aligned arrangement and twelve work items are used unchanged. TD16 — an item’s actual duration comes from the previous lessons’ flow model: work rounds plus wait rounds plus the rounds rework adds. Rationale: the “actual” the estimate will be compared against has to come from somewhere; here that somewhere is the model already run and printed earlier, so the actual duration is not independent data for this lesson but the output of that same model.

TD17 — the deviation is drawn from a generator; the generator is a linear map, and its seed is visibly 20250401. Rationale: estimation error has a random component, and the run has to be reproducible.

TD18 — there are three estimation stages. In stage 1 only the item’s title is known, and the number of modules it will touch is estimated with ±80% deviation. In stage 2 the item has been reviewed, the touched modules are known exactly, and the deviation is ±30%. In stage 3 the first team has finished its work; the work rounds and how many teams the item will spread across are known, one round of waiting is added per handoff, and the deviation is ±10%. Rationale: the stages represent the three main waypoints where uncertainty naturally decreases; at each stage what the estimator knows is written down separately, because the cone’s narrowing comes from an increase in knowledge.

// estimate.mjs — three-stage estimation, the same items compared against actual flow time (model)
import { writeFileSync } from "node:fs";

// TD15 — the previous two lessons' stream-aligned arrangement and twelve items are used unchanged
const TEAMS = {
  experience: "branchFront staffFront mobileAccess",
  loan: "loanFlow loanRule reservationFlow storeAccess",
  fee: "feeFlow feeRule",
  membership: "membershipFlow membershipRule identityBridge",
  catalog: "catalogRule catalogBridge",
  platform: "sharedFormat eventLog notificationQueue",
};
const CHANNELS = "experience-loan experience-fee experience-membership loan-fee loan-catalog " +
  "loan-platform fee-platform membership-platform catalog-platform";
const ITEMS = [
  ["late fee rate", "feeFlow feeRule branchFront sharedFormat"],
  ["reservation cancellation", "mobileAccess reservationFlow loanRule notificationQueue"],
  ["membership reminder", "membershipFlow membershipRule notificationQueue staffFront"],
  ["catalog record field", "catalogBridge catalogRule sharedFormat mobileAccess"],
  ["loan period extension", "loanFlow loanRule branchFront"],
  ["second identity step", "identityBridge membershipFlow staffFront sharedFormat"],
  ["fee refund record", "feeFlow storeAccess eventLog"],
  ["branch delay report", "staffFront storeAccess loanFlow"],
  ["notification text format", "notificationQueue sharedFormat"],
  ["reservation queue", "mobileAccess reservationFlow branchFront"],
  ["loan rule exception", "loanRule feeRule loanFlow feeFlow"],
  ["event log field", "eventLog sharedFormat storeAccess"],
];

const owner = {}, T = Object.keys(TEAMS), adj = Object.fromEntries(T.map((e) => [e, []]));
for (const [e, ms] of Object.entries(TEAMS)) for (const m of ms.split(" ")) owner[m] = e;
for (const k of CHANNELS.split(" ")) {
  const [x, y] = k.split("-");
  adj[x].push(y); adj[y].push(x);
}
const d = {};
for (const s of T) {
  d[s] = Object.fromEntries(T.map((e) => [e, -1]));
  d[s][s] = 0;
  for (let q = [s]; q.length; ) {
    const u = q.shift();
    for (const v of adj[u]) if (d[s][v] < 0) { d[s][v] = d[s][u] + 1; q.push(v); }
  }
}
const contributors = {};
for (const [, ms] of ITEMS) for (const m of ms.split(" "))
  (contributors[m] ??= new Set([owner[m]])).add(owner[ms.split(" ")[0]]);

// TD16 — actual duration comes from the previous lessons' flow model: work rounds + wait rounds + rework rounds
const WAIT = [0, 1, 3, 5], NONE = 8;
const ACTUAL = ITEMS.map(([name, mods]) => {
  const ms = mods.split(" "), order = [...new Set(ms.map((m) => owner[m]))];
  let wait = 0, longest = 0, broken = 0;
  for (let j = 1; j < order.length; j++) {
    const u = d[order[j - 1]][order[j]], b = u < 0 ? NONE : WAIT[u];
    wait += b; longest = Math.max(longest, b);
    if (u < 0 || u >= 2) broken += 1;
  }
  const reasons = [];
  if (broken) reasons.push("late");
  if (ms.some((m) => contributors[m].size >= 3)) reasons.push("bound");
  if (order.length >= 4) reasons.push("info");
  const work = ms.length + reasons.length * 2, waitT = wait + reasons.length * longest;
  return { name, modules: ms.length, team: order.length, work, wait: waitT,
    rework: reasons.length, actual: work + waitT };
});

// TD17 — deviation is drawn from a generator; the generator is a linear map, its seed is visibly 20250401
const SEED = 20250401;
function generator(t) {
  return () => { t = (t * 1103515245 + 12345) % 2147483648; return t / 2147483648; };
}
const r = generator(SEED);
const deviate = (width) => 1 + (r() * 2 - 1) * width;

// TD18 — three estimation stages: at the request, once the item is reviewed, once the first team is done
const T2 = ACTUAL.map((g) => {
  const s1 = Math.max(1, Math.round(g.modules * deviate(0.8)));    // sees only the title
  const s2 = Math.round(g.modules * deviate(0.3));                 // knows the touched modules
  const s3 = Math.round((g.work + (g.team - 1)) * deviate(0.1));   // also knows the team count
  return { ...g, s1, s2, s3 };
});
writeFileSync("estimate.json", JSON.stringify(T2));

console.log(`${"work item".padEnd(28)}${"mods".padStart(6)}${"team".padStart(5)}` +
  `${"stage1".padStart(8)}${"stage2".padStart(8)}${"stage3".padStart(8)}${"actual".padStart(8)}` +
  `${"  = work + wait"}`);
for (const t of T2)
  console.log(`${t.name.padEnd(28)}${String(t.modules).padStart(6)}${String(t.team).padStart(5)}` +
    `${String(t.s1).padStart(8)}${String(t.s2).padStart(8)}${String(t.s3).padStart(8)}` +
    `${String(t.actual).padStart(8)}    ${String(t.work).padStart(2)} + ${String(t.wait).padStart(2)}` +
    `${t.rework ? `  (${t.rework} rework)` : ""}`);
console.log(`\nseed ${SEED}; total actual ${T2.reduce((s, t) => s + t.actual, 0)} rounds ` +
  `(${T2.reduce((s, t) => s + t.work, 0)} work + ${T2.reduce((s, t) => s + t.wait, 0)} wait)`);

work item                     mods team  stage1  stage2  stage3  actual  = work + wait
late fee rate                    4    3       5       4      11      18     8 + 10  (2 rework)
reservation cancellation         4    3       1       3       7       9     6 +  3  (1 rework)
membership reminder              4    3       6       5      10      18     8 + 10  (2 rework)
catalog record field             4    3       4       3       9      18     8 + 10  (2 rework)
loan period extension            3    2       1       2       6       7     5 +  2  (1 rework)
second identity step             4    3       3       3      10      18     8 + 10  (2 rework)
fee refund record                3    3       4       4       6       8     5 +  3  (1 rework)
branch delay report              3    2       4       2       6       7     5 +  2  (1 rework)
notification text format         2    1       3       3       4       4     4 +  0  (1 rework)
reservation queue                3    2       5       3       6       7     5 +  2  (1 rework)
loan rule exception              4    2       6       5       5       5     4 +  1
event log field                  3    2       5       4       6       7     5 +  2  (1 rework)

seed 20250401; total actual 126 rounds (71 work + 55 wait)

Margin of Error and the Cone’s Narrowing

TD19 — the plan is counted as “held” for an item if the estimate stays within a 20% band of the actual. TD20 — items are split into two sets by how many teams they touch: those touching one or two teams, and those touching three or more. Rationale: without a band, the word “held” cannot be turned into a number; the second split isolates the team boundary’s share of the estimate.

// cone.mjs — margin of error, the uncertainty cone narrowing, single-item vs. total estimate
import { readFileSync } from "node:fs";
const T = JSON.parse(readFileSync("estimate.json", "utf8"));

const STAGES = ["s1", "s2", "s3"];
const LABEL = { s1: "stage 1 (request)", s2: "stage 2 (review)", s3: "stage 3 (first team done)" };
const total = (f) => T.reduce((s, t) => s + f(t), 0);
const pct = (x) => `${(100 * x).toFixed(1)}%`;
// TD19 — the plan counts as "held" for an item if the estimate stays within a 20% band of the actual
const BAND = 0.2;

console.log(`${"stage".padEnd(26)}${"mean absolute error".padStart(21)}${"mean bias".padStart(13)}` +
  `${"cone: estimate/actual".padStart(23)}${"width".padStart(9)}${"plan held".padStart(12)}`);
for (const a of STAGES) {
  const ratio = T.map((t) => t[a] / t.actual).sort((x, y) => x - y);
  const mape = total((t) => Math.abs(t[a] - t.actual) / t.actual) / T.length;
  const bias = total((t) => (t[a] - t.actual) / t.actual) / T.length;
  const held = T.filter((t) => Math.abs(t[a] - t.actual) / t.actual <= BAND).length;
  console.log(`${LABEL[a].padEnd(26)}${pct(mape).padStart(21)}${pct(bias).padStart(13)}` +
    `${`${ratio[0].toFixed(2)} - ${ratio[ratio.length - 1].toFixed(2)}`.padStart(23)}` +
    `${(ratio[ratio.length - 1] - ratio[0]).toFixed(2).padStart(9)}${`${held}/${T.length}`.padStart(12)}`);
}

console.log(`\n${"stage".padEnd(26)}${"total estimate".padStart(16)}${"total actual".padStart(14)}` +
  `${"error of the total".padStart(20)}${"mean single-item error".padStart(24)}`);
for (const a of STAGES) {
  const te = total((t) => t[a]), ta = total((t) => t.actual);
  const mape = total((t) => Math.abs(t[a] - t.actual) / t.actual) / T.length;
  console.log(`${LABEL[a].padEnd(26)}${String(te).padStart(16)}${String(ta).padStart(14)}` +
    `${pct((te - ta) / ta).padStart(20)}${pct(mape).padStart(24)}`);
}

// where the error comes from: stage 2 estimates only the work
const s2 = total((t) => t.s2), actual = total((t) => t.actual);
console.log(`\nstage 2 total ${s2} rounds; actual ${actual} rounds; difference ${actual - s2} rounds`);
console.log(`  components of the difference: wait ${total((t) => t.wait)} rounds, ` +
  `work added by rework ${total((t) => 2 * t.rework)} rounds, ` +
  `modules touched ${total((t) => t.modules)} rounds (stage 2 estimates this)`);

// TD20 — items are split by how many teams they touch
console.log(`\n${"item set".padEnd(26)}${"items".padStart(7)}${"mean actual".padStart(13)}` +
  `${"wait share".padStart(14)}${"stage 2 error".padStart(16)}${"stage 3 error".padStart(16)}`);
for (const [name, f] of [["touches 1-2 teams", (t) => t.team <= 2], ["touches 3+ teams", (t) => t.team >= 3]]) {
  const k = T.filter(f), s = (g) => k.reduce((x, t) => x + g(t), 0);
  console.log(`${name.padEnd(26)}${String(k.length).padStart(7)}` +
    `${(s((t) => t.actual) / k.length).toFixed(1).padStart(13)}` +
    `${pct(s((t) => t.wait) / s((t) => t.actual)).padStart(14)}` +
    `${pct(s((t) => Math.abs(t.s2 - t.actual) / t.actual) / k.length).padStart(16)}` +
    `${pct(s((t) => Math.abs(t.s3 - t.actual) / t.actual) / k.length).padStart(16)}`);
}

stage                       mean absolute error    mean bias  cone: estimate/actual    width   plan held
stage 1 (request)                         55.8%       -52.5%            0.11 - 1.20     1.09        1/12
stage 2 (review)                          58.4%       -58.4%            0.17 - 1.00     0.83        1/12
stage 3 (first team done)                 23.5%       -23.5%            0.50 - 1.00     0.50        6/12

stage                       total estimate  total actual  error of the total  mean single-item error
stage 1 (request)                       47           126              -62.7%                   55.8%
stage 2 (review)                        41           126              -67.5%                   58.4%
stage 3 (first team done)               86           126              -31.7%                   23.5%

stage 2 total 41 rounds; actual 126 rounds; difference 85 rounds
  components of the difference: wait 55 rounds, work added by rework 30 rounds, modules touched 41 rounds (stage 2 estimates this)

item set                    items  mean actual    wait share   stage 2 error   stage 3 error
touches 1-2 teams               6          6.2         24.3%           44.6%            9.5%
touches 3+ teams                6         14.8         51.7%           72.2%           37.5%

Why the Error Stopped While the Cone Narrowed

Two columns in the first table go in opposite directions. The cone really does narrow: the width of the estimate/actual ratio falls from 1.09 to 0.83, then to 0.50. But the mean absolute error does not follow the same path — 55.8% at stage 1, 58.4% at stage 2. The error did not decrease while the uncertainty decreased.

The reason is written in the bias column. Bias is negative in all three stages: -52.5%, -58.4%, -23.5%. That is, the estimates are not merely scattered — they are all wrong in the same direction: at every stage the duration is estimated lower than it is. What the estimator learns going from stage 1 to stage 2 is the full list of touched modules; this knowledge brings the deviation down from ±80% to ±30% but does not touch the bias, because the bias was not coming from a wrong module count.

Where it comes from is named by the line in the middle of the second output block. Stage 2’s total estimate is 41 rounds, the total actual is 126 rounds; of the 85 rounds between them, 55 are waiting, 30 are the work added by rework. Stage 2 knows the number of touched modules (41 rounds) almost exactly — and that is the only thing it estimates. The estimate estimates the work; the actual duration is the flow time. The gap between the two is exactly what the previous two lessons measured: the rounds expected in a handoff, and the items that come back because of a wrong boundary.

Stage 3 brings the error down to 23.5% and raises the number of items the plan held for from 1 to 6. All it does is bring the concept of waiting into the estimate — one round per handoff. Even this crude correction closes more than half the error; the remaining 23.5% comes from handoffs where the wait is longer than one round, and from rework.

Single-Item Estimate Versus Total Estimate

The second table tests a common expectation: that the errors made on individual items cancel out in the total. They do not. At stage 2, the mean single-item absolute error is 58.4%, while the error of the total is -67.5% — the total is worse than the single item. The same relationship holds across all three stages.

Errors cancel out only when they scatter in two directions. Here, the stage 2 estimate does not exceed the actual for any of the twelve items; the highest ratio in the cone column is 1.00. When errors are biased in one direction, their magnitudes add up too. The total of a set of estimates is more reliable than a single-item error only if the errors in the set are unbiased; when bias is present, the total estimate is less reliable than the single-item estimate.

The last table shows where the bias comes from. The six items touching one or two teams have a mean actual duration of 6.2 rounds, of which 24.3% is waiting; the stage 2 error is 44.6%, the stage 3 error 9.5%. The six items touching three or more teams have a mean duration of 14.8 rounds, a wait share of 51.7%, a stage 2 error of 72.2%, and a stage 3 error of 37.5%. The same estimation method works about twice as badly on items that cross the team boundary heavily. Where estimation gets worse is the same place where the previous two lessons’ flow time got longer — items where the module boundary and the team boundary do not overlap.

Summary

• The actual duration of the twelve items was taken from the previous lessons’ flow model (total 126 rounds = 71 work + 55 wait) and compared against estimates produced in three stages; generator seed 20250401.
• The uncertainty cone narrows stage by stage (the width of the estimate/actual ratio goes 1.09 → 0.83 → 0.50), but the mean absolute error does not follow the same path: 55.8% → 58.4% → 23.5%. Going from stage 1 to stage 2, the cone narrowed but the error did not decrease.
• Bias is negative in all three stages (-52.5%, -58.4%, -23.5%): the estimates are not scattered, they are all wrong in the same direction. Of the 85 rounds between stage 2’s 41-round total estimate and the 126-round actual, 55 are waiting and 30 are the work added by rework.
• Bringing waiting into the estimate crudely (one round per handoff) brings the error down to 23.5% and raises the number of items held within the 20% band from 1/12 to 6/12.
• Errors do not cancel out: at stage 2, the mean single-item error is 58.4% while the error of the total is -67.5%. When errors are biased in one direction, their magnitudes add up too.
• Estimation gets about twice as bad on items that cross the team boundary heavily: for items touching 1–2 teams the stage 2 error is 44.6% and the wait share is 24.3%, while for items touching 3+ teams these are 72.2% and 51.7%.

Next Step

Every number in this lesson rested on one assumption: the set of work items stays fixed for the quarter. Twelve items were counted, estimated, and run through, all from the start. In reality the set of items does not stand still — a new request arrives mid-quarter, an existing item’s scope grows, another one gets canceled. In that case the estimate’s margin of error grows not only for the reasons above but because the work being estimated changes too. The next lesson takes up that change: what happens in the flow when an item’s scope grows, past what point an accepted change makes rework unavoidable, and how many rounds a control regime for change itself adds to the flow.