Conversion Funnel

The previous lesson measured a task as a whole: whether it was completed, how long it took, how many errors were made. Three of the fourteen sessions failed to complete the task. What it did not measure is where those three sessions got stuck.

This question is answered not by a small-sample test but by the behavior record of all users. The conversion funnel is an analysis that breaks a task into sequential steps and counts how many users pass through each one. In the catalog interface, the funnel’s steps are the borrow flow: search, results list, record detail, copy selection, borrow confirmation, completion.

The Funnel’s Three Columns

Each step has three separate numbers, and they get confused with one another.

Step conversion is the proportion of those entering a step who move on to the next one.

Drop-off rate is the proportion of those who leave at the same step; it is the complement of the step conversion.

Cumulative conversion is the proportion of those entering the first step who reach a given step. Overall conversion is the product of the step conversions — this is the funnel’s most important structural property.

// funnel.mjs — step-by-step drop-off analysis of the borrow flow

const FUNNEL = [
  ["search performed", 10000],
  ["results list viewed", 8200],
  ["record detail opened", 5330],
  ["copy selected", 4530],
  ["borrow confirmation opened", 2500],
  ["borrow completed", 2350],
];

function wilson(k, N, z = 1.96) {
  const p = k / N, d = 1 + z * z / N;
  const m = (p + z * z / (2 * N)) / d;
  const y = (z * Math.sqrt(p * (1 - p) / N + z * z / (4 * N * N))) / d;
  return [m - y, m + y];
}

console.log("step                         entering  advancing  step conversion  drop-off rate  lost  cumulative");
for (let i = 1; i < FUNNEL.length; i++) {
  const entering = FUNNEL[i - 1][1], advancing = FUNNEL[i][1];
  const rate = advancing / entering;
  const [a, u] = wilson(advancing, entering);
  console.log(
    `${FUNNEL[i][0].padEnd(28)} ${String(entering).padStart(6)} ${String(advancing).padStart(8)}` +
    ` ${((rate * 100).toFixed(1) + " %").padStart(15)} ${(((1 - rate) * 100).toFixed(1) + " %").padStart(13)}` +
    ` ${String(entering - advancing).padStart(6)} ${((advancing / FUNNEL[0][1] * 100).toFixed(1) + " %").padStart(11)}` +
    `   [${(a * 100).toFixed(1)}, ${(u * 100).toFixed(1)}]`
  );
}
const overall = FUNNEL[FUNNEL.length - 1][1] / FUNNEL[0][1];
console.log(`overall conversion: ${(overall * 100).toFixed(2)} %`);

// Is the highest drop-off rate the same step as the largest user loss?
let lowestRate = 1, lowestRateStep = "", highestLoss = 0, highestLossStep = "";
for (let i = 1; i < FUNNEL.length; i++) {
  const rate = FUNNEL[i][1] / FUNNEL[i - 1][1], loss = FUNNEL[i - 1][1] - FUNNEL[i][1];
  if (rate < lowestRate) { lowestRate = rate; lowestRateStep = FUNNEL[i][0]; }
  if (loss > highestLoss) { highestLoss = loss; highestLossStep = FUNNEL[i][0]; }
}
console.log(`\nhighest drop-off rate: ${lowestRateStep} (${((1 - lowestRate) * 100).toFixed(1)} %)`);
console.log(`largest user loss: ${highestLossStep} (${highestLoss} users)`);

// Effect on overall conversion of raising every step to 0.98 (the funnel is multiplicative)
console.log("\nstep                         current rate  raised to 0.98: overall conversion  relative gain");
for (let i = 1; i < FUNNEL.length; i++) {
  const rate = FUNNEL[i][1] / FUNNEL[i - 1][1];
  const updated = overall * (0.98 / rate);
  console.log(
    `${FUNNEL[i][0].padEnd(28)} ${((rate * 100).toFixed(1) + " %").padStart(13)}` +
    ` ${((updated * 100).toFixed(2) + " %").padStart(32)} ${("x" + (updated / overall).toFixed(2)).padStart(13)}`
  );
}

// An aggregated funnel can hide two segments
const SEGMENTS = {
  "first-time visitor": [6000, 4800, 2900, 2400, 900, 820],
  "registered user": [4000, 3400, 2430, 2130, 1600, 1530],
};
console.log("\nsegment              n     step conversions                        overall");
for (const [name, v] of Object.entries(SEGMENTS)) {
  const rates = [];
  for (let i = 1; i < v.length; i++) rates.push((v[i] / v[i - 1] * 100).toFixed(1) + "%");
  console.log(`${name.padEnd(20)} ${String(v[0]).padStart(5)} ${rates.join("  ").padEnd(40)} ${(v[v.length - 1] / v[0] * 100).toFixed(2)} %`);
}
const combined = FUNNEL.map((_, i) => Object.values(SEGMENTS).reduce((a, v) => a + v[i], 0));
const combinedRates = [];
for (let i = 1; i < combined.length; i++) combinedRates.push((combined[i] / combined[i - 1] * 100).toFixed(1) + "%");
console.log(`${"aggregated".padEnd(20)} ${String(combined[0]).padStart(5)} ${combinedRates.join("  ").padEnd(40)} ${(combined[5] / combined[0] * 100).toFixed(2)} %`);

// Is the difference between the two segments at the fourth step real?
const [a1, u1] = wilson(SEGMENTS["first-time visitor"][4], SEGMENTS["first-time visitor"][3]);
const [a2, u2] = wilson(SEGMENTS["registered user"][4], SEGMENTS["registered user"][3]);
console.log(`\ntransition to borrow confirmation  first-time [${(a1 * 100).toFixed(1)}, ${(u1 * 100).toFixed(1)}]  registered [${(a2 * 100).toFixed(1)}, ${(u2 * 100).toFixed(1)}]  intervals ${u1 < a2 || u2 < a1 ? "disjoint" : "overlapping"}`);

step                         entering  advancing  step conversion  drop-off rate  lost  cumulative
results list viewed           10000     8200          82.0 %        18.0 %   1800      82.0 %   [81.2, 82.7]
record detail opened           8200     5330          65.0 %        35.0 %   2870      53.3 %   [64.0, 66.0]
copy selected                  5330     4530          85.0 %        15.0 %    800      45.3 %   [84.0, 85.9]
borrow confirmation opened     4530     2500          55.2 %        44.8 %   2030      25.0 %   [53.7, 56.6]
borrow completed               2500     2350          94.0 %         6.0 %    150      23.5 %   [93.0, 94.9]
overall conversion: 23.50 %

highest drop-off rate: borrow confirmation opened (44.8 %)
largest user loss: record detail opened (2870 users)

step                         current rate  raised to 0.98: overall conversion  relative gain
results list viewed                 82.0 %                          28.09 %         x1.20
record detail opened                65.0 %                          35.43 %         x1.51
copy selected                       85.0 %                          27.10 %         x1.15
borrow confirmation opened          55.2 %                          41.73 %         x1.78
borrow completed                    94.0 %                          24.50 %         x1.04

segment              n     step conversions                        overall
first-time visitor    6000 80.0%  60.4%  82.8%  37.5%  91.1%        13.67 %
registered user       4000 85.0%  71.5%  87.7%  75.1%  95.6%        38.25 %
aggregated           10000 82.0%  65.0%  85.0%  55.2%  94.0%        23.50 %

transition to borrow confirmation  first-time [35.6, 39.5]  registered [73.2, 76.9]  intervals disjoint

The Highest Drop-off Rate and the Largest Loss Are Not the Same Step

The first table gives two different “worst steps.” The step with the highest drop-off rate is the transition to borrow confirmation: 44.8 percent. The step that loses the most users is opening the record detail: 2870 users.

The difference comes from the steps working with audiences of different sizes. 8200 users enter the record detail step; a 35 percent drop-off amounts to 2870 people. 4530 users enter the borrow confirmation step; the higher 44.8 percent drop-off amounts to 2030 people.

The two numbers answer two different questions. The drop-off rate says how bad the step itself is; the number of users lost says that step’s contribution to the total loss. If a report writes only one of them, the reader assumes the other.

The Multiplicative Funnel and the Share of Improvement

The second table determines which step to intervene on, and the answer gives a third ranking.

Because overall conversion is the product of the step rates, raising a step’s rate from $r$ to $r'$ multiplies overall conversion by a factor of $r'/r$. This factor depends only on that step’s current rate; it does not depend on how many users enter the step.

The table computes what happens if every step’s rate is raised to 0.98. The largest relative gain comes from the borrow confirmation step: x1.78, that is, overall conversion goes from 23.50 percent to 41.73. The record detail step — the one that loses the most users — gives x1.51. The completion step, whose rate is already 94 percent, gives only x1.04.

Rule: the share of improvement lies with the lowest-rate step, not the step that loses the most users. Intuition runs the other way; the place where the most people leave looks like the most urgent problem.

The computation carries two assumptions that must be stated. First, the steps are treated as independent of one another: improving one step does not change the rates of the following steps. In reality it can — carrying more undecided users into the next step can lower that step’s rate. Second, the 0.98 target was chosen as a common ceiling, not as a reachability calculation; not every step can be raised to it. The table gives a priority ranking, not a promise.

The Aggregated Funnel Hides Segments

The third table shows the funnel’s most common mistake. The aggregated rate for the transition to borrow confirmation is 55.2 percent. The same step is 37.5 percent for first-time visitors and 75.1 percent for registered users.

55.2 percent describes no user segment at all; it is a number born of the mixing ratio of two different behaviors. An intervention on this step should be designed around the first-time visitor; someone looking at the aggregated number underestimates the size of the problem by half.

The last line tests whether the difference is real. The 95 percent confidence interval for the first-time segment is [35.6, 39.5], and for the registered segment [73.2, 76.9]. The intervals are disjoint; the difference cannot be explained by sample fluctuation.

This split connects directly to the framework from the behavior design topic. A first-time visitor meets membership registration, pickup-branch selection, and duration decisions on the borrow confirmation screen; a registered user has already made these decisions. A difference in ability is not a difference in motivation, and an intervention aimed at raising motivation will not work at this step.

Rules for Building a Funnel

Steps are defined around the user’s task. “Page viewed” is not a step; “copy selected” is a step. A funnel built around the interface’s internal events shows not where the user got stuck but what the system logged.

Drop-off rate and loss count are written together. One gives the step’s quality, the other its contribution.

Every step’s rate is given with a confidence interval. In a funnel of ten thousand, the user count shrinks quickly through the last steps; the rates at the later steps are more uncertain.

The funnel is broken out along at least one dimension. First-time visitor versus registered user, narrow screen versus wide screen, arriving by search versus by navigation. A “worst step” found without breaking out the segments is often a mixing effect.

The funnel does not say why. At the borrow confirmation step, 62.5 percent of first-time visitors leave; why they leave is not in this table. The funnel says which step to look at.

Summary

• The conversion funnel breaks a task into sequential steps; overall conversion is the product of the step rates.
• The highest drop-off rate and the largest user loss can occur at different steps: in this computation, the borrow confirmation step has the worst rate at 44.8 percent drop-off, while the record detail step has the largest loss at 2870 users.
• The share of improvement depends on the step’s current rate, not its user count; raising the lowest-rate step to 0.98 moves overall conversion by a factor of x1.78, while raising the step with the largest loss moves it by x1.51.
• The computation assumes the steps are independent and takes 0.98 as a common ceiling; its output is a priority ranking, not a reachable target.
• The aggregated funnel hides segments: the 55.2 percent step rate is a mixture of two distinct behaviors ranging between 37.5 and 75.1 percent, and their confidence intervals are disjoint.
• The funnel says which step to look at; it does not say what happens at that step.

Next Step

When the funnel shows that a step is bad, the next task is to try a change. But the measured number coming out different after the change does not mean the change worked; funnel rates move on their own from week to week. The next lesson builds this distinction: how many users are needed to test a change, when a measured difference is real, why stopping an experiment early inflates the wrong result, and what correction is required when testing several changes at once.