Watch the drop-off, not the total

Only 30% of people who start onboarding finish it.

Everyone nods. Someone suggests the copy could be clearer. Someone else mentions the email. Next quarter the number is 31% and nobody knows why.

The problem with that sentence is not that it is wrong. It is that it names the size of a loss and nothing about its location, so every response to it is a guess.

What a funnel adds

Break the same journey into steps and the conversation changes shape:

TOTAL                          FUNNEL

 onboarding completion:         create account      1,000   100%
   30%                          verify email          840    84%
                                connect data source   410    49%   <- here
 "we should improve             invite a teammate     390    95%
  onboarding"                   first report          301    77%

 three quarters of this         84% of the loss is one step, and it
 conversation have been         is the step where we ask them for a
 about the copy                 database credential

Now there is somewhere to go on Monday. Not improve onboarding, but watch four people try to connect a data source, which is a task with a beginning and an end.

That is the whole value, and it is worth being precise about what it is: a funnel locates a loss. It does not explain it. The explanation comes from watching somebody, which is an earlier move in this book, and the funnel’s job is to tell you which screen to watch.

The move

Break the journey into steps and find the one step that owns most of the loss, then go and watch that step.

A funnel is a set of choices, not a measurement

This is the part that gets left out, and it is what stops you being embarrassed in a review.

The sequence is something you chose. Real users do not move through your product in the order your funnel assumes. Every person who did the steps out of order is either miscounted or dropped, depending on how you configured it.

The conversion window is a parameter with a vendor default. Amplitude defaults to one day. Mixpanel defaults to seven. The same underlying behaviour will produce materially different percentages depending on which tool you happen to use, and nobody in the review will know that the number moved because of a setting.

And every step is conditional on the one before it. This is the trap. Step four’s percentage is computed only on the people who survived step three. Fix step three, more people arrive at step four, and step four’s percentage will change without anything about step four having changed. Teams routinely ship an improvement, watch a later step get worse, and conclude they broke something.

So read the funnel as counts as well as percentages. The percentages move against each other. The counts do not.

Where I am narrowing the outline

This move’s stated intent is that the total tells you nothing you can act on. That is too strong, and I could not support it.

There is no controlled evidence that step-level metrics produce better decisions than aggregate ones. Everything I found making that claim was published by a company selling funnel analytics. The honest argument is mechanical: a total gives you one number and no location, a funnel gives you a location, and a location is a thing you can go and look at.

Two things from earlier in the book are worth carrying here, because they explain why locating matters more than reasoning. Microsoft’s experimentation teams found only something like one idea in five to one in ten produced the improvement it was designed to produce. And people are unreliable narrators of their own behaviour, with self-reports correlating only moderately with logged reality.

Which means the room’s guesses about why people leave are worth very little, including yours. The funnel is not there to settle that argument. It is there to make it a short one, by pointing at a single screen.

What it costs

Steps are arbitrary and you will be tempted to redraw them. Splitting one step into two makes each half look better. Merging two makes a bad number disappear. Fix your step definitions and date them, because a funnel that gets redrawn is a funnel that can be made to say anything.

And survivorship makes late steps look great. The people who reach your final step are the ones who got through everything else, so they are unrepresentative by construction. A 95% completion rate on the last step usually means the last step is easy and that you already lost everybody who would have struggled with it.

Try this week

Take the metric your team quotes most often and write down the four or five steps that sit under it. Not in a tool: on paper, from memory.

Then get the counts for each step. Not percentages, counts.

Find the largest single drop, and put twenty minutes in your calendar to watch one person attempt exactly that step. You now have a specific thing to look at, chosen by arithmetic rather than by whoever spoke first in the review.