Where is the biggest leak in your funnel?

That question has two correct answers. They point at different steps, and they fund different work.

2,500 accounts · every figure checked three ways · the data and the code are below

What is funnel analysis?

Funnel analysis measures where people stop in an ordered sequence of steps. Signup, checkout, onboarding, a job application, a support escalation: anything where reaching step three means you already got through steps one and two. It is sometimes called conversion analysis or drop-off analysis, and in older marketing writing a purchase funnel, after the shape it draws.

The idea is old and the arithmetic is simple. What makes it worth doing carefully is that the obvious summary of a funnel is ambiguous, and the ambiguity is invisible: the phrase "the biggest leak" describes two different measurements that usually disagree about which step to fix.

It answers questions a single conversion rate cannot:

• Which step loses the most people, in absolute numbers?
• Which step converts worst, as a share of the people who reached it?
• Does either answer change once you split by channel, plan or device?

One thing it deliberately does not handle: people who are still mid-journey. Someone who signed up yesterday has not failed to convert, they simply have not got there yet, and a funnel counts them as a drop-off. If the timing matters more than the sequence, the tool you want is survival analysis, which is built for exactly that.

The ambiguity in "the biggest leak"

Somebody puts a funnel chart on a slide, points at the widest gap, and a sprint gets funded. The problem is that "biggest" was never defined, and the two available definitions disagree.

The volume answer is which step loses the most accounts. The efficiency answer is which step converts worst, as a proportion of the people who actually reached it. Both are correct. Both are useful. They are usually different steps, and only one of them is the step whose behaviour is genuinely unusual.

Both answers, on real numbers

Everything below runs on one dataset: a quarter of self-serve signups at a B2B software company. 2,500 accounts arrive at a plans page, and 180 end up paying. Five steps, one row per account per step reached.

End to end, 7.2% of visitors pay. The volume answer is the first step: 1,450 accounts never start a trial, more than any other gap loses. The efficiency answer is three steps further down: of the accounts that activate, only 36% ever invite a teammate.

Why the first step almost always wins the volume answer

Worth naming, because it stops the volume answer being over-read: the first gap draws from the largest pool. A step that loses a modest share of 2,500 people will lose more humans than a step that loses a terrible share of 680. So "loses the most" is partly a fact about arithmetic and position, not only about how bad the step is. The efficiency answer is what tells you a step is behaving unusually.

The finding the aggregate was hiding

A rate for the whole funnel is a summary of groups that may behave nothing alike. Split these accounts by acquisition channel and one of them does something the aggregate cannot show.

Partner accounts activate at 76% against paid search's 60%, a gap of nearly 16 points that chance would produce about three times in a hundred thousand. Then at the next step they invite a teammate at 20% against paid search's 40%, a reversal of 20 points that chance would produce about eight times in a million.

The aggregate 36% invite rate is an average over a channel running at 20% and one running at 40%. It describes neither. And the business reading is specific: partner accounts arrive enthusiastic, get set up, and then stay single-seat. Cut the channel on its invite rate and you lose the thing it was best at.

What the analysis needs from your file

One row per account per step reached. Three columns, plus one optional column that unlocks the most useful part.

Real rows from the file below. One account appears once per step it reached, and simply stops appearing at the step it did not reach. That is how the analysis knows where it fell out; there is no "did not convert" row to add.

1. An account or session id, so steps can be linked into a journey.
2. A stage label, and the steps have to be genuinely ordered. Optional or parallel stages produce a funnel whose narrowing is an artifact of the order you chose.
3. A timestamp for when that step was reached.
4. Something to break it down by, and this is the one worth adding: channel, plan, device. Without it you cannot find a reversal like the one above.

And per-step totals are not enough. If all you have is 2,500 then 1,050 then 680, you cannot segment, cannot compute an interval, and cannot tell the two biggest-leak answers apart. Totals cannot be turned back into rows.

Every number here, you can re-run

This page is not a screenshot of an analysis. It is an analysis. The document derives every figure when it runs, and it was checked three ways: by the document itself, by R's two-proportion test called directly on each step, and by an independent Python reimplementation. All three agree to every digit shown.

Read the full knitted analysis, the document itself with every chart, table and line of code as it ran. Or take the source and the data below and run it yourself.

The limits, stated plainly

Three things this analysis cannot tell you, and none of them announces itself in the output.

It cannot see people who are still mid-journey. An account that started last week and has not activated yet is counted as a drop-off. Over a short window, or a period where signups grew, that alone can move every rate downward. Funnel analysis has no concept of an unfinished journey; survival analysis does.

It cannot tell you why. The reversal above is a real, measured difference. Whether partner accounts stay single-seat because of who they are, how they were sold, or what the product does after activation is not in this data.

It will draw whatever order you map. The narrowing shape is produced by the sequence you declare, so a mis-declared sequence produces a confident and meaningless funnel.

Download

Two files. The notebook regenerates the dataset from its seed, so re-knitting reproduces this page's figures exactly.

• funnel.Rmd · the analysis, R Markdown source10 KB
• funnel_events.csv · 4,655 events across 2,500 accounts251 KB

Both files are released under CC BY 4.0: use them anywhere, including commercially, with attribution.