The business question

A B2B software company signs 75 new business accounts a month. Eight months of data. The board deck shows active accounts, and that line has gone up every single month.

Underneath it, nearly half of every account ever signed has already churned.

Both statements are true at the same time, and this worked example shows how. It also answers a second question the top line cannot: from month 5 the company replaced self-serve onboarding with a guided setup call for every new account. Did it work?

The data

One row per account per month it was active. Columns: account_id, activity_date (first of the active month), signup_month, onboarding, seats, mrr.

The dataset is CONSTRUCTED from two retention curves — one for the self-serve cohorts, one for the guided-setup cohorts — with set.seed(20260824) fixing seat counts and row order. An account is active in tenure month t only if it was active in every month before t, so churn is absorbing: nobody comes back from the dead. Re-knitting reproduces data.csv byte-for-byte.

set.seed(20260824)

cohort_size  <- 75
cohorts      <- sprintf("2026-%02d", 1:8)
cohort_start <- as.Date(sprintf("2026-%02d-01", 1:8))

# accounts still active at tenure 0,1,2,... out of 75
curve_self   <- c(75, 47, 38, 32, 27, 24, 21, 19)   # cohorts 1-4, self-serve
curve_guided <- c(75, 59, 48, 41, 36, 32, 29, 27)   # cohorts 5-8, guided setup call

rows <- do.call(rbind, lapply(seq_along(cohorts), function(ci) {
  guided <- ci >= 5
  curve  <- if (guided) curve_guided else curve_self
  window <- 0:(8 - ci)                       # months observable before the data ends
  ids    <- sprintf("acct_%03d_%02d", ci, seq_len(cohort_size))
  seats  <- sample(3:40, cohort_size, replace = TRUE)

  do.call(rbind, lapply(window, function(t) {
    alive <- seq_len(curve[t + 1])           # curve is decreasing => activity is contiguous
    data.frame(
      account_id    = ids[alive],
      activity_date = seq(cohort_start[ci], by = "month", length.out = t + 1)[t + 1],
      signup_month  = cohorts[ci],
      onboarding    = if (guided) "guided_setup" else "self_serve",
      seats         = seats[alive],
      mrr           = seats[alive] * 49,
      stringsAsFactors = FALSE
    )
  }))
}))

rows <- rows[order(rows$account_id, rows$activity_date), ]
rownames(rows) <- NULL
write.csv(rows, "data.csv", row.names = FALSE)

nrow(rows)
## [1] 1623
head(rows, 6)
account_id activity_date signup_month onboarding seats mrr
acct_001_01 2026-01-01 2026-01 self_serve 28 1372
acct_001_01 2026-02-01 2026-01 self_serve 28 1372
acct_001_01 2026-03-01 2026-01 self_serve 28 1372
acct_001_01 2026-04-01 2026-01 self_serve 28 1372
acct_001_01 2026-05-01 2026-01 self_serve 28 1372
acct_001_01 2026-06-01 2026-01 self_serve 28 1372

The line the board sees

months  <- sprintf("2026-%02d", 1:8)
active  <- sapply(cohort_start, function(m) sum(rows$activity_date == m))
new     <- rep(cohort_size, 8)
cancels <- c(NA, head(active, -1) + cohort_size - tail(active, -1))
net     <- c(NA, diff(active))

topline <- data.frame(month = months, active_accounts = active,
                      new_accounts = new, cancellations = cancels,
                      net_change = net)
knitr::kable(topline, align = "lrrrr",
             col.names = c("month", "active accounts", "new", "cancelled", "net change"),
             caption = "The top line rises every month. The cancellation column is why that is misleading.")
The top line rises every month. The cancellation column is why that is misleading.
month active accounts new cancelled net change
2026-01 75 75 NA NA
2026-02 122 75 28 47
2026-03 160 75 37 38
2026-04 192 75 43 32
2026-05 219 75 48 27
2026-06 255 75 39 36
2026-07 286 75 44 31
2026-08 314 75 47 28

Every month is higher than the last. And every month after the first, the company is losing accounts it already had — 48 of them in the worst month — while the top line still prints a gain, because 75 new accounts walk in the door behind them.

par(mar = c(4, 5, 3, 1))
bp <- barplot(active, names.arg = months, col = "#5fa9dd", border = NA,
              ylim = c(0, 380), ylab = "active accounts",
              main = "Active accounts by month, with monthly cancellations")
lines(bp, cancels, type = "b", pch = 19, col = "#F97316", lwd = 2)
text(bp, active + 16, active, cex = 0.85, font = 2)
legend("topleft", c("active accounts", "accounts cancelled that month"),
       fill = c("#5fa9dd", NA), border = NA, lty = c(NA, 1), pch = c(NA, 19),
       col = c(NA, "#F97316"), bty = "n")

ever    <- length(unique(rows$account_id))
alive_8 <- sum(rows$activity_date == cohort_start[8])
c(accounts_ever = ever, active_month_8 = alive_8,
  churned = ever - alive_8, churn_pct = round(100 * (ever - alive_8) / ever, 4))
##  accounts_ever active_month_8        churned      churn_pct 
##       600.0000       314.0000       286.0000        47.6667

The cohort table

Same rows, grouped by the month an account signed rather than the calendar.

tenure <- function(d, s) {
  (as.integer(format(d, "%Y")) - as.integer(substr(s, 1, 4))) * 12 +
    (as.integer(format(d, "%m")) - as.integer(substr(s, 6, 7)))
}
rows$tenure <- tenure(as.Date(rows$activity_date), rows$signup_month)

tri <- matrix(NA_real_, 8, 8, dimnames = list(cohorts, paste0("m", 0:7)))
for (i in seq_along(cohorts)) for (t in 0:(8 - i)) {
  n <- sum(rows$signup_month == cohorts[i] & rows$tenure == t)
  tri[i, t + 1] <- 100 * n / cohort_size
}
knitr::kable(round(tri, 1), caption =
  "Retention % by cohort (rows) and months since signup (columns). Read DOWN a column: same age, fair comparison.")
Retention % by cohort (rows) and months since signup (columns). Read DOWN a column: same age, fair comparison.
m0 m1 m2 m3 m4 m5 m6 m7
2026-01 100 62.7 50.7 42.7 36 32 28 25.3
2026-02 100 62.7 50.7 42.7 36 32 28 NA
2026-03 100 62.7 50.7 42.7 36 32 NA NA
2026-04 100 62.7 50.7 42.7 36 NA NA NA
2026-05 100 78.7 64.0 54.7 NA NA NA NA
2026-06 100 78.7 64.0 NA NA NA NA NA
2026-07 100 78.7 NA NA NA NA NA NA
2026-08 100 NA NA NA NA NA NA NA

Reading across a row shows how one cohort decays. Reading down a column compares cohorts at the same age — the only fair comparison, and the one that answers whether the product got better at keeping accounts.

par(mar = c(4, 5.5, 3, 1))
image(x = 0:7, y = 1:8, z = t(tri[8:1, ]), col = hcl.colors(24, "Blues", rev = TRUE),
      axes = FALSE, xlab = "months since signup", ylab = "",
      main = "Cohort retention heatmap")
axis(1, at = 0:7); axis(2, at = 1:8, labels = rev(cohorts), las = 1)
for (i in 1:8) for (t in 0:7) if (!is.na(tri[i, t + 1]))
  text(t, 9 - i, sprintf("%.0f", tri[i, t + 1]), cex = 0.75)
box()

Did the guided setup call work?

Column m1 is month-1 retention. Cohorts 1–4 were self-serve; cohorts 5–8 got the guided setup call.

Cohort 8 is excluded from this comparison. It signed up in the last month of the data, so its month 1 has not happened yet — that is why the table is a triangle. Counting its 75 accounts as “not retained” would not be a small distortion: it would drag the guided-setup group from 78.7% to 59.0% and reverse the finding. Never compare a cell that exists against a cell that has not been observed.

m1 <- sapply(seq_along(cohorts), function(i)
  sum(rows$signup_month == cohorts[i] & rows$tenure == 1))
observed_m1 <- which(sapply(seq_along(cohorts), function(i) (8 - i) >= 1))  # cohorts 1-7
pre_i  <- intersect(1:4, observed_m1)
post_i <- intersect(5:8, observed_m1)

pre  <- sum(m1[pre_i]);  n_pre  <- length(pre_i)  * cohort_size
post <- sum(m1[post_i]); n_post <- length(post_i) * cohort_size

data.frame(group = c(sprintf("self_serve (cohorts %s)",  paste(range(pre_i),  collapse = "-")),
                     sprintf("guided_setup (cohorts %s)", paste(range(post_i), collapse = "-"))),
           retained_m1 = c(pre, post), accounts = c(n_pre, n_post),
           rate = sprintf("%.4f%%", 100 * c(pre / n_pre, post / n_post)))
group retained_m1 accounts rate
self_serve (cohorts 1-4) 188 300 62.6667%
guided_setup (cohorts 5-7) 177 225 78.6667% 
pt <- prop.test(c(post, pre), c(n_post, n_pre))
pt
## 
##  2-sample test for equality of proportions with continuity correction
## 
## data:  c(post, pre) out of c(n_post, n_pre)
## X-squared = 14.788, df = 1, p-value = 0.0001203
## alternative hypothesis: two.sided
## 95 percent confidence interval:
##  0.07955379 0.24044621
## sample estimates:
##    prop 1    prop 2 
## 0.7866667 0.6266667
c(diff_pp = 100 * (post / n_post - pre / n_pre),
  ci_lo = 100 * pt$conf.int[1], ci_hi = 100 * pt$conf.int[2])
##   diff_pp     ci_lo     ci_hi 
## 16.000000  7.955379 24.044621
chisq.test(matrix(c(post, n_post - post, pre, n_pre - pre), 2, byrow = TRUE))
## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  matrix(c(post, n_post - post, pre, n_pre - pre), 2, byrow = TRUE)
## X-squared = 14.788, df = 1, p-value = 0.0001203
fisher.test(matrix(c(post, n_post - post, pre, n_pre - pre), 2, byrow = TRUE))
## 
##  Fisher's Exact Test for Count Data
## 
## data:  matrix(c(post, n_post - post, pre, n_pre - pre), 2, byrow = TRUE)
## p-value = 8.238e-05
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
##  1.455114 3.339714
## sample estimates:
## odds ratio 
##   2.193568
par(mar = c(4, 5, 3, 1))
shown <- observed_m1
cols  <- ifelse(shown >= 5, "#3fbf6f", "#5fa9dd")
vals  <- 100 * m1[shown] / cohort_size
bp <- barplot(vals, names.arg = cohorts[shown], col = cols, border = NA,
              ylim = c(0, 95), ylab = "month-1 retention (%)",
              main = "Month-1 retention by cohort (green = guided setup call)")
text(bp, vals + 4, sprintf("%.1f%%", vals), cex = 0.85)
segments(bp[4] + 0.6, 0, bp[4] + 0.6, 90, lty = 2, col = "#918e98")
mtext(sprintf("cohort %s omitted: its month 1 has not happened yet", cohorts[8]),
      side = 1, line = 2.6, cex = 0.8, col = "#918e98")

The chooser the lesson teaches

  1. A calendar month and a cohort answer different questions. A calendar month mixes accounts of every age, so it answers how many were active. A cohort follows accounts that arrived together, so it answers how many stayed. Only the second is retention.
  2. The top line cannot see a leak while acquisition covers it. Active accounts rose every month here while 47.7% of all accounts churned. The only month a top line tells the truth about retention is a month with no new accounts.
  3. Compare down the columns, never across the rows. Same-age comparison is the whole point of the triangle; a row tells you about ageing, not about improvement.
  4. Judge a product change by the cohorts who arrived after it. The guided setup call is testable precisely because cohorts 5–8 are a clean post-change group measured at the same age as the pre-change ones.