Brief

A B2B software company runs demand generation on a weekly cycle. Finance wants one question answered before next year’s budget is set:

Marketing spend and closed deals move together. If we raise spend, do we get deals?

Two years of weekly operating data are on hand. This review runs the correlation matrix, then tests whether the headline relationship survives the two questions every correlation has to answer: what else could be driving both, and is it just time.

The answer to the budget question is no, not directly — and the second half of this document shows a correlation in the same dataset that passes a careful significance test and is still an artefact.

The data

One row per operating week. All figures are the company’s own weekly totals.

column meaning unit
week week index since the start of the window 1–104
week_start Monday of that week date
marketing_spend_usd demand-gen spend booked that week USD
sessions unique website sessions count
demos_booked demos booked from those sessions count
closed_deals new business closed that week count
support_tickets inbound support tickets from the installed base count

The dataset is simulated to a stated ground truth so that this document can be used to teach: spend buys sessions, sessions produce demos, demos produce deals, and support_tickets is generated from the growth of the installed base alone — it has no dependency on spend whatsoever. Generation is deterministic (set.seed(90210)); re-knitting reproduces data.csv byte-for-byte. Every number reported below is computed from the rounded, written-out data.csv — the same file the platform tool consumes.

set.seed(90210)
n <- 104
w <- 1:n

# Demand-gen budget: an annual plan that ramps, an annual seasonal shape, and noisy
# monthly execution against it.
spend <- round(pmax(9000 + 52 * w + 900 * sin(2 * pi * w / 52) + rnorm(n, 0, 1800), 3000))

# The funnel, in order. Note there is NO direct spend -> deals term: every dollar's
# effect on deals is routed through sessions, then demos.
sessions <- round(380 + 0.048 * spend + rnorm(n, 0, 62))
demos    <- round(0.062 * sessions + rnorm(n, 0, 2.8))
deals    <- round(0.34 * demos + rnorm(n, 0, 1.1))

# Support load tracks the installed base, which grows on its own trend.
tickets  <- round(140 + 1.9 * w + rnorm(n, 0, 46))

d <- data.frame(
  week                = w,
  week_start          = as.character(seq(as.Date("2024-09-02"), by = "week", length.out = n)),
  marketing_spend_usd = spend,
  sessions            = sessions,
  demos_booked        = demos,
  closed_deals        = deals,
  support_tickets     = tickets
)
write.csv(d, "data.csv", row.names = FALSE)
nrow(d)
## [1] 104
num <- d[, c("marketing_spend_usd", "sessions", "demos_booked",
             "closed_deals", "support_tickets")]
knitr::kable(
  data.frame(
    metric = names(num),
    min    = sapply(num, min),
    median = sapply(num, median),
    mean   = round(sapply(num, mean), 1),
    max    = sapply(num, max),
    sd     = round(sapply(num, sd), 1),
    row.names = NULL
  ),
  caption = "Weekly operating metrics, 104 weeks"
)
Weekly operating metrics, 104 weeks
metric min median mean max sd
marketing_spend_usd 5470 11299.0 11594.0 18288 2587.3
sessions 607 954.0 934.2 1393 134.3
demos_booked 39 58.0 57.5 86 8.8
closed_deals 12 20.0 19.7 30 3.1
support_tickets 101 230.5 234.1 397 66.9

The correlation matrix

M <- cor(num)
knitr::kable(round(M, 4), caption = "Pearson correlation matrix")
Pearson correlation matrix
marketing_spend_usd sessions demos_booked closed_deals support_tickets
marketing_spend_usd 1.0000 0.8832 0.8120 0.7848 0.4613
sessions 0.8832 1.0000 0.9282 0.8953 0.3409
demos_booked 0.8120 0.9282 1.0000 0.9361 0.2963
closed_deals 0.7848 0.8953 0.9361 1.0000 0.2977
support_tickets 0.4613 0.3409 0.2963 0.2977 1.0000 
op <- par(mar = c(8, 8, 2, 2))
image(1:5, 1:5, M[, 5:1], axes = FALSE, xlab = "", ylab = "", zlim = c(-1, 1),
      col = colorRampPalette(c("#5fa9dd", "#f2f0ec", "#F97316"))(41))
axis(1, 1:5, gsub("_", " ", colnames(M)), las = 2, cex.axis = 0.85)
axis(2, 1:5, rev(gsub("_", " ", colnames(M))), las = 2, cex.axis = 0.85)
for (i in 1:5) for (j in 1:5) {
  text(i, 6 - j, sprintf("%.2f", M[i, j]), cex = 0.9)
}
box()

par(op)

Every pair is worth a sentence, but the budget question is one cell: spend against closed deals.

ct_headline <- cor.test(d$marketing_spend_usd, d$closed_deals)
ct_headline
## 
##  Pearson's product-moment correlation
## 
## data:  d$marketing_spend_usd and d$closed_deals
## t = 12.79, df = 102, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.6976972 0.8490800
## sample estimates:
##       cor 
## 0.7848293
plot(d$marketing_spend_usd, d$closed_deals, pch = 19, col = "#F97316",
     xlab = "marketing spend (USD / week)", ylab = "closed deals / week",
     main = sprintf("r = %.3f  (p = %.2g)", ct_headline$estimate, ct_headline$p.value))
abline(lm(closed_deals ~ marketing_spend_usd, data = d), col = "#333", lwd = 2)

r = 0.7848 is a strong relationship on 104 weeks of real operating data. Most budget decks stop here.

Question one: what else could be driving both?

Spend is not the only thing correlated with deals. Sessions and demos are correlated with deals more strongly — and both sit on the path between spend and deals.

cand <- rbind(
  data.frame(pair = "sessions ~ closed_deals",
             r = cor(d$sessions, d$closed_deals),
             p = cor.test(d$sessions, d$closed_deals)$p.value),
  data.frame(pair = "demos_booked ~ closed_deals",
             r = cor(d$demos_booked, d$closed_deals),
             p = cor.test(d$demos_booked, d$closed_deals)$p.value),
  data.frame(pair = "marketing_spend_usd ~ sessions",
             r = cor(d$marketing_spend_usd, d$sessions),
             p = cor.test(d$marketing_spend_usd, d$sessions)$p.value)
)
knitr::kable(transform(cand, r = round(r, 4), p = signif(p, 4)),
             caption = "The candidate drivers")
The candidate drivers
pair r p
sessions ~ closed_deals 0.8953 0
demos_booked ~ closed_deals 0.9361 0
marketing_spend_usd ~ sessions 0.8832 0

Partial correlation answers the question directly: hold sessions statistically fixed, and ask what is left between spend and deals. Mechanically, regress each of the two variables on sessions, keep the residuals, and correlate those.

partial_cor <- function(x, y, z) {
  cor.test(residuals(lm(x ~ z)), residuals(lm(y ~ z)))
}
pc_sessions <- partial_cor(d$marketing_spend_usd, d$closed_deals, d$sessions)
pc_sessions
## 
##  Pearson's product-moment correlation
## 
## data:  residuals(lm(x ~ z)) and residuals(lm(y ~ z))
## t = -0.28633, df = 102, p-value = 0.7752
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.2197287  0.1651499
## sample estimates:
##         cor 
## -0.02833973
bars <- c(`spend ~ deals` = unname(ct_headline$estimate),
          `holding sessions fixed` = unname(pc_sessions$estimate))
bp <- barplot(bars, ylim = c(-0.1, 0.9), col = c("#F97316", "#9aa0a6"),
              ylab = "correlation with closed deals")
abline(h = 0, col = "#333")
text(bp, bars + 0.05 * sign(bars + 1e-9), sprintf("%.3f", bars), cex = 1.1)

The relationship collapses: 0.7848 → -0.0283 (p = 0.775).

Among weeks that drew the same number of sessions, spending more closed no additional deals. That is exactly what the ground truth says: spend has no direct path to deals. Its entire apparent effect was sessions wearing a marketing badge.

Question two: is it just time?

The same dataset holds a second, more dangerous correlation.

ct_drift <- cor.test(d$marketing_spend_usd, d$support_tickets)
ct_drift
## 
##  Pearson's product-moment correlation
## 
## data:  d$marketing_spend_usd and d$support_tickets
## t = 5.2515, df = 102, p-value = 8.275e-07
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.2949530 0.6005647
## sample estimates:
##       cor 
## 0.4613353

Spend against support tickets: r = 0.4613, p = 8.3e-07. A plausible story writes itself — paid acquisition brings lower-quality accounts, and they file more tickets. The story is false. Tickets in this dataset are generated from the installed base’s own growth and never see the spend column.

A permutation test is the careful analyst’s move here: shuffle one column many times, rebuild the correlation each time, and see how often chance alone produces something this big.

set.seed(11)
obs  <- cor(d$marketing_spend_usd, d$support_tickets)
perm <- replicate(20000, cor(d$marketing_spend_usd, sample(d$support_tickets)))
n_extreme <- sum(abs(perm) >= abs(obs))
p_perm    <- (n_extreme + 1) / (length(perm) + 1)
c(observed = obs, n_at_least_as_extreme = n_extreme, p_permutation = p_perm)
##              observed n_at_least_as_extreme         p_permutation 
##          0.4613353445          0.0000000000          0.0000499975
hist(perm, breaks = 60, col = "#d7d9dc", border = "white",
     main = "Permutation null vs the observed correlation",
     xlab = "correlation under shuffling", xlim = c(-0.6, 0.6))
abline(v = obs, col = "#F97316", lwd = 3)
text(obs, par("usr")[4] * 0.8, sprintf(" observed %.3f", obs), col = "#F97316", pos = 4)

The permutation test confirms the correlation: 0 of 20,000 shuffles reached it. And the permutation test is wrong, because shuffling assumes the weeks are interchangeable. They are not — both columns climb across the two years, and shuffling destroys the very trend that created the correlation. The null it builds is a null that never existed.

Ask the third-variable question again, with time as the third variable:

pc_week <- partial_cor(d$marketing_spend_usd, d$support_tickets, d$week)
pc_week
## 
##  Pearson's product-moment correlation
## 
## data:  residuals(lm(x ~ z)) and residuals(lm(y ~ z))
## t = 0.64496, df = 102, p-value = 0.5204
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.1304588  0.2532110
## sample estimates:
##        cor 
## 0.06373066
op <- par(mfrow = c(1, 2), mar = c(4, 4, 3, 1))
plot(d$week, scale(d$marketing_spend_usd), type = "l", col = "#F97316", lwd = 2,
     ylim = c(-3, 3), xlab = "week", ylab = "standardised", main = "Both simply drift up")
lines(d$week, scale(d$support_tickets), col = "#5fa9dd", lwd = 2)
legend("topleft", c("spend", "tickets"), col = c("#F97316", "#5fa9dd"), lwd = 2, bty = "n", cex = 0.85)
plot(residuals(lm(d$marketing_spend_usd ~ d$week)), residuals(lm(d$support_tickets ~ d$week)),
     pch = 19, col = "#9aa0a6", xlab = "spend, trend removed", ylab = "tickets, trend removed",
     main = sprintf("r = %.3f (p = %.2f)", pc_week$estimate, pc_week$p.value))
abline(h = 0, v = 0, col = "#ccc")

par(op)

0.4613 → 0.0637 (p = 0.520). Nothing left.

Results

res <- data.frame(
  question = c("Does spend move deals?",
               "…holding sessions fixed",
               "Do tickets track spend?",
               "…permutation test",
               "…holding week fixed"),
  statistic = c(sprintf("r = %.4f", ct_headline$estimate),
                sprintf("r = %.4f", pc_sessions$estimate),
                sprintf("r = %.4f", ct_drift$estimate),
                sprintf("%d / 20000 shuffles", n_extreme),
                sprintf("r = %.4f", pc_week$estimate)),
  p = c(signif(ct_headline$p.value, 4), round(pc_sessions$p.value, 4),
        signif(ct_drift$p.value, 4), signif(p_perm, 4), round(pc_week$p.value, 4)),
  verdict = c("strong", "gone", "moderate", "'confirmed'", "gone")
)
knitr::kable(res, caption = "Every quotable figure in this review")
Every quotable figure in this review
question statistic p verdict
Does spend move deals? r = 0.7848 0.0000000 strong
…holding sessions fixed r = -0.0283 0.7752000 gone
Do tickets track spend? r = 0.4613 0.0000008 moderate
…permutation test 0 / 20000 shuffles 0.0000500 ‘confirmed’
…holding week fixed r = 0.0637 0.5204000 gone

What the data has to look like

standard_correlation takes one row per period with each metric in its own numeric column, and a column_mapping of feature_1 … feature_12. This file is directly consumable: five numeric features plus the week index.

The constraint that matters: partial correlation can only control for a column you actually collected. If the weekly export had carried spend and deals but not sessions, nothing in this document could have been run — the analysis would have reported r = 0.78 and the budget would have gone up. Log the mediating steps of your funnel, and log the date, or you cannot ask either question.

Decision

  1. Raising spend does not close deals directly. The lever is sessions; spend is one of several ways to buy them. Test the cost per session against other channels before assuming the spend line is the constraint.
  2. The ticket correlation is retired, and no headcount decision should reference it.
  3. A significance test cannot rescue a badly posed question. The permutation test was run correctly and returned the wrong conclusion, because its assumption of exchangeable weeks was false. Time is the first third variable to test on any series collected in order.