ds@mcpanalytics.aiEmail your data and your question and get an answer about your own data, with the method shown.Email your data and question. Get an answer.
Browse analyses by question | MCP Analytics
Browse by question

What are you actually asking of the data?

Updated November 2026

Compare two groups? Find what drives an outcome? Spot a trend? The question decides which statistics fit, so it is the first thing to get right.

See a worked example

How the analyses here work

Not yet covered

Nothing prebuilt is filed under these yet. The analyses above may still cover them.

Study design
Worked example

Did either new checkout design convert better than the current one?

The data. standard_ab_test_v2.csv, the published sample file for A/B test: 4,795 rows; columns User ID, Assigned Group, Converted, Order Value, Signup Date, Device. Sample data, not a customer's.

The calculation. Python, standard library only. This exact code ran against the file when this page was built, and printed the output below it.

import csv
rows = list(csv.DictReader(open("standard_ab_test_v2.csv")))
n, won = {}, {}
for r in rows:
    g, c = r["Assigned Group"], r["Converted"]
    if not g or c not in ("Yes", "No"):
        continue  # no group or no outcome recorded
    n[g] = n.get(g, 0) + 1
    won[g] = won.get(g, 0) + (c == "Yes")
rate = {g: won[g] / n[g] for g in n}
for g in sorted(n):
    print(g, n[g], "users,", won[g], "converted,", f"{rate[g]:.2%}")
best = max((g for g in n if g != "Control"), key=rate.get)
d = rate[best] - rate["Control"]
se = (rate[best] * (1 - rate[best]) / n[best]
      + rate["Control"] * (1 - rate["Control"]) / n["Control"]) ** 0.5
print(best, "minus Control:", f"{100 * d:+.2f}", "points, 95% interval",
      f"{100 * (d - 1.96 * se):+.2f} to {100 * (d + 1.96 * se):+.2f}")
print("skipped", len(rows) - sum(n.values()), "incomplete rows")
Control 1601 users, 78 converted, 4.87%
Variant B 1584 users, 116 converted, 7.32%
Variant C 1595 users, 85 converted, 5.33%
Variant B minus Control: +2.45 points, 95% interval +0.79 to +4.11
skipped 15 incomplete rows

What it says. Variant B converted 7.32% of users against 4.87% for Control, a difference of 2.45 percentage points. The 95% interval (0.79 to 4.11) stays above zero, so in this sample the lift is unlikely to be noise alone.

What it does not say. 15 rows had no group or no outcome and were skipped, not guessed. This is the simple two-proportion interval for the better of two variants, picked after looking, so it is a little optimistic; the library analysis adjusts every comparison with the Holm method and also checks that traffic was split as planned.

Run A/B test on this file or your own, or open its stored sample report.

Sources

Further reading on the method in the worked example. Each reference was looked up and its authors, title, year and venue confirmed between 27 October 2026 and 30 October 2026.

  1. Wilson EB (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.1927.10502953
  2. Wasserstein R, Lazar N (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician. https://doi.org/10.1080/00031305.2016.1154108

By MCP Analytics Team. Updated November 2026. Analysis list from the library catalog.