Paired Comparisons: Before/After Tests Done Right

A manufacturing quality team needed to test whether a new calibration procedure improved measurement accuracy. They could have recruited 100 machines for a standard two-group comparison. Instead, they measured the same 25 machines before and after recalibration, using a paired design. The result? They detected a statistically significant improvement with 75% fewer test subjects, cutting experiment costs from $42,000 to $11,500. That's the power of proper pairing—and why getting the analysis right matters.

Most before/after analyses get analyzed wrong. Researchers collect paired data—same subjects measured twice, matched case-control pairs, left-right comparisons—then analyze it using independent samples methods. This throws away the most valuable information in the dataset: the correlation structure between measurements. The cost? Reduced statistical power, inflated sample size requirements, and missed opportunities to detect real effects.

This guide covers when to use paired comparisons, how to run them correctly, and the three critical mistakes that invalidate your results. We'll work through a complete example with real numbers, discuss paired t-test vs Wilcoxon signed-rank selection, and calculate the ROI of proper experimental design.

What Makes Data "Paired" vs "Independent"

Before we discuss methodology, let's clarify what constitutes paired data. This isn't about having two groups—it's about the relationship between observations.

Paired data requirements:

• Each observation in group A has exactly one corresponding observation in group B
• The pairing is established before data collection, not after
• The paired observations share something that creates correlation (same subject, matched characteristics, spatial proximity)
• You expect that if observation #1 in group A is high, observation #1 in group B is also likely to be high (or low, for negative correlation)

Common paired scenarios:

Before/after on same subjects: Blood pressure measured before and after medication in the same 30 patients. Each patient serves as their own control.

Matched case-control: Each cancer patient matched with a healthy control of the same age, sex, and smoking status. The matching creates the pairing.

Bilateral measurements: Intraocular pressure measured in left and right eyes of the same individuals. Eyes within a person are more similar than eyes across people.

Spatial or temporal proximity: Customer satisfaction ratings at two time points for the same stores, or sales before/after a promotion at the same retail locations.

The opposite is independent samples: two groups with no meaningful connection between specific observations. Drug group vs placebo group with different patients in each. Men vs women. Store type A vs store type B with different locations in each group. Here, knowing that observation #3 in group A is high tells you nothing about observation #3 in group B.

Why does this matter? Because paired data analysis uses the difference within each pair. This eliminates between-subject variability. If patient #7 has naturally high blood pressure, that individual baseline gets subtracted out when you compute (after - before). What remains is the signal you care about: the treatment effect.

The Cost Math: Why Paired Designs Save Money

Let's work through the sample size economics with concrete numbers. Suppose you're testing whether a training program improves employee productivity, measured by widgets produced per hour. Historical data shows a standard deviation of 12 widgets/hour. You want to detect a 5 widget/hour improvement with 80% power at α = 0.05.

Option 1: Independent samples design

Recruit one group for training, one control group with no training. Required sample size per group:

n = 2 * (Z_α/2 + Z_β)² * σ² / δ²
n = 2 * (1.96 + 0.84)² * 12² / 5²
n = 2 * 7.84 * 144 / 25
n = 90.3 ≈ 91 per group
Total subjects needed: 182

If each participant costs $500 to recruit and train (or not train), your experiment costs $91,000.

Option 2: Paired design (before/after)

Measure the same workers before and after training. Now you're analyzing differences, and the key parameter is the standard deviation of the differences (σ_d). If the before/after correlation is 0.6—reasonable for measurements on the same people—then:

σ_d = σ * √(2(1 - ρ))
σ_d = 12 * √(2(1 - 0.6))
σ_d = 12 * √0.8
σ_d = 10.7

n = (Z_α/2 + Z_β)² * σ_d² / δ²
n = (1.96 + 0.84)² * 10.7² / 5²
n = 7.84 * 114.5 / 25
n = 35.9 ≈ 36 pairs

You need 36 workers measured twice. If the "before" measurement is free (baseline productivity data you already have) and you only pay the $500 training cost per worker, your experiment costs $18,000. That's an $73,000 savings—80% cost reduction—for the same statistical power.

Even if the before measurement isn't free, you're comparing 36 workers measured twice (72 measurements) to 182 workers measured once (182 measurements). The paired design still requires 60% fewer measurement occasions.

The savings scale with correlation strength:

• ρ = 0.3: Paired design needs ~60% as many subjects as independent
• ρ = 0.5: Paired design needs ~50% as many subjects
• ρ = 0.7: Paired design needs ~35% as many subjects
• ρ = 0.9: Paired design needs ~20% as many subjects

This is why paired designs dominate in expensive domains: clinical trials ($5,000-$50,000 per patient), industrial testing (machinery downtime costs), and anywhere subject recruitment is the limiting factor.

The Paired T-Test: Mechanics and Assumptions

Now let's look at how the paired t-test actually works. The procedure is surprisingly simple—simpler than an independent samples t-test—because you reduce the two-sample problem to a one-sample problem.

Step 1: Calculate differences

For each pair i, compute d_i = X_i - Y_i (or Y_i - X_i; the sign just affects interpretation). These differences become your data vector.

Step 2: Test if mean difference equals zero

Compute the mean difference d̄ and standard deviation of differences s_d. The test statistic is:

t = d̄ / (s_d / √n)

This follows a t-distribution with n-1 degrees of freedom under the null hypothesis that the true mean difference is zero.

Step 3: Compare to critical value

If |t| > t_critical (from t-table at α = 0.05 and df = n-1), reject the null hypothesis. Alternatively, compute the p-value as P(|T| > |t_observed|).

That's it. You've converted a two-sample problem into a one-sample t-test on the differences. The entire analysis focuses on whether the average difference is distinguishable from zero.

Assumptions you're making:

1. The differences are independent: Knowing the difference for pair #1 doesn't tell you about the difference for pair #2. This is usually satisfied if you sampled subjects randomly and didn't pair subjects with each other (only within-subject measurements).
2. The differences are approximately normally distributed: This matters most for small samples (n < 30). With larger samples, the Central Limit Theorem makes the test robust to moderate non-normality.
3. The pairing is appropriate: Each observation in group A truly corresponds to exactly one observation in group B, and the pairing reduces variance rather than introducing irrelevant noise.

Notice what's not required: You don't need the original measurements (X and Y) to be normal—only the differences. You don't need equal variances between groups. You don't need large samples if the differences are normal. These are major relaxations compared to independent samples tests.

Worked Example: Blood Pressure Reduction

A cardiologist wants to test whether a new medication reduces systolic blood pressure. She recruits 20 hypertensive patients, measures baseline blood pressure, administers the medication for 30 days, then measures again. Here's the data (mmHg):

Patient   Before   After   Difference
1         142      138     -4
2         156      149     -7
3         168      165     -3
4         151      145     -6
5         149      148     -1
6         163      152     -11
7         144      140     -4
8         159      151     -8
9         147      143     -4
10        155      150     -5
11        161      158     -3
12        152      149     -3
13        158      153     -5
14        146      144     -2
15        164      157     -7
16        150      146     -4
17        157      154     -3
18        162      156     -6
19        148      145     -3
20        153      148     -5

Step 1: Calculate summary statistics for differences

Mean difference (d̄) = -4.6 mmHg
Standard deviation (s_d) = 2.28 mmHg
Sample size (n) = 20 pairs

Step 2: Calculate t-statistic

SE = s_d / √n = 2.28 / √20 = 2.28 / 4.47 = 0.51
t = d̄ / SE = -4.6 / 0.51 = -9.02
df = n - 1 = 19

Step 3: Determine significance

For a two-tailed test at α = 0.05 with 19 degrees of freedom, the critical value is ±2.093. Our observed |t| = 9.02 far exceeds this threshold. The p-value is < 0.0001.

Step 4: Construct confidence interval

The 95% confidence interval for the mean difference is:

CI = d̄ ± t_critical * SE
CI = -4.6 ± 2.093 * 0.51
CI = -4.6 ± 1.07
CI = [-5.67, -3.53] mmHg

Interpretation: The medication reduces systolic blood pressure by an average of 4.6 mmHg (95% CI: 3.5 to 5.7 mmHg reduction). This difference is statistically significant (t = -9.02, p < 0.0001) and clinically meaningful—a 4-5 mmHg reduction reduces cardiovascular event risk by approximately 10%.

What if we'd analyzed this wrong? Suppose we ignored the pairing and ran an independent samples t-test comparing the 20 "before" measurements to the 20 "after" measurements. Let's check the numbers:

Before group: mean = 154.3, SD = 7.02
After group: mean = 149.7, SD = 6.21
Pooled SD = 6.63

t_independent = (154.3 - 149.7) / (6.63 * √(2/20))
t_independent = 4.6 / (6.63 * 0.316)
t_independent = 4.6 / 2.10
t_independent = 2.19, p = 0.036

We'd still get a significant result (barely), but look at what happened: The t-statistic dropped from 9.02 to 2.19. The p-value increased from <0.0001 to 0.036—crossing the common threshold for publication. The confidence interval would widen from ±1.07 to ±4.40 mmHg. We'd lose precision and power because we're including all the between-subject variability (people have naturally different blood pressures) instead of focusing only on within-subject changes.

When Paired T-Test Assumptions Fail: Enter Wilcoxon

The paired t-test assumes the differences follow a normal distribution. Check this assumption before you run the test. With small samples (n < 15), even moderate departures from normality can inflate Type I error rates. With larger samples (n > 30), the test is robust to non-normality due to the Central Limit Theorem.

How to check normality of differences:

• Visual: Q-Q plot of differences. Points should fall roughly on a straight line.
• Statistical: Shapiro-Wilk test on differences. Non-significant result (p > 0.05) suggests normality is reasonable.
• Descriptive: Skewness should be between -1 and +1; kurtosis between -2 and +2 for approximate normality.

When to use Wilcoxon signed-rank instead:

1. Outliers in differences: One or two pairs show extreme differences that don't reflect measurement error but could dominate the mean. Example: In a reaction time experiment, most differences are 20-50ms but two subjects show 200ms differences due to distraction.
2. Skewed differences: You're measuring change in a variable that's inherently skewed (income, medical costs, time-to-event). The differences inherit this skewness.
3. Ordinal data: Your measurements are rankings or Likert scales. You can compute differences (rating increased by 2 points) but the underlying scale isn't truly interval.
4. Small sample with non-normality: You have n < 15 and the Shapiro-Wilk test rejects normality (p < 0.05), or the Q-Q plot shows clear deviation from a straight line.

The Wilcoxon signed-rank test ranks the absolute differences, then tests whether the sum of ranks for positive differences differs from the sum for negative differences. It's robust to outliers (ranks cap the influence of extreme values) and makes no distributional assumptions beyond symmetry.

The cost of switching to Wilcoxon: If the differences truly are normal, the Wilcoxon test has efficiency of about 95% relative to the paired t-test. You'll need roughly 5% more pairs to achieve the same power. That's a small price for robustness.

The benefit of switching when needed: If you have outliers or severe skewness, the paired t-test can have inflated Type I error (you reject the null too often when it's true) or reduced power. The Wilcoxon test maintains correct error rates regardless of distribution shape.

As a practical rule: For n ≥ 30, start with the paired t-test unless you have severe outliers. For n < 30, check the Q-Q plot and run both tests. If they agree, report the paired t-test (more familiar to most readers). If they disagree, report Wilcoxon and note that you observed non-normality in the differences.

Three Critical Mistakes That Invalidate Paired Comparisons

Before we discuss applications, let's address the three most common errors that produce misleading results. I see these mistakes in published papers, internal reports, and graduate student analyses. Each one undermines the validity of your conclusions.

Mistake #1: Breaking the Pairing in Analysis

You collected paired data but analyzed it as independent samples. We showed this in the blood pressure example—you lose power and precision. But there's a more subtle version of this error: You started with paired data but lost some pairs to missing values, then tried to "salvage" the unpaired observations.

Example: You measured 50 patients before and after treatment. Five patients dropped out after baseline measurement, so you only have "after" data for them. Three patients missed their baseline visit but completed the follow-up, so you only have "before" data for them.

The wrong approach: Analyze all 50 "before" measurements and all 50 "after" measurements using an independent samples test. This breaks the pairing for the 42 patients with complete data.

The correct approach: Analyze only the 42 complete pairs using a paired test. Exclude the 8 patients with incomplete data. Yes, you lose some data. But mixing paired and unpaired observations creates a statistical mess with no valid analysis method.

If you lose more than 20% of pairs to missing data, investigate the missingness pattern. Is it random (equipment failure, scheduling conflicts) or systematic (sicker patients can't complete follow-up)? Systematic missingness can bias your results. For example, if only healthier patients complete both measurements, your estimated treatment effect may be overly optimistic.

Mistake #2: Wrong Pairing Structure

You declared observations "paired" when they aren't naturally paired, or you paired based on post-hoc data patterns rather than experimental design.

Example: You have customer satisfaction scores for 30 stores before a renovation and 30 different stores after a renovation. You sort both groups by baseline sales volume and declare the top store in the "before" group "paired" with the top store in the "after" group. This is not a valid pairing—these are independent samples that you artificially aligned.

Valid pairing must be established before data collection based on a real connection between observations. Legitimate pairing sources:

• Same subject measured twice (repeated measures)
• Matched case-control based on confounding variables (age, sex, disease severity matched before group assignment)
• Natural bilateral or spatial pairing (left/right eye, twin siblings, adjacent fields in an agricultural trial)
• Temporal or spatial blocking in an experiment (measurements taken on the same day, plots in the same field section)

If you can't explain why pair #5 in group A should be compared specifically to pair #5 in group B based on experimental design, you don't have paired data.

Mistake #3: Ignoring Carryover Effects in Crossover Designs

A crossover design measures the same subjects under treatment A, then treatment B (or vice versa with randomization). This creates paired data—you compare each subject's response under A vs B. But if treatment A has lasting effects that persist into the B period, the comparison is contaminated.

Example: Testing two sleep medications, you give treatment A for one week, then treatment B for one week, measuring sleep quality each week. If treatment A causes residual drowsiness that persists into week 2, the week 2 measurement under treatment B is affected by both B and the residual effect of A.

How to handle this:

• Include a washout period: Insert a delay between treatments to let effects dissipate. Minimum washout should be 5 half-lives of the treatment effect.
• Randomize order: Half the subjects get A then B; half get B then A. This balances carryover effects across groups.
• Test for period effects: Compare response in period 1 vs period 2 collapsed across treatments. A significant period effect suggests carryover or learning.
• Model the carryover explicitly: Use a mixed model that includes treatment, period, and sequence as factors. This requires sufficient sample size and a more complex analysis.

If you can't include an adequate washout period and you expect carryover effects, don't use a crossover design. Use a parallel groups design instead—different subjects receive different treatments, analyzed as independent samples. You'll need more subjects, but the analysis will be valid.

Sample Size Planning: The Right Way

Never run an experiment without calculating required sample size first. Underpowered experiments waste resources and produce ambiguous results. Here's how to plan properly for paired comparisons.

What you need to specify:

• Minimum detectable effect (δ): The smallest difference worth detecting. Base this on practical significance, not statistical significance. A blood pressure medication that reduces SBP by 1 mmHg might be statistically detectable with 1,000 patients, but it's clinically meaningless.
• Standard deviation of differences (σ_d): Estimate this from pilot data or published studies on similar measurements. If you only know the SD of individual measurements (σ), estimate σ_d = σ * √(2(1-ρ)), where ρ is the expected correlation between paired measurements.
• Significance level (α): Usually 0.05 for two-tailed tests. Use 0.01 if you're conducting multiple comparisons or if false positives are especially costly.
• Desired power (1-β): Usually 0.80 (80% power). Use 0.90 if missing a true effect is very costly. Never go below 0.70—underpowered studies are ethically questionable if they involve patients or expensive resources.

The formula:

n = (Z_α/2 + Z_β)² * σ_d² / δ²

Where:
Z_α/2 = 1.96 for α = 0.05 (two-tailed)
Z_β = 0.84 for power = 0.80
Z_β = 1.28 for power = 0.90

Worked example: You're testing whether a cognitive training program improves working memory span. Current mean span is 7 digits with SD = 2. You expect training to increase span by 1 digit (a medium effect). You estimate the before/after correlation at ρ = 0.6 based on test-retest reliability studies.

σ_d = 2 * √(2(1-0.6)) = 2 * √0.8 = 1.79
δ = 1
α = 0.05, so Z_α/2 = 1.96
Power = 0.80, so Z_β = 0.84

n = (1.96 + 0.84)² * 1.79² / 1²
n = 7.84 * 3.20 / 1
n = 25.1 ≈ 26 pairs

You need 26 subjects measured before and after training. If you'd used an independent samples design (separate training and control groups), you'd need:

n_per_group = 2 * (1.96 + 0.84)² * 2² / 1²
n_per_group = 2 * 7.84 * 4
n_per_group = 62.7 ≈ 63 per group
Total: 126 subjects

The paired design reduces required subjects from 126 to 26—an 80% reduction. If subject recruitment costs $200 per person, that's a $20,000 savings.

Sensitivity to correlation: That 80% reduction assumes ρ = 0.6. Let's see how sample size changes with correlation:

• ρ = 0.2: n = 40 pairs (vs 126 independent)
• ρ = 0.4: n = 32 pairs
• ρ = 0.6: n = 26 pairs
• ρ = 0.8: n = 20 pairs

Higher correlation = greater efficiency. But even with low correlation (ρ = 0.2), the paired design still needs 68% fewer subjects than independent samples. The pairing is almost always worth it when you have natural pairs.

Paired Comparisons Beyond Two Time Points

Everything so far assumes you have exactly two measurements per subject: before and after, treatment A and treatment B, left and right. What if you have three or more time points? The paired t-test doesn't extend directly, but you have options.

Option 1: Repeated Measures ANOVA

If you have the same subjects measured at three or more time points, use repeated measures ANOVA. This generalizes the paired t-test to multiple measurements while accounting for within-subject correlation. The null hypothesis is that all time points have the same mean.

Example: You measure anxiety scores at baseline, 1 month, 3 months, and 6 months after therapy starts. Repeated measures ANOVA tests whether anxiety changes over time, accounting for the fact that measurements on the same person are correlated.

Assumptions: Sphericity (the variance of differences between all pairs of time points is equal). If violated, apply Greenhouse-Geisser or Huynh-Feldt corrections. For more flexible modeling, consider mixed effects models, which handle missing data better and don't require sphericity.

Option 2: Multiple Paired T-Tests with Correction

If you have specific pairwise comparisons in mind (e.g., baseline vs 1 month, baseline vs 3 months, baseline vs 6 months), run separate paired t-tests and correct for multiple comparisons using Bonferroni or Holm-Bonferroni methods.

Example: Testing 3 comparisons with α = 0.05, Bonferroni correction requires each test to be significant at α/3 = 0.017. This controls family-wise error rate at 0.05.

Caveat: Bonferroni can be conservative (low power) when testing many comparisons. For exploratory analyses, consider the false discovery rate (FDR) approach instead.

Option 3: Analyze Change Scores Directly

If you care specifically about change from baseline, compute change scores for each follow-up time (Month 1 - Baseline, Month 3 - Baseline, Month 6 - Baseline) and analyze those using ANOVA or regression. This approach focuses on "How much did subjects improve?" rather than "Are time points different?"

This is particularly useful when baseline values vary widely across subjects and you want to control for baseline in the analysis. You can run a regression with change score as the outcome and baseline as a covariate, testing whether change differs from zero while adjusting for initial level.

Effect Size Reporting: Making Results Interpretable

Statistical significance tells you whether an effect exists. Effect size tells you whether it matters. Always report both for paired comparisons.

Cohen's d for paired data:

d = d̄ / s_d

Where d̄ is the mean difference and s_d is the standard deviation of differences. This measures the standardized mean difference in units of "standard deviations of change."

Interpretation (Cohen's guidelines):

• d = 0.2: Small effect
• d = 0.5: Medium effect
• d = 0.8: Large effect

In our blood pressure example: d = 4.6 / 2.28 = 2.02. That's a very large effect—the medication reduces blood pressure by about 2 standard deviations of the typical person-to-person variation in response.

Correlation as effect size:

The correlation between before and after measurements tells you how much variance the pairing explains. High correlation (ρ > 0.7) means individuals maintain their relative ranking—those high at baseline tend to be high after treatment. Low correlation (ρ < 0.3) means individuals move around substantially in the ranking.

You can compute this from the differences:

ρ = (σ² - 0.5 * σ_d²) / σ²

Where σ is the standard deviation of individual measurements and σ_d is the SD of differences. Or just compute it directly as cor(before, after) from your data.

In the blood pressure example, if we calculate the correlation between before and after measurements, we get ρ = 0.91. That's very high—patients with high baseline BP tend to have high follow-up BP, even after the medication reduces everyone's pressure. This high correlation is exactly why the paired design is so much more powerful than independent samples.

Percentage change:

For outcomes where relative change is more meaningful than absolute change (prices, medical costs, enzyme activity), report percentage change:

% change = (After - Before) / Before * 100%

You can compute this for each pair, then test whether the mean percentage change differs from zero. Be cautious with this approach if "Before" values are near zero—percentage changes become unstable.

When Paired Comparisons Answer the Wrong Question

Paired designs are powerful, but they're not always appropriate. Here are scenarios where pairing leads you astray.

When You Care About Population-Level Differences

Paired tests answer: "Do individuals change?" Independent samples tests answer: "Do groups differ?" These aren't always the same question.

Example: You want to know if men and women have different risk tolerance. You can't "pair" men and women (each person has one sex), so paired comparison is impossible. You need an independent samples test comparing the male group to the female group.

Even when pairing is technically possible, it might not answer your question. Suppose you want to compare customer satisfaction at Starbucks vs Dunkin' Donuts. You could recruit customers who visit both chains and measure satisfaction at each—creating paired data. But this restricts your sample to "chain switchers" who might not represent typical customers of either brand. An independent samples comparison of separate Starbucks and Dunkin' customers might better answer the business question.

When Pairing Introduces Confounds

In crossover designs, the "pairing" comes from measuring the same subject under both treatments. But if receiving treatment A first affects how subjects respond to treatment B, you've introduced an order effect that confounds the treatment comparison.

Example: Testing two teaching methods by having students learn topic A with method 1, then topic B with method 2. The pairing (same students) is contaminated by learning effects—students might do better on topic B simply because they've had more practice, regardless of which method is better.

Solution: Use randomized parallel groups (different students for each method) or a crossover design with proper counterbalancing and washout periods.

When Individual Differences Are the Signal

Paired designs eliminate between-subject variance to increase power for detecting treatment effects. But if your research question is about individual differences, pairing removes exactly what you want to study.

Example: You want to identify which patient characteristics predict treatment response. A paired before/after design tells you whether the average patient improves, but it doesn't help you understand why some patients improve more than others. For that, you need between-subjects variation: analyze the relationship between patient characteristics and the amount of change they experience.

Paired designs are for questions about average effects. Individual differences questions require keeping between-subjects variation in the analysis, often through regression or mixed models with individual-level predictors.

Software Implementation: Making It Reproducible

Running paired comparisons in statistical software is straightforward. Here's how to do it correctly in R and Python.

R implementation:

# Paired t-test
t.test(after, before, paired = TRUE)

# Or equivalently, on the differences
differences <- after - before
t.test(differences, mu = 0)

# Wilcoxon signed-rank test
wilcox.test(after, before, paired = TRUE)

# Effect size (Cohen's d)
library(effsize)
cohen.d(after, before, paired = TRUE)

Python implementation:

from scipy import stats
import numpy as np

# Paired t-test
t_stat, p_value = stats.ttest_rel(after, before)

# Wilcoxon signed-rank test
w_stat, p_value = stats.wilcoxon(after, before)

# Cohen's d
differences = after - before
d = np.mean(differences) / np.std(differences, ddof=1)

Common implementation errors:

1. Forgetting the paired=TRUE flag: In R, t.test(after, before) without paired=TRUE runs an independent samples test. You must specify pairing explicitly.
2. Mismatched vector lengths: If "after" has 25 observations but "before" has 24, you've lost a pair somewhere. The software will either error or silently drop one observation—both are bad. Fix your data to include only complete pairs.
3. Reversed subtraction: Computing "before - after" vs "after - before" flips the sign of your t-statistic and mean difference. This doesn't affect significance testing (the p-value is the same), but it affects interpretation. Be clear about the direction.
4. Ignoring assumptions: The software runs the test whether or not your data meet assumptions. Check normality of differences with a Q-Q plot or Shapiro-Wilk test before trusting the paired t-test results.

For full reproducibility, include: random seed if you're doing permutation tests or bootstrapping, exact software version, and the complete dataset or a link to it. Results from t.test() on 20 pairs with R version 4.3.0 should match exactly across machines—make sure they do by documenting your environment.

FAQ: Paired Comparisons in Practice

When should I use a paired t-test instead of an independent samples t-test?

Use a paired t-test when each observation in one group has a natural pairing with exactly one observation in the other group. Common scenarios include before/after measurements on the same subjects, matched case-control studies, or measurements on left vs right (eyes, hands, kidneys). The key is that the pairing reduces variance by controlling for individual differences, making your test more powerful.

What's the difference between paired t-test and Wilcoxon signed-rank test?

The paired t-test assumes the differences between pairs follow a normal distribution and uses the mean difference. The Wilcoxon signed-rank test makes no distribution assumptions and uses ranks instead of raw values, making it robust to outliers and skewed data. If your differences are approximately normal, use the paired t-test for maximum power. If you have outliers, severe skewness, or ordinal data, use Wilcoxon.

How many pairs do I need for a paired t-test?

This depends on your expected effect size and desired power. For a medium effect (d=0.5) with 80% power at α=0.05, you need approximately 34 pairs. For a large effect (d=0.8), you need about 15 pairs. For small effects (d=0.2), you need over 200 pairs. Always calculate required sample size before running your experiment—underpowered tests waste resources and produce unreliable results.

What if my paired differences aren't normally distributed?

First, check how severe the non-normality is. The paired t-test is robust to moderate violations with sample sizes above 30 due to the Central Limit Theorem. If you have severe skewness, heavy outliers, or small samples (n<15), switch to the Wilcoxon signed-rank test. Alternatively, consider transforming your data (log, square root) if that makes theoretical sense for your measurement.

Can I use paired comparisons if I have some missing pairs?

Yes, but only complete pairs contribute to the analysis—you must exclude any observation where either the before or after measurement is missing. If you lose more than 20% of your pairs to missingness, investigate whether the missing data pattern is random or systematic. Systematic missingness (e.g., sicker patients drop out) can bias your results. The paired t-test uses only complete pairs, which reduces your effective sample size and statistical power.

Final Recommendations: Design First, Test Second

The most important decision about paired comparisons happens before you collect a single data point: recognizing when you have naturally paired observations and designing your experiment to preserve that structure.

If you have the option to measure the same subjects before and after an intervention, take it. The statistical efficiency gain—typically 40-70% fewer required subjects for the same power—translates directly to cost savings, faster completion, and more precise estimates. This matters whether you're running a clinical trial at $10,000 per patient, an industrial experiment with expensive downtime, or a customer experience test where recruitment is slow.

But pairing only helps if you do it right. Calculate required sample size before you start. Verify that your pairing structure makes sense. Check assumptions before trusting test results. And use the paired analysis method—don't throw away the pairing correlation by running an independent samples test.

When you set up the experiment correctly and analyze it appropriately, paired comparisons give you the most precise answer per dollar spent. That's the difference between an adequately powered study that detects real effects and an underpowered study that burns resources for ambiguous results.

Before you collect your next before/after dataset, ask yourself: Did I calculate the required sample size? Will I have adequate power? Did I plan how to handle missing data? If you can't answer yes to all three, stop and plan the study properly. The time you spend on design saves multiples of that time in data collection and prevents the nightmare scenario of finishing data collection only to discover your study was underpowered all along.

Related Articles