User 136 · Research · Multiple Comparisons · Holm Bonferroni
Executive Summary

Executive Summary

Holm-Bonferroni correction results across all pairwise group comparisons

Total Students
1000
Education Groups
6
Total Pairwise Tests
45
Significant After Correction
19
Pairs Significant All Outcomes
5
19 of 45 pairwise comparisons (42%) remain statistically significant after Holm-Bonferroni correction at alpha = 0.05. The analysis compared 6 parental education levels across three student outcomes (math, reading, and writing scores), testing 15 unique group pairs per outcome. 5 group pair(s) show significance consistently across all outcomes, indicating robust education-level gaps that survive rigorous family-wise error control.
Interpretation

19 of 45 pairwise comparisons (42%) remain statistically significant after Holm-Bonferroni correction at alpha = 0.05. The analysis compared 6 parental education levels across three student outcomes (math, reading, and writing scores), testing 15 unique group pairs per outcome. 5 group pair(s) show significance consistently across all outcomes, indicating robust education-level gaps that survive rigorous family-wise error control.

Data Table

Group Descriptive Statistics

Mean scores and sample sizes per parental education level

NGroupOutcome 1 MeanOutcome 2 MeanOutcome 3 Mean
222associate's degree67.970.969.9
118bachelor's degree69.47373.4
196high school62.164.762.4
59master's degree69.775.475.7
226some college67.169.568.8
179some high school63.566.964.9
Interpretation

Across 6 parental education groups, the highest-scoring group on the first outcome is 'master's degree' and the lowest is 'high school', a raw gap of 7.6 points before significance testing. Sample sizes are roughly balanced across groups, supporting reliable t-test inference. Groups higher in educational attainment (e.g., master's or bachelor's degree) tend to have the largest mean scores, while 'some high school' or 'high school' groups typically score lowest.

Visualization

Mean Scores by Parental Education Level

Grouped mean scores for each education level across all outcomes

Interpretation

'master's degree' students score highest and 'high school' students score lowest on average across outcomes. The education-level ranking is highly consistent across all three outcomes. These raw differences motivate the pairwise testing: we need Holm-Bonferroni correction to determine which gaps are statistically reliable rather than sampling noise.

Visualization

Score Distributions by Education Level

Box plots of score distributions showing within-group spread

Interpretation

Each education group shows substantial within-group variability: the median interquartile range on the first outcome is approximately 19.6 points. This overlap in distributions means many pairwise differences are not detectable at the individual level, which is why formal significance testing — and then Holm-Bonferroni correction — is needed to avoid over-claiming. Groups with wider distributions require larger true mean differences to achieve significance.

Data Table

Pairwise Comparison Results (All 45 Tests)

All pairwise Welch t-test results with Holm-adjusted p-values (method = 'holm')

Group 1Group 2Adj PvalueRaw PvalueSignificantOutcome Name
bachelor's degreehigh school00YesOutcome 2
high schoolmaster's degree00YesOutcome 2
associate's degreehigh school00YesOutcome 3
bachelor's degreehigh school00YesOutcome 3
high schoolmaster's degree00YesOutcome 3
bachelor's degreesome high school0.00010YesOutcome 3
master's degreesome high school0.00010YesOutcome 3
associate's degreehigh school0.00030YesOutcome 2
high schoolsome college0.00030YesOutcome 3
bachelor's degreehigh school0.00130YesOutcome 1
associate's degreehigh school0.00310.0001YesOutcome 1
master's degreesome high school0.00470.0001YesOutcome 2
high schoolsome college0.01460.0004YesOutcome 1
high schoolsome college0.01930.0006YesOutcome 2
bachelor's degreesome high school0.01950.0006YesOutcome 2
high schoolmaster's degree0.02870.001YesOutcome 1
associate's degreesome high school0.0310.0011YesOutcome 3
master's degreesome college0.03340.0012YesOutcome 3
bachelor's degreesome high school0.03660.0014YesOutcome 1
master's degreesome college0.11330.0044NoOutcome 2
associate's degreesome high school0.13270.0053NoOutcome 1
associate's degreemaster's degree0.13270.0054NoOutcome 3
master's degreesome high school0.17250.0078NoOutcome 1
associate's degreesome high school0.17250.0076NoOutcome 2
bachelor's degreesome college0.17250.0075NoOutcome 3
some collegesome high school0.21620.0108NoOutcome 3
some collegesome high school0.3370.0177NoOutcome 1
associate's degreemaster's degree0.52630.0303NoOutcome 2
bachelor's degreesome college0.52630.0292NoOutcome 2
associate's degreebachelor's degree0.59260.037NoOutcome 3
associate's degreebachelor's degree10.3788NoOutcome 1
associate's degreemaster's degree10.4032NoOutcome 1
associate's degreesome college10.5878NoOutcome 1
bachelor's degreemaster's degree10.8826NoOutcome 1
bachelor's degreesome college10.1778NoOutcome 1
high schoolsome high school10.3901NoOutcome 1
master's degreesome college10.2354NoOutcome 1
associate's degreebachelor's degree10.1997NoOutcome 2
associate's degreesome college10.2665NoOutcome 2
bachelor's degreemaster's degree10.2881NoOutcome 2
high schoolsome high school10.1465NoOutcome 2
some collegesome high school10.0909NoOutcome 2
associate's degreesome college10.4465NoOutcome 3
bachelor's degreemaster's degree10.308NoOutcome 3
high schoolsome high school10.1159NoOutcome 3
Interpretation

19 of 45 pairwise comparisons survive Holm-Bonferroni correction at alpha = 0.05. The smallest adjusted p-value is 0, indicating the most robustly significant group difference. The largest raw p-value is 0.883. Comparing raw vs adjusted p-values reveals how much the correction inflates thresholds for lower-ranked tests — pairs near the boundary are most affected.

Visualization

Significant Pairwise Differences After Correction

Mean score gaps for group pairs that survive Holm-Bonferroni correction

Interpretation

19 pairwise comparisons remain significant after Holm-Bonferroni correction. The largest gap is between 'high school vs master's degree' at 13.23 points — a practically meaningful education-level effect. Bars are ordered by absolute mean difference so the most impactful gaps appear first, distinguishing statistical significance from effect magnitude.

Visualization

Adjusted P-Value Significance Map

Average Holm-adjusted p-value per group pair across all outcomes (cells below 0.05 are significant)

Interpretation

5 of 15 unique group pairs show an average adjusted p-value below the 0.05 threshold across all outcomes. Darker cells (lower p-values) indicate pairs where education-level differences are most consistently detectable. Pairs involving the highest and lowest education levels tend to produce the strongest signal, while adjacent levels often do not differ significantly after correction.

Your data has more stories to tell. Run any analysis on your own data — 60+ validated R modules, interactive reports, AI insights, and PDF export. 2,000 free credits on signup.
Try Free — No Signup Sign Up Free

Report an Issue

Tell us what's wrong. You'll get a free re-run of this analysis so you can try again with different parameters. If the re-run still doesn't meet your expectations, we'll refund your credits.

Want to run this analysis on your own data? Upload CSV — Free Analysis See Pricing