ds@mcpanalytics.aiEmail your data and your question and get an answer about your own data, with the method shown.Email your data and question. Get an answer.

Employee Engagement Survey Analysis

By MCP Analytics Team | Published , updated

MCP Analytics · People Analytics

Your engagement survey came back and Engineering scored 3.8 while Sales scored 3.4. Is that a real gap that requires intervention, or is it noise from a small sample? Your survey vendor shows you averages and heatmaps but doesn't answer that question. This analysis applies proper statistical testing to your survey data (chi-square for response patterns, ANOVA for group comparisons, Tukey HSD for pairwise differences) and tells you which gaps are real, which are noise, and where to focus your limited intervention budget.

Worked Example (Simulated): Do Engagement Scores Differ Across Three Departments?

Question: Do simulated engagement scores differ across three departments?

Inputs: Simulated data, seed 2026. The script makes one score from 1 to 5 for each of 40 simulated responses in each of three departments. The score chances are set in the script. They build in a mean score of 3.70 for Engineering, 2.95 for Sales and 3.05 for Support, and shares of scores 1 to 2 of 15%, 35% and 30%. So Engineering is built about 0.75 points higher, and Sales and Support are built almost level.

Method: The Kruskal-Wallis rank test compares the score ranks across groups. Epsilon-squared gives a rank-based estimate of effect size.

import numpy as np
from scipy.stats import kruskal

seed = 2026
rng = np.random.default_rng(seed)
levels = np.arange(1, 6)
profiles = {
    "Engineering": [0.05, 0.10, 0.20, 0.40, 0.25],
    "Sales": [0.15, 0.20, 0.30, 0.25, 0.10],
    "Support": [0.10, 0.20, 0.35, 0.25, 0.10],
}
n_per_group = 40
scores = {
    name: rng.choice(levels, size=n_per_group, p=probabilities)
    for name, probabilities in profiles.items()
}
statistic, p_value = kruskal(*scores.values())
n_total = sum(len(values) for values in scores.values())
k = len(scores)
epsilon_squared = max(0.0, (statistic - k + 1) / (n_total - k))

print(f"Simulated data, seed {seed}")
print(f"Input: {k} departments, {n_per_group} simulated responses each; "
      "score scale 1 to 5")
print("Method: Kruskal-Wallis rank test; epsilon-squared effect size")
for name, values in scores.items():
    print(f"{name}: mean={values.mean():.2f}, median={np.median(values):.1f}, "
          f"score 1-2 share={(values <= 2).mean():.0%}")
p_text = "below 0.001" if p_value < 0.001 else format(p_value, ".3f")
print(f"Kruskal-Wallis H={statistic:.3f}, p={p_text}, "
      f"epsilon-squared={epsilon_squared:.3f}")
built_in_means = {name: float(levels @ np.array(probs))
                  for name, probs in profiles.items()}
print("Built-in mean score: "
      + ", ".join(f"{name} {value:.2f}" for name, value in built_in_means.items()))

Printed output:

Simulated data, seed 2026
Input: 3 departments, 40 simulated responses each; score scale 1 to 5
Method: Kruskal-Wallis rank test; epsilon-squared effect size
Engineering: mean=3.90, median=4.0, score 1-2 share=2%
Sales: mean=3.00, median=3.0, score 1-2 share=30%
Support: mean=2.95, median=3.0, score 1-2 share=30%
Kruskal-Wallis H=19.932, p=below 0.001, epsilon-squared=0.153
Built-in mean score: Engineering 3.70, Sales 2.95, Support 3.05

Result: The test gives evidence of a difference among the three simulated groups (H=19.932, p below 0.001), which is expected because Engineering was built higher. Epsilon-squared is 0.153. The drawn means were 3.90 for Engineering, 3.00 for Sales and 2.95 for Support. This draw ran high for Engineering: its drawn share of low scores (2%, 1 of 40 responses) is far under the 15% built in, where about 6 responses were expected, so treat that share as a chance draw and not a finding. These results describe only the made-up data. The test does not show why scores differ or whether an action would help.

Limit: The script sets the score chances, so this example cannot show how a real workforce would respond. It is not evidence about any real team.

Sources

The Problem with Averages

Yet most organizations analyze their survey results the same way: export the data, compute department averages in Excel, build a heatmap in PowerPoint, and present it to leadership as fact. The problem is that averages hide the signal. A department with an average score of 3.5 might have 60% of employees at 4-5 and 40% at 1-2: a polarized team with a very different problem than a department where everyone scores 3-4.

The bigger problem is treating every difference as meaningful. When your survey vendor shows Engineering at 3.8 and Sales at 3.4, that 0.4-point gap looks significant on a color-coded heatmap. But with 40 respondents in each group and Likert-scale data, that gap may not survive a statistical test. Without testing, HR teams allocate intervention budgets to phantom problems while real issues, masked by acceptable averages, go unaddressed.

Knowing which gaps are real and which are noise is what statistical testing provides.

What This Analysis Does

The worked example above uses simulated survey scores to show one test, its output, and one limit. It is not a result from a real survey. The three kinds of test below are the ones this page covers.

ANOVA (Analysis of Variance): compares the mean score across three or more groups (departments, tenure bands, job levels) and tells you whether at least one group is significantly different from the rest. The F-statistic and p-value tell you whether the between-group differences are larger than you'd expect from random variation. A small p-value is evidence against the null model of equal group means. It does not prove a gap is real or useful. Read it with the effect size, uncertainty, study design, sample size, and test assumptions.

Tukey HSD (Honestly Significant Difference): once ANOVA confirms that groups differ, Tukey tells you which specific pairs differ. Maybe it's Engineering vs. Support that drives the significant result, while Sales vs. Marketing is noise. Tukey tests every pair with proper correction for multiple comparisons, so you don't get false positives from testing many pairs.

Chi-square tests: examines whether the pattern of responses differs between groups. ANOVA compares means; chi-square compares distributions. It catches cases where two departments have the same average but completely different response patterns: one uniformly moderate, the other bimodally split between highly engaged and deeply disengaged.

Together, these three tests answer the question that separates data-informed HR from presentation-driven HR: are the differences in our engagement data real, or just noise?

When to Use This Analysis

Small groups may give uncertain estimates, so use the sample size guidance below for each comparison.

What Data Do You Need?

A CSV export from your survey platform (Culture Amp, Qualtrics, SurveyMonkey, Google Forms, 15Five, Lattice, or any tool with a "Download raw responses" option).

Required columns

Optional columns that enrich the analysis

Sample size guidance

How to Read the Report

Score distributions: density curves for each survey dimension. Look at the shape, not just the center. A bimodal distribution (two humps) in one department means polarized opinions. A mean can hide a split response pattern. Inspect the full score spread and report the share in each relevant score range. This is often the most important finding in the entire report.

Categorical distributions: shows how many respondents fall into each group. Severely imbalanced groups (one department with 200 respondents and another with 12) reduce the statistical power of group comparisons. The report flags this.

Cross-tabulation heatmaps: shows the relationship between categorical variables. A significant chi-square result (p < 0.05) means the variables are associated, for example, tenure band and job level are not independent in your organization. This matters because a "department effect" might actually be a "tenure effect" if one department has systematically different tenure distributions.

ANOVA results: the F-statistic and p-value for each score dimension across each grouping variable. A small p-value is evidence against the null model of equal group means. It does not prove a gap is real or useful. Read it with the effect size, uncertainty, study design, sample size, and test assumptions. Look at the effect size (eta-squared) too, since a difference can be statistically detectable but small.

Post-hoc comparisons (Tukey HSD): the pairs that actually differ. If ANOVA says "departments differ on satisfaction," Tukey tells you which specific pair, such as Support vs. Engineering, drives the result, while another pair, such as Sales vs. Marketing, may not differ. This is where you find the actionable findings. Focus on pairs where the confidence interval for the difference does not cross zero.

Score correlations, when you map multiple survey dimensions, the correlation matrix shows which dimensions move together. A strong correlation between "manager effectiveness" and "career growth" suggests they measure the same underlying factor. Low correlation suggests they're independent, improving one won't automatically improve the other.

What to Do With the Results

Immediate

Strategic

When to Use Something Else

Your turn

Bring your own data and the question you actually need answered.

MCP AnalyticsData science by email Send us your data and your question, and we will reply. ds@mcpanalytics.ai