Correlation Analysis: Find What's Connected in Your Data

Before you build a model, run a regression, or make a prediction, you need to know which variables in your data actually relate to each other. Correlation analysis answers that question: upload a CSV with numeric columns, and get a heatmap, ranked pairs, scatter plots, and significance tests that show you exactly where the relationships are.

Worked Example (Simulated)

Question: In this made-up set, do weekly ad spend and signups rise in a straight-line pattern?

Inputs: Simulated data, not customer data. The script needs numpy and scipy. It makes twelve weekly pairs, seed 314159. Weekly spend is drawn at random between 5 and 15 made-up units, and the week number is kept apart from spend. The script sets signups to rise 2.5 per unit of spend, plus noise (standard deviation 3), so the link is built in.

Method: Pearson r measures a straight-line link. The script also shows Spearman rho, a rank-based measure. The p-value and the interval rely on the assumptions of these methods.

import numpy as np
from scipy import stats

seed = 314159
rng = np.random.default_rng(seed)
weeks = np.arange(1, 13)
x = np.round(rng.uniform(5, 15, size=weeks.size), 1)
y = 2.5 * x + rng.normal(0, 3, size=x.size)
n = len(x)
r, p = stats.pearsonr(x, y)
rho, rho_p = stats.spearmanr(x, y)
t_stat = r * np.sqrt((n - 2) / (1 - r**2))
mean_x, mean_y = x.mean(), y.mean()
sxx = np.sum((x - mean_x)**2)
syy = np.sum((y - mean_y)**2)
sxy = np.sum((x - mean_x) * (y - mean_y))
z = np.arctanh(r)
se_z = 1 / np.sqrt(n - 3)
ci = np.tanh([z - stats.norm.ppf(0.975) * se_z,
              z + stats.norm.ppf(0.975) * se_z])
print(f'Simulated data, seed {seed}')
print('Weeks:', ', '.join(map(str, weeks)))
print('Spend units (x):', ', '.join(f'{v:.1f}' for v in x))
print('Signups units (y):', ', '.join(f'{v:.3f}' for v in y))
print(f'n = {n}')
print(f'mean x = {mean_x:.3f}, mean y = {mean_y:.3f}')
print(f'Sxx = {sxx:.3f}, Syy = {syy:.3f}, Sxy = {sxy:.3f}')
print(f'Pearson r = {r:.4f}')
print(f'r-squared = {r**2:.4f}')
print(f't = {t_stat:.4f}, df = {n-2}, two-sided p = {p:.6f}')
print(f'95% Fisher z interval for r = [{ci[0]:.4f}, {ci[1]:.4f}]')
print(f'Spearman rho = {rho:.4f}, two-sided p = {rho_p:.6f}')

Printed output:

Simulated data, seed 314159
Weeks: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
Spend units (x): 14.1, 8.1, 11.5, 13.1, 6.0, 10.1, 9.5, 6.2, 8.6, 11.0, 12.0, 11.7
Signups units (y): 32.926, 19.721, 26.082, 33.806, 19.374, 26.286, 27.360, 20.779, 20.091, 28.754, 27.775, 34.178
n = 12
mean x = 10.158, mean y = 26.428
Sxx = 72.529, Syy = 331.759, Sxy = 138.007
Pearson r = 0.8897
r-squared = 0.7915
t = 6.1619, df = 10, two-sided p = 0.000107
95% Fisher z interval for r = [0.6452, 0.9689]
Spearman rho = 0.8601, two-sided p = 0.000332

Result: Pearson r is 0.8897 and Spearman rho is 0.8601. In this simulated set, higher spend goes with more signups, as the script built it to. With only 12 pairs, the 95% interval for r is wide, from 0.6452 to 0.9689. A small p-value does not show that a link is real.

Limit: These made-up pairs do not show that spend caused signups. Other factors could affect both. A p-value is the chance, if the true correlation were zero and the test assumptions held, of seeing a result at least this far from zero.

Sources

• Wasserstein, R. L., and Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician. doi:10.1080/00031305.2016.1154108. Cited for what a p-value does and does not say.

What Is Correlation Analysis?

Correlation measures how two variables move together. The result is a number between -1 and +1. A correlation of +1 means an exact positive linear relationship between the two variables. A correlation of -1 means they move in perfectly opposite directions. A correlation near 0 means there is no linear relationship between them at all.

Think of practical examples. Ad spend and revenue often have a positive correlation, spend more, earn more. Price and quantity sold typically have a negative correlation, raise the price, sell fewer units. These are intuitive, but correlation analysis finds the non-obvious relationships too: the ones hiding in your data that you would never think to check manually.

The real power shows up when you have a dataset with dozens of columns. Instead of guessing which variables matter, correlation analysis tests every pair and ranks them. You get a complete picture of your data's internal structure in one pass.

When to Use Correlation Analysis

Correlation analysis is often a reasonable first step when you are exploring a new dataset. Before you build a regression model, you need to know which variables are worth including. Before you run an A/B test, you need to understand what else might be influencing your outcome. Correlation gives you that map. In marketing, it answers questions like "which channels correlate most strongly with revenue?" or "does email open rate relate to purchase frequency?" In operations, it reveals whether overtime hours correlate with defect rates, or whether shipping distance drives return rates.

It is especially useful for variable selection. If you are building a predictive model and have 30 potential input variables, correlation analysis immediately tells you which ones have a relationship with your target. Strong correlations between input variables can also point to multicollinearity, when two inputs are so highly correlated that including both in a model causes problems.

Research teams use correlation as a screening tool before running more expensive analyses. If you are planning an experiment, checking correlations in historical data tells you which factors to control for. Zero pairwise correlation does not rule out confounding. Check the study design and other variables before drawing a causal conclusion. It is fast, cheap, and gives you a foundation for every analysis that follows.

What Data Do You Need?

You need a CSV (or Excel file) with at least two numeric columns. That is the minimum, but correlation analysis gets more useful the more columns you have. With 5 numeric columns you get 10 pairs to examine. With 10 columns, 45 pairs. With 20 columns, 190 pairs. The analysis tests all of them and ranks the results so you can focus on what matters.

Missing values are handled automatically. If a row has a missing value in one of the two columns being compared, that row is excluded from that specific pair's calculation but still used for every other pair where the data exists. This means you do not need a perfectly clean dataset to get useful results.

The report computes Pearson correlations for each pair of columns. Spearman is a rank-based alternative; it is not run by the free tool. The tool needs at least two numeric columns and 10 usable rows.

How to Read the Report

The report opens with a correlation heatmap: a grid where every cell represents one pair of variables, and the color represents the strength and direction of the relationship. Dark red means strong positive correlation, dark blue means strong negative, and white means no relationship. Scan the heatmap for clusters of color: if a group of variables all correlate with each other, they might be measuring the same underlying thing.

Below the heatmap, you will find a ranked table of the top correlation pairs, sorted by absolute correlation strength. This is where the actionable findings live. Each row shows the two variables, the correlation coefficient, and the p-value. The p-value tells you whether the correlation is statistically significant or could have occurred by random chance. The report lists a p-value and stars for each pair. A p-value is the chance, if the true correlation were zero and the test assumptions held, of seeing a result at least this far from zero. It does not show that a relationship is real. The report does not adjust for the number of pairs tested, so with many columns some pairs will pass by chance.

The report also includes a scatter plot of the strongest relationship found, so you can check whether the correlation reflects a pattern or a few outlier points driving the number. AI-generated insights summarize the key findings in plain language: which variables are most connected, which ones are surprisingly unrelated, and what the patterns suggest about your data. Every chart and number links back to the exact R code that produced it, so the results are fully reproducible.

When to Use Something Else

Correlation tells you that two variables are associated. It does not tell you that one causes the other. If you need to establish a causal relationship or predict one variable from another, use regression analysis instead. Regression quantifies the predictive relationship and gives you a model you can use for forecasting.

If you have many highly correlated variables and want to reduce them to a smaller set of underlying factors, PCA (Principal Component Analysis) is the right next step. PCA takes a group of correlated variables and distills them into a few independent components that capture most of the variance. This is common in survey data where 20 questions might really measure 3 or 4 underlying attitudes.

If your variables are categorical rather than numeric (for example, product category and customer region) correlation does not apply. Use chi-square tests to test whether categorical variables are associated. Remember that even a strong, statistically significant correlation does not prove causation. It shows association. Establishing causation requires experimental design or specialized causal inference methods like difference-in-differences or instrumental variables.

The R Code Behind the Analysis

Every report names its method and includes the R code that produced it, so you can reproduce, audit and cite it. The method code is fixed, not rewritten for each run, so the same data gives the same analysis every time.

The correlation module uses cor() from base R to compute the correlation matrix across all numeric columns, and cor.test() to generate p-values for each pair. The heatmap is rendered with ggplot2 using geom_tile(), and the scatter plot of the strongest pair uses geom_point() with a fitted trend line from geom_smooth().