Linear Regression: What Drives Your Numbers?

You have a number you care about: revenue, conversions, test scores, customer satisfaction. Linear regression estimates the link between that number and each input you give it, while the other inputs stay fixed. This shows a link, not cause. A cause claim needs a sound study plan and clear rules.

Worked Example (made-up data)

Question: How does weekly revenue link to weekly search and social ad spend in this made-up set?

Inputs: The script makes 24 weeks of spend and revenue data. Each unit is $1,000. The data are made up, not real ad results.

Built in: The script builds revenue as 18 + 2.4 x search + 1.1 x social + noise, where the noise has a spread (standard deviation) of 3. The two spends are drawn independently of each other, which real campaigns rarely are.

Method: Use ordinary least squares, a way to fit a line by making the errors small. Add an intercept, the model's base value. Each coefficient is the model's link to revenue while the other spend stays fixed.

import numpy as np
from scipy import stats

# Simulated weekly spend and revenue, all in thousands of dollars.
seed = 17
rng = np.random.default_rng(seed)
n = 24
search = rng.uniform(2.0, 12.0, n)
social = rng.uniform(1.0, 8.0, n)
noise = rng.normal(0.0, 3.0, n)
revenue = 18.0 + 2.4 * search + 1.1 * social + noise
X = np.column_stack((np.ones(n), search, social))
beta = np.linalg.lstsq(X, revenue, rcond=None)[0]
predicted = X @ beta
resid = revenue - predicted
df = n - X.shape[1]
sigma2 = np.sum(resid ** 2) / df
cov = sigma2 * np.linalg.inv(X.T @ X)
se = np.sqrt(np.diag(cov))
t_stat = beta / se
p_value = 2 * stats.t.sf(np.abs(t_stat), df)
ci_half = stats.t.ppf(0.975, df) * se
ss_total = np.sum((revenue - revenue.mean()) ** 2)
r_squared = 1 - np.sum(resid ** 2) / ss_total

def show_p(p):
    return "p below 0.001" if p < 0.001 else f"p = {p:.4f}"

print(f"Simulated data, seed {seed}")
print("Inputs: weekly search spend and social spend; "
      "each unit is $1,000;")
print("target is weekly revenue in $1,000s")
print("Built in: revenue = 18 + 2.4 x search + 1.1 x social + noise (SD 3)")
print(f"Rows: {n}")
print("Fit: ordinary least squares with an intercept")
print(f"Intercept: {beta[0]:.3f}")
for i, name in enumerate(("Search", "Social"), start=1):
    lo = beta[i] - ci_half[i]
    hi = beta[i] + ci_half[i]
    print(f"{name} coefficient: {beta[i]:.3f}; 95% CI [{lo:.3f}, {hi:.3f}]; "
          f"{show_p(p_value[i])}")
print(f"R-squared: {r_squared:.3f}")
print(f"Residual standard error: {np.sqrt(sigma2):.3f}")

Printed output:

Simulated data, seed 17
Inputs: weekly search spend and social spend; each unit is $1,000;
target is weekly revenue in $1,000s
Built in: revenue = 18 + 2.4 x search + 1.1 x social + noise (SD 3)
Rows: 24
Fit: ordinary least squares with an intercept
Intercept: 18.833
Search coefficient: 2.278; 95% CI [1.887, 2.668]; p below 0.001
Social coefficient: 0.899; 95% CI [0.326, 1.473]; p = 0.0037
R-squared: 0.899
Residual standard error: 2.458

Result: Search has an estimated link of 2.278, with a 95% range of 1.887 to 2.668. That range contains the built-in 2.4. Social has an estimated link of 0.899, with a 95% range of 0.326 to 1.473, which contains the built-in 1.1. The model fit score, called R-squared, is 0.899, which is expected when the data are built from two inputs plus modest noise. These are links in this made-up set. They do not prove that more spend led to more revenue.

Limit: The data follow a simple made-up rule. Real ad data may have other causes of sales, may have spends that move together, and may not fit this model.

What Is Linear Regression?

Linear regression is the workhorse of statistical modeling. You pick a target variable, the thing you want to explain or predict, and one or more predictor variables that you think influence it. The model fits a line (or hyperplane, with multiple predictors) through your data and tells you exactly how much each predictor contributes to the outcome. A one-unit increase in ad spend is associated with a $3.20 increase in revenue? That is a coefficient, and the model gives you one for every predictor.

What makes linear regression practical is that it does not just give you a prediction. Every coefficient comes with a p-value and a confidence range. A p-value tells how odd the data would be under a set of rules for a no-effect case, such as a zero coefficient. This only holds if the model rules fit. It does not say if an effect matters in real life.

Here is a concrete example. You are running ads on Facebook and Google, and you want to know how much each dollar of spend drives in revenue. Linear regression takes your daily data (revenue as the target, Facebook spend and Google spend as predictors) and returns two coefficients. Maybe Facebook returns $2.80 per dollar and Google returns $4.10 per dollar. That is a link, not proof of cause, but it is the kind of question this analysis supports.

When to Use Linear Regression

Marketing teams use linear regression constantly. Which channels actually drive sales? If you have weekly data on email sends, paid search spend, social media impressions, and revenue, regression separates the signal from the noise. It answers questions like "does increasing our Google Ads budget by $1,000 actually move revenue?" with a number and a confidence level, not a gut feeling.

Pricing decisions are another natural fit. If you have transaction data with price and quantity sold, regression quantifies the relationship. A coefficient of -12 on price means each $1 price increase is associated with 12 fewer units sold. Combine that with your margin data and you can calculate the optimal price point. HR teams use the same approach: what predicts employee performance? Is it years of experience, training hours completed, or team size? Regression tells you which factors carry weight and which ones are just correlated by coincidence.

Operations teams lean on regression for cost modeling. What drives shipping costs? Package weight, distance, carrier, and time of year might all play a role. Regression quantifies each factor so you can forecast costs accurately and identify which levers you can pull. The common thread across all these use cases: you have a numeric outcome and want to understand what influences it, with evidence you can defend to stakeholders.

What Data Do You Need?

You need a CSV with at least two columns: one numeric target variable (the thing you want to predict or explain) and one or more numeric predictor columns. Revenue and ad spend. Test scores and study hours. Weight and calorie intake. The target must be continuous, if it is a yes/no outcome, you need logistic regression instead.

More rows give you more reliable results. With 30 rows, the model might find a pattern but the confidence intervals will be wide. With 300 rows, the estimates tighten up considerably.

Do not worry about correlated predictors. If your Facebook spend and Google spend tend to go up and down together, the model can still work, but the report flags this with VIF (Variance Inflation Factor) scores. A high VIF means two predictors are so correlated that the model struggles to separate their individual effects. The report tells you which predictors are affected so you can decide whether to drop one or keep both and accept wider confidence intervals.

How to Read the Report

Start with R-squared at the top of the report. This number ranges from 0 to 1 and tells you how much of the variation in your target the model explains. An R-squared of 0.75 means the predictors account for 75% of the variation. The rest is change that the model does not explain. There is no universal "good" threshold; it depends on your domain.

Next, look at the coefficient table. Each row is a predictor. The "Estimate" column tells you the effect size: how much the target changes for a one-unit increase in that predictor, holding everything else constant. The "p-value" column tells how odd the data would be under a set of rules for a no-effect case. Read it with the estimate and its confidence range. It does not show the size or worth of a link. Focus your attention on predictors with a meaningful estimate and a range that excludes values you would not act on.

The residual plots check whether the model's assumptions hold. The residuals-vs-fitted plot should look like a random cloud, if you see a curve or a funnel shape, the model is missing something. The Q-Q plot checks whether residuals are approximately normal. The actual-vs-predicted chart gives you a visual sense of fit: points clustered tightly around the diagonal line mean the model predicts well. Points scattered far from the line mean the model struggles with those observations.

When to Use Something Else

If you have many predictors and suspect several of them are correlated or irrelevant, consider ridge regression or lasso regression. Ridge handles correlated predictors by shrinking coefficients toward zero without eliminating them. Lasso goes further and sets some coefficients exactly to zero, effectively selecting the most important predictors for you.

If your target variable is a yes/no outcome (did the customer convert? did the patient recover?) you need logistic regression instead. Linear regression assumes a continuous numeric target and will give misleading results on binary outcomes. If the relationship between predictors and target is clearly non-linear (for example, revenue grows exponentially with ad spend up to a saturation point) then XGBoost or random forest will capture those patterns better.

Sometimes you do not need prediction at all. You just want to see which variables are related to each other. In that case, a simple correlation analysis gives you a matrix of pairwise relationships without building a model. Correlation means two measures tend to move together. Regression links a target to one or more inputs, while the other inputs stay fixed. Neither shows cause on its own; a cause claim needs a sound study plan and clear rules. Choose based on whether you need a model of a target or just want to explore your data.

The R Code Behind the Analysis

Every report names its method and includes the R code that produced it, so you can reproduce, audit and cite it.

The analysis uses lm() from base R: the same ordinary least squares implementation found in every statistics textbook and university course. Multicollinearity is checked with car::vif(), which calculates the Variance Inflation Factor for each predictor. Diagnostic plots use base R's plot() on the model object, producing residuals-vs-fitted, Q-Q, scale-location, and leverage plots. The summary() function produces the coefficient table with standard errors, t-statistics, and p-values. No custom or experimental code, just standard, well-tested statistical functions that have been tested by decades of use.