LASSO Regression: Find the Predictors That Actually Matter

You have a target variable and fifteen predictors. Maybe five of them matter. Maybe two. You do not know which ones, and ordinary regression will happily assign a coefficient to every single one, including the noise. LASSO regression solves this by shrinking irrelevant coefficients to exactly zero, automatically telling you which predictors to keep and which to drop. Upload a CSV to see which variables have nonzero weights in the model.

Worked Example (Simulated): Which of Eight Inputs Does LASSO Keep?

Question: Which of eight inputs does LASSO keep when it predicts a number?

Inputs: Simulated data, seed 24. The script makes 180 rows with 8 random inputs, x0 to x7. The outcome is built as y = 3 × x0 − 2 × x2 + noise, with noise SD 0.7. That means six of the eight true weights are built in at zero, so finding x0 and x2 is expected. The script sets 45 rows aside as test data and uses the other 135 to fit.

Method: The script scales each input, then uses five-fold cross-validation to pick the penalty (called alpha in scikit-learn and lambda elsewhere on this page). Cross-validation tests each penalty on held-out rows. It then lists the inputs with nonzero weights and reports test RMSE, a measure of how far the predictions are from the true values.

import numpy as np
from sklearn.linear_model import LassoCV
from sklearn.model_selection import KFold, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_squared_error

seed = 24
rng = np.random.default_rng(seed)
n_rows, n_features = 180, 8
X = rng.normal(size=(n_rows, n_features))
noise = rng.normal(scale=0.7, size=n_rows)
y = 3.0 * X[:, 0] - 2.0 * X[:, 2] + noise
names = [f"x{i}" for i in range(n_features)]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=seed
)
model = make_pipeline(
    StandardScaler(),
    LassoCV(cv=KFold(n_splits=5, shuffle=True, random_state=seed),
            max_iter=100000),
)
model.fit(X_train, y_train)
lasso = model.named_steps["lassocv"]
selected = [name for name, coef in zip(names, lasso.coef_) if coef != 0.0]
pred = model.predict(X_test)
rmse = mean_squared_error(y_test, pred) ** 0.5
print(f"Seed: {seed}")
print(f"Rows: {n_rows}; predictors: {n_features}; "
      f"train: {len(y_train)}; test: {len(y_test)}")
print("Generated outcome: y = 3*x0 - 2*x2 + noise (noise SD 0.7)")
print(f"Chosen alpha: {lasso.alpha_:.6f}")
print(f"Selected predictors: {', '.join(selected)}")
print(f"Test RMSE: {rmse:.3f}")

Printed output:

Seed: 24
Rows: 180; predictors: 8; train: 135; test: 45
Generated outcome: y = 3*x0 - 2*x2 + noise (noise SD 0.7)
Chosen alpha: 0.040382
Selected predictors: x0, x2, x4, x6
Test RMSE: 0.693

Result: In this run LASSO kept x0, x2, x4 and x6. The outcome was built from x0 and x2, so finding those two is expected. It also kept x4 and x6. Test RMSE is 0.693, and the noise SD built into the data is 0.7. That is about as low as the error can go for this data, because 0.7 is the noise built in.

Limit: The data were made for this example, so the result may not match real data. Keeping x4 and x6 does not show that either input has a real effect. This is not advice on ad spend or any real model.

Sources

Cited for who published each work and when, not for any number on this page.

• Tibshirani R (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B. doi 10.1111/j.2517-6161.1996.tb02080.x
• James G, Witten D, Hastie T, Tibshirani R. An Introduction to Statistical Learning (2nd ed.). statlearning.com
• scikit-learn developers. scikit-learn: linear models (the example uses LassoCV). scikit-learn.org/stable/modules/linear_model.html

What Is LASSO Regression?

LASSO stands for Least Absolute Shrinkage and Selection Operator. It is linear regression with a penalty, specifically, an L1 penalty that adds the sum of the absolute values of the coefficients to the loss function. That penalty does something ordinary regression cannot: it forces some coefficients all the way to zero. Not close to zero. Exactly zero. The variable is gone from the model entirely.

To see how much your own drivers explain an outcome, and which matter most, try the free linear regression tool.

This is what makes LASSO fundamentally different from standard linear regression. Ordinary least squares (OLS) fits a coefficient to every predictor you give it, even if a predictor is pure noise. With many predictors, OLS can overfit. LASSO adds a penalty on complexity. As the penalty strength (called lambda) increases, more and more coefficients get pushed to zero. The variables that survive are the ones the model keeps at that penalty (Tibshirani, 1996).

Think of it this way. Suppose you are a marketing director with spend data across fifteen channels: TV, paid search, social media, radio, print, podcasts, influencer campaigns, and more. You want to know which channels drive revenue. Ordinary regression will give you fifteen coefficients, some of them tiny, some with unexpected signs when channels overlap. LASSO may keep a few channels, such as paid search, social and TV, and set the rest to zero. A model with three variables is easier to interpret and to explain to your CFO. The worked example above shows what this looks like on made-up data.

The math behind the penalty is elegant. In OLS, you minimize the sum of squared residuals. In LASSO, you minimize the sum of squared residuals plus lambda times the sum of absolute coefficient values. That absolute value is the key. It creates a diamond-shaped constraint region in coefficient space, and the corners of a diamond lie on the axes. When the optimal solution hits a corner, that coefficient is exactly zero. Ridge regression, by contrast, uses squared coefficients (L2 penalty), which creates a circular constraint: coefficients shrink toward zero but never reach it.

When to Use LASSO Regression

The clearest signal that you need LASSO is when you have more predictors than you know what to do with and suspect that only a handful actually matter. A marketing dataset might have twenty campaign variables but only three that drive the outcome. A genomics study might have thousands of gene expression features but only a dozen that predict disease status. An economics model might include forty macroeconomic indicators when GDP growth really depends on five or six.

LASSO is also the right choice when interpretability matters more than squeezing out the last bit of predictive accuracy. A model with four predictors is something you can explain in a meeting. A model with forty predictors is a black box that no one trusts. If you need stakeholders to act on the results (reallocate budget, change strategy, redesign a process) fewer variables means clearer communication.

Use LASSO when you suspect multicollinearity among your predictors. If several predictors are highly correlated (like TV spend and total ad spend, or temperature and season), OLS produces unstable coefficients that swing wildly with small data changes. LASSO tends to keep one of a group of correlated predictors and set the others to zero. Which one it keeps can depend on the data and the penalty.

Finally, LASSO is useful when the number of predictors is close to, or larger than, the number of observations. With more inputs than rows, ordinary least squares has no single best set of weights: many sets fit the rows equally well (James et al., An Introduction to Statistical Learning). The LASSO penalty can still return one set of weights.

What Data Do You Need?

You need a CSV with one numeric outcome column (the thing you are trying to predict) and one or more numeric predictor columns. The module requires you to map at least one predictor column, and you can map as many as your dataset has: the more predictors you provide, the more useful LASSO's variable selection becomes. If you only have two or three predictors, LASSO still works, but its main advantage (automatic selection) is less impactful.

All columns must be numeric. If you have categorical predictors (like region or product category), you will need to encode them as dummy variables before uploading. The outcome variable should be continuous: revenue, score, weight, duration, or any measured quantity. For binary outcomes (yes/no, pass/fail), logistic regression with LASSO is more appropriate, but the standard LASSO module handles the continuous case.

The module also offers a standardization option. LASSO penalizes coefficients by their absolute size, so predictors measured on different scales (one in dollars, another in percentages) would be penalized unequally. Standardization puts all predictors on the same scale before fitting, which is a common choice. The report shows coefficients on the original scale for interpretability.

How to Read the Report

The report has eight sections, each answering a specific question about your data and model.

The Analysis Overview summarizes what was analyzed: the outcome variable, the predictors, and the key configuration choices (alpha, number of folds, lambda selection method). This is your quick reference for what the model was asked to do.

The Data Pipeline section shows how the data was preprocessed: how many rows survived cleaning, whether standardization was applied, and any columns that were excluded due to missing values or zero variance. Check this first if the results look unexpected.

The Regularization Path is the signature LASSO visualization. It shows how each coefficient changes as lambda increases from left to right. At the far left (low penalty), all predictors have non-zero coefficients. The model looks like ordinary regression. As lambda increases, coefficients shrink and eventually hit zero. The path shows how weights change as the penalty grows. Dropout order alone does not show which input matters most. If inputs overlap, the choice and order can depend on the data and the penalty.

The Cross-Validation Error plot shows the model's prediction error at each lambda value, typically with error bars for each fold. Two vertical lines mark the optimal lambda (minimum cross-validation error) and the "one-standard-error" lambda (the simplest model within one standard error of the minimum).

The Selected Coefficients card lists the variables that survived: the ones with non-zero coefficients at the chosen lambda. This is the answer to the question "which predictors matter?" The magnitude and sign of each coefficient tell you how much each surviving predictor contributes to the outcome and in which direction. Variables not listed here have zero weights in this model at the chosen penalty. That does not show that they never help or have no link to the outcome.

The Model Fit section shows residual diagnostics, residuals vs. fitted values, Q-Q plots, and other checks for whether the linear model assumptions hold. Look for patterns in the residuals (which suggest non-linearity) and deviations from the Q-Q line (which suggest non-normality). These diagnostics apply to the final sparse model, not the full predictor set.

The Performance Metrics card reports R-squared, adjusted R-squared, and prediction error metrics (like RMSE or MAE). R-squared tells you what fraction of the outcome's variance is explained by the surviving predictors. Compare the LASSO model's R-squared against what you would get from a full OLS model, if LASSO achieves a similar R-squared with far fewer variables, the dropped variables added little in this model.

The Summary brings it all together with AI-generated insights that highlight the key findings: how many variables survived, which ones they are, and what the model's overall fit looks like. This is the section to read first if you want the bottom line before diving into the charts.

When to Use Something Else

If you want to keep all your predictors in the model rather than eliminate any, use Ridge Regression. Ridge uses an L2 penalty that shrinks coefficients toward zero but never reaches it: every predictor stays in the model with a small coefficient. This is better when you believe all predictors contribute at least a little, or when you have groups of highly correlated predictors and want to spread the coefficient weight across all of them rather than picking one.

If you are torn between LASSO and Ridge, Elastic Net is the compromise. It combines L1 and L2 penalties, controlled by a mixing parameter (alpha). At alpha = 1 you get pure LASSO; at alpha = 0 you get pure Ridge. Elastic Net is particularly useful when you have groups of correlated predictors, LASSO tends to arbitrarily pick one from each group, while Elastic Net can keep several. The LASSO module lets you set the alpha parameter, so you can move toward Elastic Net behavior by lowering alpha below 1.

If your primary goal is predictive accuracy rather than interpretability, and you have enough data, Random Forest or XGBoost may fit better than LASSO. These methods capture non-linear relationships and interactions that LASSO (being linear) cannot. They also provide feature importance rankings, but those rankings are not the same as LASSO's coefficient-based selection. They measure predictive contribution, not linear association. Use tree-based methods when you care about prediction; use LASSO when you need a clear, interpretable equation.

LASSO can help when a model with few inputs is the goal. Stepwise selection (forward, backward, or both) is a different approach that adds or drops one variable at a time. The two use different rules, and the right choice depends on your goal and your data.

The R Code Behind the Analysis

Every report names its method and includes the R code that produced it, so you can reproduce, audit and cite it. The method code is fixed, not rewritten for each run, so the same data gives the same analysis every time.

The analysis uses glmnet with alpha = 1 (the L1 penalty that defines LASSO) and cv.glmnet() for lambda selection by cross-validation: both from the glmnet package. The regularization path is computed across a grid of lambda values, and cv.glmnet() evaluates prediction error at each lambda using k-fold cross-validation. The final model uses either lambda.min (lowest error) or lambda.1se (simplest model within one standard error), depending on your configuration. Coefficients are extracted with coef(), which shows exactly which variables survived: non-zero entries are your selected predictors, zero entries were eliminated. Every step is visible in the code tab of your report, so you or a statistician can check exactly what was done.