Executive Summary
Model fit and top drivers of Weekly Sales across 200 observations
Linear regression on 200 observations explains 99% of Weekly Sales variation (R² = 0.99, adjusted 0.99) with typical error of 9.973. TV Spend is the dominant driver, followed by Radio Spend. Four of five driver terms achieve statistical significance at p < 0.05. This strong fit indicates marketing spend channels—particularly TV and Radio—alongside unit price and region, are reliable predictors of weekly sales outcomes in this dataset.
Analysis Overview
Ordinary least squares regression of Weekly Sales on 5 driver column(s) across 200 observations.
This analysis applies ordinary least squares regression to explain Weekly Sales using 5 drivers: TV Spend, Radio Spend, Social Spend, Unit Price, and Region across 200 observations. The model achieves an R² of 0.99 (adjusted 0.99), meaning 99% of variation in Weekly Sales is captured by these drivers. Each coefficient quantifies that driver's independent effect on Weekly Sales, holding others constant. Typical prediction error (RMSE) is 9.973 in Weekly Sales units. The exceptionally high explanatory power indicates the drivers account for nearly all systematic movement in weekly sales, with minimal unexplained residual variation.
Data Quality
Missing value handling, driver typing, and exclusions.
All 200 rows retained—no observations were excluded due to missing Weekly Sales values. Numeric drivers underwent median imputation where values were missing; categorical variables (Region) were encoded as dummy variables with rare levels grouped as 'Other'. All resulting driver columns proved usable for modelling. Weekly Sales ranged from 34.34 to 462.61 with a median of 244.515, establishing the observational domain within which coefficients apply. No drivers were excluded as constant or identifier-like, ensuring all five specified drivers entered the model.
Distribution of the Outcome
How Weekly Sales is distributed across the dataset.
Weekly Sales spans [0,50] to (450,500], with the modal range (300,350] containing 33 observations (16.5% of the total). The distribution is relatively even across the middle ranges: (150,200] holds 32 observations, (200,250] holds 30, and (250,300] holds 31. Only 2 observations fall in the lowest range [0,50], and just 1 in the highest (450,500]. This spread across 200 observations anchors the regression coefficients: they describe expected sales movement within this observed range, not beyond it.
Coefficient Estimates
Effect of each driver on Weekly Sales with standard errors, 95% confidence intervals, and significance.
| Term | Estimate | Std Error | T Value | P Value | CI Low | CI High | Significance |
|---|---|---|---|---|---|---|---|
| Intercept | 50.14 | 3.288 | 15.25 | 6.43e-35 | 43.65 | 56.62 | *** |
| TV Spend | 3.162 | 0.0243 | 130.3 | 2.46e-189 | 3.114 | 3.21 | *** |
| Radio Spend | 1.79 | 0.0522 | 34.31 | 7.63e-84 | 1.687 | 1.892 | *** |
| Social Spend | 0.8963 | 0.0317 | 28.25 | 2.62e-70 | 0.8337 | 0.9589 | *** |
| Unit Price | -2.381 | 0.1193 | -19.96 | 1e-48 | -2.617 | -2.146 | *** |
| Region: North | -0.5501 | 2.145 | -0.256 | 0.798 | -4.781 | 3.681 | |
| Region: South | 2.038 | 2.15 | 0.948 | 0.344 | -2.203 | 6.28 | |
| Region: West | 3.089 | 2.12 | 1.457 | 0.147 | -1.094 | 7.271 |
TV Spend shows the largest effect: a one-unit increase is associated with 3.1621 more in Weekly Sales (95% CI: 3.1143–3.21, p = 2.46e-189). Radio Spend follows with 1.7895 per unit (95% CI: 1.6867–1.8924, p = 7.63e-84), and Social Spend adds 0.8963 per unit (95% CI: 0.8337–0.9589, p = 2.62e-70). Unit Price shows a negative association: −2.3815 per unit increase (95% CI: −2.6169 to −2.1462, p = 1e-48). Region terms (North, South, West) are not statistically significant at p < 0.05, with confidence intervals crossing zero. All three marketing channels show strong, positive, reliable effects on sales.
Driver Ranking
Relative statistical influence of each driver, scaled 0-100.
TV Spend dominates with an importance score of 100, reflecting its t-statistic of 130.348 and p-value of 2.46e-189. Radio Spend ranks second at 26.3 (t = 34.313), followed by Social Spend at 21.7 (t = 28.25) and Unit Price at 15.3 (t = −19.963). Region achieves only 1.1 importance with p = 0.147, indicating negligible statistical influence. This ranking shows TV is roughly 3.8 times more influential than Radio and 4.6 times more influential than Social in predicting weekly sales, while geographic location contributes minimally after accounting for spending levels.
Top Driver Effect
Observed relationship between the strongest numeric driver and the outcome.
Weekly Sales rises with TV Spend in a strong positive relationship (r = 0.93) across 200 observations. TV Spend ranges from 0.2 to 100. The scatter shows consistent positive trend: lower TV spend values (0.2–10) cluster in lower sales ranges, while higher values (50+) associate with higher sales. The regression coefficient of 3.1621 quantifies this slope. This tight correlation (r = 0.93) and high statistical significance (p = 2.46e-189) establish TV as the primary marketing channel associated with weekly sales variation in this dataset.
Predicted vs Actual
Model predictions against observed Weekly Sales.
Model predictions track actual Weekly Sales closely, with points clustering near the diagonal. Predicted values range from 55.545 to 453.8918, matching the observed outcome range. With R² = 0.99, the model captures nearly all systematic variation; residual scatter (typical error 9.973) reflects only minor unmodelled factors. A few predictions show larger mismatches—for instance, predicted 90.2772 versus actual 68.01, and predicted 446.6269 versus actual 427.51—but these are exceptions. The tight fit confirms that TV, Radio, Social spend, Unit Price, and Region together explain weekly sales reliably.
Residual Diagnostics
Prediction errors across the fitted range — a check of the linear model's assumptions.
Residuals center near 0 with a spread of 9.998, consistent with the reported RMSE of 9.973. Approximately 50.5% of predictions undershoot actual values, indicating unbiased centering. Residual spread remains stable across the fitted range (from ~55 to ~454), with no evidence of heteroscedasticity—error variance does not grow or shrink with prediction magnitude. This stability supports the linear model's assumptions. A few outliers exist (e.g., residual −22.2672 at fitted 90.2772), but they are rare and do not systematically undermine the model's validity.
Methodology
Statistical methodology and diagnostics for Linear Regression — Outcome Drivers
Statistical Method
Standard-library analysis: explains any numeric outcome from the driver columns you choose, using ordinary least squares regression. Reports coefficient estimates with 95% confidence intervals and significance, ranks drivers by standardized influence, shows the strongest driver's effect, and checks model fit with predicted-vs-actual and residual diagnostics. Works on any dataset: map one numeric outcome and 1-8 drivers (numeric or categorical).
- The outcome is numeric and approximately linear in the drivers
- Observations are independent
- Residuals are roughly constant-variance (checked in diagnostics)
- Correlational, not causal — coefficients describe association
- Non-linear relationships are only partially captured
- High-cardinality text drivers are grouped or excluded
Analysis Code
Complete R source code for this analysis
Linear Regression — Outcome Drivers
Fits an ordinary least squares regression to explain a numeric outcome from one or more driver columns chosen by the user. Reports coefficient estimates with confidence intervals and significance, ranks drivers by standardized effect, and surfaces model fit and residual diagnostics.
Why This Method?
Linear regression is the workhorse of explanatory analytics: coefficients are directly interpretable ("one unit of X changes Y by b"), significance tests separate signal from noise, and diagnostics reveal when the linear story is incomplete. It is the right first model for almost any "what drives this number?" question.
What This Analysis Covers
- Outcome distribution
- Coefficient table with 95% confidence intervals and p-values
- Driver ranking by standardized effect size
- Effect of the strongest numeric driver
- Predicted vs actual and residual diagnostics
Standard Library
This is a platform standard-library module (LAT-1441): it runs on ANY dataset via the semantic mapping {outcome, driver_1..driver_N}. All narrative text is derived from the user's own column names and computed values — nothing dataset-specific is hardcoded.
suppressPackageStartupMessages(library(htmltools))
suppressPackageStartupMessages(library(jsonlite))
suppressPackageStartupMessages(library(plotly))
suppressPackageStartupMessages(library(DT))
suppressPackageStartupMessages(library(htmlwidgets))
suppressPackageStartupMessages(library(arrow))
suppressPackageStartupMessages(library(knitr))
suppressPackageStartupMessages(library(rmarkdown))
suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(tidyr))
suppressPackageStartupMessages(library(ggplot2))
suppressPackageStartupMessages(library(stringr))
suppressPackageStartupMessages(library(lubridate))
suppressPackageStartupMessages(library(broom))
suppressPackageStartupMessages(library(Matrix))
suppressPackageStartupMessages(library(cluster))
suppressPackageStartupMessages(library(data.table))Step 1: Row accounting + semantic column discovery
initial_rows <- nrow(df)
if (!"outcome" %in% names(df)) {
stop("column_mapping must map an 'outcome' column (the numeric value to explain)")
}
driver_cols <- grep("^driver_[0-9]+$", names(df), value = TRUE)
driver_cols <- driver_cols[order(as.integer(sub("^driver_", "", driver_cols)))]
if (length(driver_cols) == 0) {
stop("column_mapping must map at least one driver column(driver_1)")
}
outcome_name <- humanize_semantic("outcome", col_map)
driver_names <- setNames(humanize_semantic(driver_cols, col_map), driver_cols)Step 2: Coerce outcome to numeric; drop rows with missing outcome
df$outcome <- suppressWarnings(as.numeric(df$outcome))
df <- df[!is.na(df$outcome), , drop = FALSE]
if (nrow(df) < 10) {
stop(sprintf(
"Only %d rows have a usable numeric value in the outcome column('%s'). At least 10 are required.",
nrow(df), outcome_name))
}Step 3: Type each driver — numeric if >=95% of values convert, else factor
dropped_drivers <- character(0)
for (dc in driver_cols) {
v <- df[[dc]]
if (!is.numeric(v)) {
conv <- suppressWarnings(as.numeric(as.character(v)))
n_orig <- sum(!is.na(v) & as.character(v) != "")
if (n_orig > 0 && sum(!is.na(conv)) >= 0.95 * n_orig) {
df[[dc]] <- conv
}
}
v <- df[[dc]]
if (is.numeric(v)) {Numeric: impute NA with median
med <- median(v, na.rm = TRUE)
if (is.na(med)) { dropped_drivers <- c(dropped_drivers, dc); next }
v[is.na(v)] <- med
df[[dc]] <- v
} else {Categorical: blank/NA -> "Missing"; lump beyond 12 levels into "Other"
v <- as.character(v)
v[is.na(v) | trimws(v) == ""] <- "Missing"
tab <- sort(table(v), decreasing = TRUE)
if (length(tab) > 12) {
keep <- names(tab)[1:12]
v[!(v %in% keep)] <- "Other"
}
if (length(unique(v)) > nrow(df) / 2) {Near-unique text column (an ID, not a driver) — exclude
dropped_drivers <- c(dropped_drivers, dc)
next
}
df[[dc]] <- factor(v)
}
}Step 4: Drop zero-variance drivers
for (dc in setdiff(driver_cols, dropped_drivers)) {
v <- df[[dc]]
zero_var <- if (is.numeric(v)) {
isTRUE(var(v, na.rm = TRUE) == 0) || is.na(var(v, na.rm = TRUE))
} else {
length(unique(v)) <= 1
}
if (zero_var) dropped_drivers <- c(dropped_drivers, dc)
}
model_drivers <- setdiff(driver_cols, dropped_drivers)
if (length(model_drivers) == 0) {
stop("No usable driver columns remained after cleaning(all were constant, empty, or identifier-like).")
}
df_clean <- df[, c(model_drivers, "outcome"), drop = FALSE]
final_rows <- nrow(df_clean)
rows_removed <- initial_rows - final_rowsStep 5: Guard — need clearly more rows than model terms
n_terms_est <- sum(sapply(model_drivers, function(dc) {
if (is.factor(df_clean[[dc]])) nlevels(df_clean[[dc]]) - 1 else 1
}))
while (n_terms_est >= final_rows - 5 && length(model_drivers) > 1) {
drop_dc <- model_drivers[length(model_drivers)]
dropped_drivers <- c(dropped_drivers, drop_dc)
model_drivers <- model_drivers[-length(model_drivers)]
df_clean <- df_clean[, c(model_drivers, "outcome"), drop = FALSE]
n_terms_est <- sum(sapply(model_drivers, function(dc) {
if (is.factor(df_clean[[dc]])) nlevels(df_clean[[dc]]) - 1 else 1
}))
}Step 6: Fit OLS
model <- lm(outcome ~ ., data = df_clean)
glance_df <- as.data.frame(broom::glance(model))Step 7: Coefficient table with 95% CIs and significance stars
tidy_df <- broom::tidy(model, conf.int = TRUE, conf.level = 0.95)
coefficients_df <- data.frame(
term = humanize_term(tidy_df$term, model_drivers, col_map),
estimate = round(tidy_df$estimate, 4),
std_error = round(tidy_df$std.error, 4),
t_value = round(tidy_df$statistic, 3),
p_value = signif(tidy_df$p.value, 3),
ci_low = round(tidy_df$conf.low, 4),
ci_high = round(tidy_df$conf.high, 4),
significance = ifelse(is.na(tidy_df$p.value), "redundant",
ifelse(tidy_df$p.value < 0.001, "***",
ifelse(tidy_df$p.value < 0.01, "**",
ifelse(tidy_df$p.value < 0.05, "*",
ifelse(tidy_df$p.value < 0.1, ".", ""))))),
stringsAsFactors = FALSE
)Step 8: Driver importance — max |t| across a driver's terms, 0-100 scale
Direction from the sign of the strongest term (numeric drivers only).
term_raw <- tidy_df$term[tidy_df$term != "(Intercept)"]
term_t <- tidy_df$statistic[tidy_df$term != "(Intercept)"]
term_est <- tidy_df$estimate[tidy_df$term != "(Intercept)"]
term_p <- tidy_df$p.value[tidy_df$term != "(Intercept)"]
imp_rows <- lapply(model_drivers, function(dc) {
idx <- which(startsWith(term_raw, dc))
# Aliased (perfectly collinear) drivers have NA statistics — no
# estimable independent effect; exclude from the ranking instead of
# crashing on which.max(all-NA) -> integer(0).
idx <- idx[!is.na(term_t[idx])]
if (length(idx) == 0) return(NULL)
best <- idx[which.max(abs(term_t[idx]))]
data.frame(
semantic = dc,
driver = driver_names[[dc]],
abs_t = abs(term_t[best]),
direction = if (is.numeric(df_clean[[dc]])) {
if (term_est[best] > 0) "positive" else "negative"
} else "categorical",
p_value = signif(term_p[best], 3),
stringsAsFactors = FALSE
)
})
importance_df <- do.call(rbind, Filter(Negate(is.null), imp_rows))
if (!is.null(importance_df) && nrow(importance_df) > 0) {
importance_df <- importance_df[order(-importance_df$abs_t), , drop = FALSE]
max_t <- max(importance_df$abs_t, na.rm = TRUE)
importance_df$importance_score <- if (is.finite(max_t) && max_t > 0) {
round(100 * importance_df$abs_t / max_t, 1)
} else 0
rownames(importance_df) <- NULL
} else {
importance_df <- data.frame(semantic = character(0), driver = character(0),
abs_t = numeric(0), direction = character(0),
p_value = numeric(0), importance_score = numeric(0),
stringsAsFactors = FALSE)
}
top_driver_name <- if (nrow(importance_df) > 0) importance_df$driver[1] else "Unknown"
top_driver_sem <- if (nrow(importance_df) > 0) importance_df$semantic[1] else NA_character_Step 9: Outcome distribution — 7 pretty bins
brks <- pretty(range(df_clean$outcome, na.rm = TRUE), n = 7)
bins <- cut(df_clean$outcome, breaks = brks, include.lowest = TRUE, dig.lab = 10)
dist_tbl <- table(bins)
outcome_distribution_df <- data.frame(
outcome_range = as.character(names(dist_tbl)),
count = as.integer(dist_tbl),
stringsAsFactors = FALSE
)Step 10: Top numeric driver effect scatter — <=1000 rows
numeric_drivers_ranked <- importance_df$semantic[
sapply(importance_df$semantic, function(s) is.numeric(df_clean[[s]]))]
top_numeric_sem <- if (length(numeric_drivers_ranked) > 0) numeric_drivers_ranked[1] else NA_character_
top_numeric_driver <- if (!is.na(top_numeric_sem)) driver_names[[top_numeric_sem]] else NA_character_
if (!is.na(top_numeric_sem)) {
set.seed(42)
idx <- if (nrow(df_clean) > 1000) sample(nrow(df_clean), 1000) else seq_len(nrow(df_clean))
top_driver_effect_df <- data.frame(
driver_value = df_clean[[top_numeric_sem]][idx],
outcome = df_clean$outcome[idx],
stringsAsFactors = FALSE
)
top_driver_effect_df <- top_driver_effect_df[order(top_driver_effect_df$driver_value), ]
rownames(top_driver_effect_df) <- NULL
} else {
top_driver_effect_df <- data.frame(driver_value = numeric(0), outcome = numeric(0),
stringsAsFactors = FALSE)
}Step 11: Predicted vs actual and residuals — <=1000 samples
fitted_vals <- fitted(model)
resid_vals <- residuals(model)
set.seed(42)
ridx <- if (length(fitted_vals) > 1000) sample(length(fitted_vals), 1000) else seq_along(fitted_vals)
predicted_actual_df <- data.frame(
predicted = round(fitted_vals[ridx], 4),
actual = round(df_clean$outcome[ridx], 4),
stringsAsFactors = FALSE
)
rownames(predicted_actual_df) <- NULL
residuals_df <- data.frame(
fitted = round(fitted_vals[ridx], 4),
residual = round(resid_vals[ridx], 4),
stringsAsFactors = FALSE
)
rownames(residuals_df) <- NULLStep 12: KPI metrics (user-facing keys)
r2 <- glance_df$r.squared[1]
ar2 <- glance_df$adj.r.squared[1]
rmse <- sqrt(mean(resid_vals^2, na.rm = TRUE))
n_sig <- sum(coefficients_df$p_value < 0.05 & coefficients_df$term != "Intercept", na.rm = TRUE)
metrics <- list(
`Observations` = final_rows,
`R Squared` = round(r2, 3),
`Adjusted R Squared` = round(ar2, 3),
`RMSE` = round(rmse, 3),
`Significant Drivers` = n_sig,
`Top Driver` = top_driver_name
)Step 13: json_output machine channel
json_output <- list(
answer = paste0(
"Linear regression of ", outcome_name, " on ",
length(model_drivers), " driver(s) across ",
format(final_rows, big.mark = ","), " rows: R² = ", round(r2, 3),
" (adjusted ", round(ar2, 3), "), RMSE = ", round(rmse, 3), ". ",
top_driver_name, " is the strongest driver(", n_sig,
" of ", nrow(coefficients_df) - 1, " terms significant at p<0.05)."
),
cards = lapply(
c("tldr", "overview", "preprocessing", "outcome_distribution",
"coefficients_table", "driver_importance", "top_driver_effect",
"predicted_vs_actual", "residual_diagnostics"),
function(cid) list(id = cid, metrics = metrics)
)
)
list(
initial_rows = initial_rows,
final_rows = final_rows,
rows_removed = rows_removed,
outcome_name = outcome_name,
driver_names = driver_names,
model_drivers = model_drivers,
dropped_drivers = dropped_drivers,
df_clean = df_clean,
model = model,
glance_df = glance_df,
coefficients_df = coefficients_df,
importance_df = importance_df,
top_driver_name = top_driver_name,
top_driver_sem = top_driver_sem,
outcome_distribution_df = outcome_distribution_df,
top_driver_effect_df = top_driver_effect_df,
top_numeric_driver = top_numeric_driver,
predicted_actual_df = predicted_actual_df,
residuals_df = residuals_df,
rmse = rmse,
metrics = metrics,
json_output = json_output
)
}Compute shared resources
shared <- compute_shared(df, params, col_map)