Data Profile
Shows what every column holds, how much is missing, how the numbers are spread, and which columns need attention before you analyse them.
RUN DATE · 16 September 2026
DATA · 7,043 rows
What is in this customer file, and what needs attention before I analyse it?
- SummaryThe size of the file, how complete it is, and how many columns need attention.
- What is missingWhich columns have gaps, and how large they are.
- Column by columnWhat each column holds, in one line each.
- The spread of each numeric columnHow the values of each numeric column are spread.
- Which numbers move togetherPairs of numeric columns that carry the same information.
- The most common valuesWhich values fill the most repetitive column, and how unevenly.
- FindingsEach thing worth fixing before the file is analysed, and what to do.
- What this profile rests onEach condition this profile depends on, and whether it holds.
- How it was doneThe method, the columns used, and what to keep in mind.
Customer file: ready with two caveats
The file holds 7,043 rows across 21 columns with 99.99% of cells filled and 1 column flagged before analysis. customerID holds a distinct value per row and must be kept only for joining, never as a model input. TotalCharges has missing values in 0.16% of rows to resolve, and the Churn target is heavily skewed at 73.46% No, which should be addressed before modelling.
At a glance
One flag: customerID is an identifier
customerID holds one distinct value per row and must not enter a model; keep it only for joining.
No other columns require attention; all other mapped columns have sufficient rows and correct types.
One flag: customerID is an identifier
customerID contains a distinct value for each of the 7,043 rows, marking it as an identifier rather than a driver or outcome. This is the only quality finding. All other columns pass their checks and are ready to use.
No other columns require attention; all other mapped columns have sufficient rows and correct types.
One column has minor gaps, rest complete
TotalCharges alone has missing values (0.16% of rows); all other 20 columns are fully intact.
TotalCharges is the only bar shown; every other column has no missing values at all.
One column has minor gaps, rest complete
TotalCharges carries 11 missing values (0.16% of rows). All other 20 columns are complete. The gap is minor and isolated, so most analyses will proceed with a slight reduction in sample size only if TotalCharges enters the model. Decide whether imputation or row removal fits your next step.
TotalCharges is the only bar shown; every other column has no missing values at all.
What the data shows (2)
Churn heavily skewed toward retention
Churn is 73.46% No and 26.54% Yes; this imbalance should be addressed before modelling.
No at 73.46% dominates; Yes at 26.54% is the minority class that needs careful handling.
Churn heavily skewed toward retention
No accounts for 73.46% of Churn rows and Yes for 26.54%. This imbalance means churn cannot separate columns well in later analysis, since most rows cluster in one group. Check whether the minority class has enough rows to compare against the majority before running any driver analysis.
No at 73.46% dominates; Yes at 26.54% is the minority class that needs careful handling.
What the data shows (2)
Complete data, mostly binary, Churn is the target
All 21 columns present; Churn skews 73% No; most service columns are binary or three-level categories.
TotalCharges is nearly complete but not quite; all other columns are fully filled, and Churn heavily favours No over Yes.
Complete data, mostly binary, Churn is the target
Every column is mapped and complete except TotalCharges (99.84% filled). The outcome Churn is heavily skewed toward No (73.46%). Most service columns are categorical with two or three levels. Numeric columns tenure and MonthlyCharges show wide variety. The churn imbalance may affect model choice.
TotalCharges is nearly complete but not quite; all other columns are fully filled, and Churn heavily favours No over Yes.
What the data shows
TotalCharges skews right, others roughly symmetric
TotalCharges skews right with a long upper tail; tenure and MonthlyCharges are more symmetric.
TotalCharges: max 8,684.8 sits far above the median of 1,397, pulling the mean to 2,283.
TotalCharges skews right, others roughly symmetric
TotalCharges clearly skews right with extreme upper values pulling far from the median, while tenure, MonthlyCharges, and SeniorCitizen show more balanced spreads. Averages will overstate TotalCharges; use medians for that column. The positive correlations between tenure, MonthlyCharges, and TotalCharges suggest these drivers move together.
TotalCharges: max 8,684.8 sits far above the median of 1,397, pulling the mean to 2,283.
Weak correlations; no redundant columns
Strongest pair is tenure and MonthlyCharges (r=0.248); all other pairs are near zero, no duplicates.
All correlations stay loose; no pair approaches the strength that would flag a near-duplicate or data entry error.
Weak correlations; no redundant columns
Tenure and MonthlyCharges show the clearest pairing at r=0.248, still weak. SeniorCitizen and MonthlyCharges also move together slightly at r=0.220. All other pairs remain nearly flat. No pair is strong enough to suggest duplicate recording. The numeric columns carry distinct information and are ready for analysis.
All correlations stay loose; no pair approaches the strength that would flag a near-duplicate or data entry error.
Assumptions and method
All four checks hold
All four assumption checks hold; results describe all 7,043 rows with no silent omissions.
Holding: every mapped column profiled, enough rows, types from values, whole file, no strained or violated checks.
All four checks hold
Holding: every mapped column profiled (21 of 21), enough rows to describe a column (7,043 rows), types read from the values, and the whole file read rather than a sample. No checks are strained or violated, so nothing limits how far the results can be trusted.
Holding: every mapped column profiled, enough rows, types from values, whole file, no strained or violated checks.
A profile of 21 mapped columns over 7,043 rows. Each column's type is read from its values (a number held as text is reported as a finding rather than treated as a category), blank and NA both count as missing, and every mapped column appears in the summary. Findings are tested for one at a time: entirely empty, never varies, a distinct value per row, numbers held as text, a level under 1% of rows, and values beyond 3 interquartile ranges. Numeric pairs are Pearson correlations over the rows both columns fill. The repeats card shows the 8 most common values of 'Churn', the categorical column with the fewest distinct values.
7043 of 7043 rows · 21 columns → 1 findings
caveatAll 21 mapped columns profiled across all 7,043 rows; types inferred from values; findings tested individually.
The profile examined all 21 mapped columns across all 7,043 rows without sampling. Types were inferred from values, so a numeric code reads as a number. Findings are checked one at a time: empty columns, columns that never vary, identifiers with a distinct value per row, numbers stored as text, rare levels under 1%, and outliers beyond 3 interquartile ranges. Numeric pairs are Pearson correlations. A finding flags something to examine, not an error.
The code behind this report
The code that produced every figure in this report, exactly as it ran. Fingerprint 93a3175562476304. The same code on the same data gives the same report.
`standard_data_profile_v2` <- function(pf) {
`%||%` <- function(a, b) if (!is.null(a)) a else b
#' Readable figures (LAT-3181): whole numbers from a thousand up, one decimal from a hundred, two from one,
#' three significant figures below one.
tidy <- function(x) {
x <- as.numeric(x)
ifelse(is.na(x), NA_real_,
ifelse(abs(x) >= 1000, round(x, 0),
ifelse(abs(x) >= 100, round(x, 1),
ifelse(abs(x) >= 1, round(x, 2), signif(x, 3)))))
}
inputs <- pf$taskList$inputs
params <- inputs$module_parameters %||% list()
question <- (inputs$userContext %||% list())$objective %||%
"What is in this file, column by column, and what needs attention before analysing it?"
col_map <- inputs$column_mapping %||% list()
df <- renderObject.taskFunction.init(inputs, col_map) # df has SEMANTIC names
human <- function(sem) { v <- col_map[[sem]]; if (is.null(v) || !nzchar(as.character(v))) sem else as.character(v) }
#' ## The column series
#' A series arrives as column_1, column_2, ... and is sorted NUMERICALLY: sorted as text, column_10 lands before
#' column_2 and every row of the profile is in the wrong order (the sibling of LAT-3210, where a prefix match
#' mislabelled terms once a series passed nine members).
cols <- grep("^column_[0-9]+$", names(df), value = TRUE)
cols <- cols[order(as.integer(sub("^column_", "", cols)))]
if (!length(cols)) stop("column_mapping must map at least one column to profile (column_1); this tool profiles the columns you give it")
n_mapped <- length(cols)
n_rows <- nrow(df)
if (n_rows < 5) stop(sprintf("Only %d rows: a profile needs at least 5 to describe a column.", n_rows))
max_top <- suppressWarnings(as.integer(params$max_top_values %||% 8L))
if (is.na(max_top) || max_top < 3 || max_top > 20) stop("module_parameters$max_top_values must be an integer between 3 and 20")
rare_pct <- suppressWarnings(as.numeric(params$rare_level_pct %||% 1))
if (is.na(rare_pct) || rare_pct < 0.1 || rare_pct > 10) stop("module_parameters$rare_level_pct must be a number between 0.1 and 10")
#' ## One pass per column: what it holds, how much of it, and what is wrong with it.
#' A column is NUMBER, DATE, TEXT or EMPTY, read from the values rather than from the declared type, because a
#' number held as text is one of the findings this tool exists to report.
is_blank <- function(x) is.na(x) | trimws(as.character(x)) == ""
prof <- list(); flags <- list(); numeric_vals <- list(); cat_cols <- character(0)
add_flag <- function(col, flag, statistic, todo)
flags[[length(flags) + 1]] <<- data.frame(column = col, flag = flag, statistic = statistic, what_to_do = todo,
stringsAsFactors = FALSE)
for (sem in cols) {
nm <- human(sem); v <- df[[sem]]
blank <- is_blank(v); n_filled <- sum(!blank); filled_pct <- 100 * n_filled / n_rows
vals <- v[!blank]
distinct <- length(unique(as.character(vals)))
type <- "text"; summary_txt <- NA_character_; was_text_number <- FALSE
if (n_filled == 0) {
type <- "empty"; summary_txt <- "nothing recorded"
} else if (is.numeric(vals)) {
type <- "number"; numeric_vals[[nm]] <- as.numeric(vals)
summary_txt <- sprintf("median %s", format(tidy(stats::median(as.numeric(vals)))))
} else {
conv <- suppressWarnings(as.numeric(as.character(vals)))
if (sum(!is.na(conv)) >= 0.95 * length(conv) && distinct > 2) {
type <- "number"; was_text_number <- TRUE; numeric_vals[[nm]] <- conv[!is.na(conv)]
summary_txt <- sprintf("median %s", format(tidy(stats::median(conv, na.rm = TRUE))))
} else {
d <- suppressWarnings(as.Date(as.character(vals), optional = TRUE))
if (sum(!is.na(d)) >= 0.95 * length(d) && distinct > 2) {
type <- "date"; summary_txt <- sprintf("%s to %s", format(min(d, na.rm = TRUE)), format(max(d, na.rm = TRUE)))
} else {
type <- "text"; cat_cols <- c(cat_cols, nm)
tb <- sort(table(as.character(vals)), decreasing = TRUE)
summary_txt <- sprintf("most often %s (%s%%)", names(tb)[1], format(tidy(100 * tb[[1]] / n_filled)))
}
}
}
prof[[nm]] <- list(column = nm, type = type, filled_pct = tidy(filled_pct), missing_pct = tidy(100 - filled_pct),
distinct = distinct, summary = summary_txt)
#' ## The findings. Each is a DIFFERENT KIND of problem, which is why they are a table and never ranked against
#' one another (LAT-3233: they are not on one scale, so there is no worst).
if (type == "empty") {
add_flag(nm, "entirely empty", "0 values in the whole column", "drop it, or find out where the values went")
} else {
if (n_filled < n_rows) {
share <- 100 - filled_pct
if (share >= 5) add_flag(nm, "values missing", sprintf("%s%% of rows", format(tidy(share))),
"decide whether to drop those rows or fill them, and say which in any analysis")
}
if (distinct == 1) add_flag(nm, "never varies", sprintf("one value: %s", as.character(vals)[1]),
"it cannot explain or separate anything; leave it out of models")
if (distinct == n_filled && n_filled >= 20 && type != "number")
add_flag(nm, "a distinct value per row", sprintf("%d distinct in %d rows", distinct, n_filled),
"this is an identifier: keep it for joining, never as an input to a model")
if (was_text_number) add_flag(nm, "numbers held as text", sprintf("%d of %d parse as numbers", sum(!is.na(conv)), length(conv)),
"convert it before analysing, or a tool will treat each value as a category")
if (type == "text" && n_filled > 0 && distinct < n_filled) {
tb <- table(as.character(vals)); rare <- names(tb)[100 * tb / n_filled < rare_pct]
if (length(rare)) add_flag(nm, "a level too rare to compare", sprintf("%d level(s) under %s%% of rows, smallest %s",
length(rare), format(rare_pct), names(sort(tb))[1]),
"group the rare levels into Other before comparing, or the comparison rests on a handful of rows")
}
if (type == "number") {
x <- numeric_vals[[nm]]
qs <- stats::quantile(x, c(0.25, 0.75), names = FALSE); iqr <- qs[2] - qs[1]
if (!is.na(iqr) && iqr > 0) {
beyond <- x < qs[1] - 3 * iqr | x > qs[2] + 3 * iqr
far <- sum(beyond)
share <- far / length(x)
if (far > 0 && share > 0.01) {
#' MANY points beyond the fence is a SHAPE, not a list of mistakes: an ordinary skewed measure (spend,
#' duration, income) puts a few per cent out there by construction. Reporting them as values to check
#' buries the column that has ONE value in the wrong place.
add_flag(nm, "a long tail", sprintf("%s%% of values beyond 3 IQR, median %s and largest %s",
format(tidy(100 * share)), format(tidy(stats::median(x))), format(tidy(max(x)))),
"read the median rather than the mean, and consider a log scale before averaging or modelling")
} else if (far > 0) {
add_flag(nm, "values far from the rest", sprintf("%d value%s beyond 3 IQR, largest %s against a median of %s",
far, if (far > 1) "s" else "", format(tidy(max(x))), format(tidy(stats::median(x)))),
"check them before using an average; a single far value moves a mean and not a median")
}
}
}
}
}
prof_df <- do.call(rbind, lapply(prof, function(p) data.frame(column = p$column, type = p$type, filled_pct = p$filled_pct,
distinct = p$distinct, summary = p$summary, stringsAsFactors = FALSE)))
rownames(prof_df) <- NULL
#' EVERY MAPPED COLUMN APPEARS. The failure mode of a profiler is OMISSION, which logs nothing and looks like a
#' clean file, so the count is asserted here rather than left to the reader (LAT-3204 logging plan).
if (nrow(prof_df) != n_mapped)
stop(sprintf("internal: %d columns mapped but %d profiled; every mapped column must appear", n_mapped, nrow(prof_df)))
miss_df <- data.frame(column = prof_df$column, missing_pct = tidy(100 - prof_df$filled_pct), stringsAsFactors = FALSE)
miss_df <- miss_df[miss_df$missing_pct > 0, , drop = FALSE]
miss_df <- miss_df[order(-miss_df$missing_pct), , drop = FALSE]
#' ## The numeric columns' spread, as the values themselves, sampled so a large file does not carry a large object.
dist_df <- NULL
if (length(numeric_vals)) {
set.seed(42)
dist_df <- do.call(rbind, lapply(names(numeric_vals), function(nm) {
x <- numeric_vals[[nm]]; if (length(x) > 1000) x <- x[sort(sample.int(length(x), 1000))]
data.frame(column = nm, value = tidy(x), stringsAsFactors = FALSE)
}))
}
#' ## Which numbers move together: every numeric pair, ordered by ABSOLUTE correlation, which the label says.
pair_df <- NULL
if (length(numeric_vals) >= 2) {
nms <- names(numeric_vals); combos <- utils::combn(seq_along(nms), 2)
rows <- lapply(seq_len(ncol(combos)), function(j) {
a <- numeric_vals[[combos[1, j]]]; b <- numeric_vals[[combos[2, j]]]
n <- min(length(a), length(b)); if (n < 5) return(NULL)
r <- suppressWarnings(stats::cor(a[seq_len(n)], b[seq_len(n)]))
if (is.na(r)) return(NULL)
data.frame(pair = paste(nms[combos[1, j]], "and", nms[combos[2, j]]), r = tidy(r), n = n, stringsAsFactors = FALSE)
})
pair_df <- do.call(rbind, Filter(Negate(is.null), rows))
if (!is.null(pair_df)) {
pair_df <- pair_df[order(-abs(pair_df$r)), , drop = FALSE]
pair_df <- utils::head(pair_df, 10); rownames(pair_df) <- NULL
}
}
#' ## What repeats: the categorical column with the fewest distinct values relative to its rows.
top_df <- NULL; top_col <- NA_character_
if (length(cat_cols)) {
cand <- vapply(cat_cols, function(nm) prof[[nm]]$distinct, numeric(1))
cand <- cand[cand >= 2 & cand < n_rows]
if (length(cand)) {
top_col <- names(cand)[which.min(cand)]
sem <- cols[vapply(cols, function(s) human(s) == top_col, logical(1))][1]
v <- df[[sem]]; v <- as.character(v[!is_blank(v)])
tb <- sort(table(v), decreasing = TRUE)
top_df <- data.frame(value = names(tb)[seq_len(min(max_top, length(tb)))],
share_pct = tidy(100 * as.numeric(tb[seq_len(min(max_top, length(tb)))]) / length(v)),
stringsAsFactors = FALSE)
}
}
flags_df <- if (length(flags)) do.call(rbind, flags) else
data.frame(column = character(0), flag = character(0), statistic = character(0), what_to_do = character(0), stringsAsFactors = FALSE)
rownames(flags_df) <- NULL
n_flagged <- length(unique(flags_df$column))
cells_filled <- 100 * sum(prof_df$filled_pct) / (100 * n_mapped)
checks_df <- data.frame(
check = c("Every mapped column profiled", "Enough rows to describe a column", "Types read from the values", "The whole file, not a sample"),
statistic = c(sprintf("%d of %d mapped columns", nrow(prof_df), n_mapped),
sprintf("%s rows", format(n_rows, big.mark = ",")),
sprintf("%d number, %d text, %d date, %d empty",
sum(prof_df$type == "number"), sum(prof_df$type == "text"), sum(prof_df$type == "date"), sum(prof_df$type == "empty")),
"every row given to the tool was read"),
p_value = NA_real_,
verdict = c(if (nrow(prof_df) == n_mapped) "holds" else "violated",
if (n_rows >= 100) "holds" else if (n_rows >= 30) "strained" else "violated",
"holds", "holds"),
note = c("a profiler that silently skips a column is the failure this check exists for",
"under 30 rows a distinct count and a median say little",
"a number held as text is reported as a finding, not assumed to be a category",
"figures describe the rows given; a sampled upload would describe the sample"),
stringsAsFactors = FALSE)
excluded <- character(0)
unmapped <- setdiff(if (is.data.frame(inputs$dataset)) names(inputs$dataset) else character(0), as.character(unlist(col_map)))
if (length(unmapped)) excluded <- c(excluded, sprintf("columns not mapped, so not profiled: %s", paste(unmapped, collapse = ", ")))
method <- paste0(
"A profile of ", n_mapped, " mapped column", if (n_mapped > 1) "s" else "", " over ", format(n_rows, big.mark = ","), " rows. ",
"Each column's type is read from its values (a number held as text is reported as a finding rather than treated as a category), ",
"blank and NA both count as missing, and every mapped column appears in the summary. ",
"Findings are tested for one at a time: entirely empty, never varies, a distinct value per row, numbers held as text, ",
"a level under ", format(rare_pct), "% of rows, and values beyond 3 interquartile ranges. ",
if (length(numeric_vals) >= 2) "Numeric pairs are Pearson correlations over the rows both columns fill. " else "",
if (!is.na(top_col)) paste0("The repeats card shows the ", max_top, " most common values of '", top_col, "', the categorical column with the fewest distinct values. ") else "",
if (length(excluded)) paste0("Excluded: ", paste(excluded, collapse = "; "), ".") else "")
assumptions <- list(
"A profile describes the rows it was given; it cannot tell you whether those rows are the whole file.",
"Types are inferred from the values, so a column of digits that means a code (a postcode, an account number) reads as a number.",
"A finding is a thing to look at, not an error: a skewed column, a rare level and an identifier are all ordinary.",
"The findings are of different kinds and are not ranked against one another; there is no worst column.")
answer <- paste0(
"This file has ", format(n_rows, big.mark = ","), " rows and ", n_mapped, " profiled column", if (n_mapped > 1) "s" else "",
", ", format(tidy(cells_filled)), "% of cells filled. ",
if (n_flagged == 0) "No column needs attention before analysis."
else sprintf("%d column%s %s attention: %s.", n_flagged, if (n_flagged > 1) "s" else "",
if (n_flagged > 1) "need" else "needs", paste(unique(flags_df$column), collapse = ", ")),
if (!is.null(pair_df) && nrow(pair_df) && abs(pair_df$r[1]) >= 0.8)
sprintf(" %s move together almost exactly (correlation %s), so they may be one measure recorded twice.",
pair_df$pair[1], format(pair_df$r[1])) else "")
results <- list()
results$summary_metrics <- place_metric(list(n = n_rows, n_columns = n_mapped, cells_filled_pct = tidy(cells_filled),
columns_flagged = n_flagged), lead = "n_columns", place = "summary_metrics")
results$column_summary <- place_table(prof_df, place = "column_summary")
#' A CONDITIONAL PLACE IS WRITTEN EITHER WAY (LAT-3102), and an EMPTY finding table is a real answer, not an absence.
if (nrow(miss_df)) {
results$missing_by_column <- place_comparison(miss_df, category = "column", value = "missing_pct", place = "missing_by_column")
} else {
results$missing_by_column <- place_dropped("every profiled column is complete: no column has a missing value, so there is nothing to chart", place = "missing_by_column")
}
if (!is.null(dist_df)) {
results$distributions <- place_distribution(dist_df, x = "value", series = "column", place = "distributions")
} else {
results$distributions <- place_dropped("no column in this file holds numbers, so there is no spread to draw; the column summary above says what each column does hold", place = "distributions")
}
if (!is.null(pair_df) && nrow(pair_df)) {
results$correlation_pairs <- place_table(pair_df, place = "correlation_pairs")
} else {
results$correlation_pairs <- place_dropped(sprintf("%s numeric column%s in this file, and a correlation needs two",
if (length(numeric_vals) == 0) "no" else "only one", if (length(numeric_vals) == 1) "" else "s"), place = "correlation_pairs")
}
if (!is.null(top_df)) {
results$top_values <- place_comparison(top_df, category = "value", value = "share_pct", place = "top_values")
} else {
results$top_values <- place_dropped("no categorical column in this file repeats a value, so there is nothing to count; a column with a distinct value per row is reported as an identifier in the findings", place = "top_values")
}
if (nrow(flags_df)) {
results$quality_flags <- place_table(flags_df, place = "quality_flags")
} else {
results$quality_flags <- place_dropped(sprintf(
"no column needs attention: all %d profiled column%s are complete, vary, and hold no value far from the rest",
n_mapped, if (n_mapped > 1) "s" else ""), place = "quality_flags")
}
results$assumption_checks <- place_table(checks_df, place = "assumption_checks")
results$profile_method <- list(kind = "metric", values = list(
method = method, n_in = n_rows, n_used = n_rows, excluded = as.list(unname(excluded)), assumptions = assumptions,
x_column = sprintf("%d columns", n_mapped), y_column = sprintf("%d findings", nrow(flags_df))),
value_order = list("n_used", "n_in"))
objects <- list() # filled by the object layer, not here
list(answer = answer, method = method, n = n_rows, results = results, objects = objects,
json_output = list(answer = answer, method = method, n = n_rows))
}