Anomaly Detection
Finds the rows that do not fit the usual pattern across several measures, including rows whose values are only unusual in combination, and says what makes each one stand out.
RUN DATE · 14 September 2026
DATA · 500 rows
Which of these 500 transactions look unusual, and what makes each one stand out?
- SummaryHow many rows were flagged, and how that compares with chance.
- How far the unusual rows sitEvery row's distance from the usual pattern, flagged rows in their own colour.
- Map of the rowsAll rows on one picture, the flagged ones highlighted.
- What the flagged rows stand out onThe measure behind each flag, or the combination when no single value is extreme.
- The unusual rows, measure by measureHow far each flagged row sits from typical on every measure.
- The flagged rowsEach unusual row, what it stands out on, and by how much.
- What the results rely onEach condition the anomaly scores depend on, and whether it holds.
- How it was doneThe method, the data used, and what to keep in mind.
What stands out
Flagged rows sit well beyond the usual tail
Flagged transactions sit in a distinct upper tail, well separated from the bulk of usual rows.
The separation between the flagged rows in their own colour and the main cluster of usual rows below.
Flagged rows sit well beyond the usual tail
The usual rows cluster tightly in the lower range with a moderate upper tail of their own, while the flagged rows sit clearly separated and well beyond this tail. The distinction is sharp: no flagged row overlaps with the bulk, and the gap between the highest usual row and the lowest flagged row is substantial. This clear separation means each flagged transaction stands out distinctly from the ordinary pattern. Check the flagged rows next to understand what specific features drive their unusual character.
The separation between the flagged rows in their own colour and the main cluster of usual rows below.
Flagged rows scatter across edges in separate directions
Unusual transactions sit at the cloud's edges in different directions, pointing to independent causes rather than one shared pattern.
The flagged points spread around the perimeter of the scatter plot, not clustered in one area or direction.
Flagged rows scatter across edges in separate directions
The flagged rows scatter clearly across different edges of the principal component cloud: some extend far along the first component, others far along the second, and a few in mixed directions. This pattern indicates independent exceptions, each unusual in different ways across the measured features. The robust covariance holds, so the map captures the actual structure. Check the original features of each flagged row to understand what combination makes each one stand out from the typical transaction profile.
The flagged points spread around the perimeter of the scatter plot, not clustered in one area or direction.
Why they stand out
Session Duration dominates flagged anomalies
Session Duration stands out most among flagged transactions; combinations account for smaller share of unusual patterns.
The tallest bar shows Session Duration, which carries the largest share of flagged rows clearly ahead of others.
Session Duration dominates flagged anomalies
On this card's data, Session Duration has the largest share, not most, of the total.
The tallest bar shows Session Duration, which carries the largest share of flagged rows clearly ahead of others.
Transaction Amount and Session Duration drive anomalies
Two transactions stand out as extreme on multiple measures; Session Duration and Transaction Amount carry the strongest signals.
T0323 shows extreme Session Duration, T0301 shows extreme Transaction Amount and Items Purchased, and T0010 shows extreme Account Age.
Transaction Amount and Session Duration drive anomalies
T0323 is extreme on Session Duration, far above typical. T0301 is extreme on both Transaction Amount and Items Purchased, both well above typical. T0010 is extreme on Account Age, far above typical. T0347 shows a secondary extreme on Session Duration, also well above typical. Several other transactions show moderate departures on single measures: T0070 is below typical on Session Duration but above on Items Purchased, T0238 and T0087 depart on Items Purchased, and T0489 is notably below typical on Account Age. The clearest anomalies are T0323 and T0301, each extreme on multiple measures.
T0323 shows extreme Session Duration, T0301 shows extreme Transaction Amount and Items Purchased, and T0010 shows extreme Account Age.
The numbers
Most transactions typical, a few stand far apart
Most flagged rows show extreme values in single measures, mostly above typical, with a few below.
Most unusual rows spike above typical values, but a few drop far below, and some combine modest deviations across multiple measures.
Most transactions typical, a few stand far apart
T0323 stands out most on Session Duration, far above typical, followed by T0347 on the same measure and T0301 on Transaction Amount. T0070 and T0334 fall far below typical on Session Duration. T0489 shows an Account Age value that is extremely low and unusual on this data. T0087, T0042, and T0355 stand out on combinations of measures rather than any single extreme. The flagged rows are clearly separated from the typical range, and all checks hold.
Most unusual rows spike above typical values, but a few drop far below, and some combine modest deviations across multiple measures.
Assumptions and method
All five checks hold
All five assumption checks hold, so nothing here limits how far the results can be trusted.
Holding: enough rows per measure, robust covariance estimated, few filled cells, flag rate plausible, measures not heavily skewed.
All five checks hold
The checks that hold: enough rows per measure, robust covariance estimated, few filled cells, flag rate plausible, measures not heavily skewed.
Holding: enough rows per measure, robust covariance estimated, few filled cells, flag rate plausible, measures not heavily skewed.
Anomaly detection on 4 measures (Transaction Amount, Session Duration, Items Purchased, Account Age) across 500 rows, scored by robust Mahalanobis distance (minimum covariance determinant), flagged above the 0.999 chi-square quantile (4 degrees of freedom, distance 4.3); excluded or filled: Row Number (a running index); 5 blank cells filled with the measure's median; not used: Notes, Region Code (not mapped); each flag names the measure with the largest robust z-score (median and MAD), or the combination when none reaches 3; about 0.5 ordinary rows pass by chance; the map shows the first two principal components of the robust z-scores (48.6% and 27.9% of variance).
500 of 500 rows · Transaction Amount, Session Duration, Items Purchased, Account Age → 12 rows flagged
caveatRobust distance on four measures flagged rare rows; threshold and filled blanks limit precision.
The method scored all rows by robust Mahalanobis distance across Transaction Amount, Session Duration, Items Purchased, and Account Age, then flagged those in the upper tail. All checks held, but two limits shape what the flags mean: the threshold is set to catch about one ordinary row per thousand by chance, so some flags may not reflect true anomalies; and five blank cells were filled with medians, which makes rows with missing data look more typical than they truly are. A flagged row warrants checking against its source before treating it as wrong.
The code behind this report
The code that produced every figure in this report, exactly as it ran. Fingerprint dcacc0ffb2929dfc. The same code on the same data gives the same report.
`standard_anomaly_detection_v2` <- function(pf) {
`%||%` <- function(a, b) if (!is.null(a)) a else b
#' Readable figures (LAT-3180, LAT-3181): whole numbers from a thousand up, one decimal from a hundred, two from
#' one, three significant figures below one. A cell carries what the value needs, not what R prints.
tidy <- function(x) {
x <- as.numeric(x)
ifelse(is.na(x), NA_real_,
ifelse(abs(x) >= 1000, round(x, 0),
ifelse(abs(x) >= 100, round(x, 1),
ifelse(abs(x) >= 1, round(x, 2), signif(x, 3)))))
}
inputs <- pf$taskList$inputs
params <- inputs$module_parameters %||% list()
# THE QUESTION this tool answers: the customer's objective, verbatim, when given.
question <- (inputs$userContext %||% list())$objective %||%
"Which rows in this data are unusual, and what makes each one stand out?"
#' ## Column mapping
#' Two to eight numeric `feature_N` columns to check together, and optionally a `row_label` column that names each row
#' (an order, transaction or patient id). Semantic names inside; the customer's headers live in `col_map`.
col_map <- inputs$column_mapping %||% list()
df <- renderObject.taskFunction.init(inputs, col_map) # df has SEMANTIC names
human <- function(sem) {
v <- col_map[[sem]]
if (is.null(v) || !nzchar(as.character(v))) sem else as.character(v)
}
n_in <- nrow(df)
feat_cols <- grep("^feature_[0-9]+$", names(df), value = TRUE)
feat_cols <- feat_cols[order(as.integer(sub("^feature_", "", feat_cols)))]
if (length(feat_cols) < 2) stop("column_mapping must map at least two numeric columns to feature_1, feature_2, ... (the measures to check together).")
has_label <- "row_label" %in% names(df)
label_name <- if (has_label) human("row_label") else "row"
raw_names <- local({
ds <- inputs$dataset %||% inputs$df
if (is.data.frame(ds)) return(names(ds))
if (is.list(ds) && length(ds) > 0) {
rows <- ds[seq_len(min(length(ds), 50))]
nm <- unique(unlist(lapply(rows, function(r) if (is.list(r)) names(r) else NULL)))
if (length(nm)) return(nm)
if (!is.null(names(ds)) && all(nzchar(names(ds)))) return(names(ds))
}
character(0)
})
mapped_actual <- unique(as.character(unlist(col_map)))
ignored_cols <- setdiff(raw_names, unique(c(mapped_actual, make.names(mapped_actual))))
#' ## Parameters
#' `threshold`: the chi-square quantile a row's distance must pass to be flagged: 0.99, 0.995 or 0.999 (the default,
#' about one false flag per thousand ordinary rows).
thr_p <- suppressWarnings(as.numeric(params$threshold %||% 0.999))
if (is.na(thr_p) || !(thr_p %in% c(0.99, 0.995, 0.999))) stop("module_parameters$threshold must be 0.99, 0.995 or 0.999")
#' ## Data preparation
#' Each mapped measure is read as a number (95% rule). A measure that is not numeric, is constant, or is a running
#' index (every value distinct and one step apart) is excluded and named, because an index is not a trait and would
#' flag the first and last rows. Blank cells in a kept measure are filled with that measure's median and counted. At
#' least two usable measures and 30 rows are required.
excluded_cols <- character(0); why <- character(0)
X <- list()
for (fc in feat_cols) {
v <- df[[fc]]; nm <- human(fc)
if (!is.numeric(v)) {
ch <- trimws(as.character(v)); nb <- !is.na(ch) & ch != ""
conv <- suppressWarnings(as.numeric(ch))
if (sum(nb) == 0 || sum(!is.na(conv[nb])) < 0.95 * sum(nb)) { excluded_cols <- c(excluded_cols, nm); why <- c(why, "not numeric"); next }
v <- conv
}
v <- as.numeric(v); v[!is.finite(v)] <- NA
ok <- v[!is.na(v)]
if (length(ok) < 3 || isTRUE(stats::sd(ok) == 0)) { excluded_cols <- c(excluded_cols, nm); why <- c(why, "constant"); next }
if (length(unique(ok)) == length(ok) && all(abs(diff(sort(ok)) - 1) < 1e-9)) { excluded_cols <- c(excluded_cols, nm); why <- c(why, "a running index"); next }
X[[nm]] <- v
}
if (length(X) < 2) stop(sprintf("Fewer than two usable numeric measures remained (excluded: %s); anomaly detection across measures needs at least two.",
if (length(excluded_cols)) paste(paste0(excluded_cols, " (", why, ")"), collapse = ", ") else "none"))
X <- as.data.frame(X, check.names = FALSE, stringsAsFactors = FALSE)
if (nrow(X) < 30) stop(sprintf("Only %d rows; anomaly detection needs at least 30 to know what usual looks like.", nrow(X)))
n_filled <- sum(is.na(X))
for (j in seq_along(X)) X[[j]][is.na(X[[j]])] <- stats::median(X[[j]], na.rm = TRUE)
M <- as.matrix(X); k <- ncol(M); n <- nrow(M)
labels <- if (has_label) { l <- trimws(as.character(df$row_label)); l[is.na(l) | l == ""] <- paste0("Row ", which(is.na(l) | l == "")); l } else paste0("Row ", seq_len(n))
#' ## Robust per-measure z-scores (median and MAD), which the outliers they look for cannot drag
med <- apply(M, 2, stats::median)
sc <- apply(M, 2, stats::mad)
sc[sc == 0] <- apply(M, 2, function(x) stats::IQR(x) / 1.349)[sc == 0]
sc[sc == 0] <- apply(M, 2, stats::sd)[sc == 0]
Z <- sweep(sweep(M, 2, med, "-"), 2, sc, "/")
#' ## The anomaly score: robust Mahalanobis distance
#' Distance from a robust centre and covariance (minimum covariance determinant, MASS::cov.rob, seed 42), so a row is
#' flagged for an unusual COMBINATION of values even when no single measure is extreme, and a cluster of outliers
#' cannot hide itself by inflating the covariance. Falls back to the classical covariance, then to a ridge, then to the
#' largest robust z-score, and the method says which ran. Flagged: squared distance above the chi-square quantile.
score_method <- NA_character_; d2 <- NULL
if (requireNamespace("MASS", quietly = TRUE)) {
set.seed(42)
rob <- tryCatch(MASS::cov.rob(M, method = "mcd"), error = function(e) NULL)
if (!is.null(rob)) d2 <- tryCatch(stats::mahalanobis(M, rob$center, rob$cov), error = function(e) NULL)
if (!is.null(d2)) score_method <- "robust Mahalanobis distance (minimum covariance determinant)"
}
if (is.null(d2)) {
d2 <- tryCatch(stats::mahalanobis(M, colMeans(M), stats::cov(M)), error = function(e) NULL)
if (!is.null(d2)) score_method <- "classical Mahalanobis distance (the robust covariance could not be estimated)"
}
if (is.null(d2)) {
S <- stats::cov(M); S <- S + diag(1e-6 * mean(diag(S)), k)
d2 <- tryCatch(stats::mahalanobis(M, colMeans(M), S), error = function(e) NULL)
if (!is.null(d2)) score_method <- "ridge-regularised Mahalanobis distance (the covariance was singular)"
}
if (is.null(d2)) {
score <- apply(abs(Z), 1, max); cutoff <- 5; score_method <- "largest robust z-score (no covariance could be estimated)"
} else {
score <- sqrt(pmax(d2, 0)); cutoff <- sqrt(stats::qchisq(thr_p, df = k))
}
flagged <- score > cutoff
n_flag <- sum(flagged)
dom_j <- apply(abs(Z), 1, which.max)
dom_z <- Z[cbind(seq_len(n), dom_j)]
#' ## The frames for the places
status <- ifelse(flagged, "Flagged", "Within the usual range")
set.seed(42)
keep_idx <- if (n > 2000) sort(unique(c(which(flagged), sample(which(!flagged), 2000 - min(2000, n_flag))))) else seq_len(n)
dist_df <- data.frame(score = tidy(score[keep_idx]), status = status[keep_idx], stringsAsFactors = FALSE)
pc <- tryCatch(stats::prcomp(Z, center = TRUE, scale. = FALSE), error = function(e) NULL)
pc_var <- if (!is.null(pc)) round(100 * pc$sdev^2 / sum(pc$sdev^2), 1) else c(NA, NA)
map_idx <- if (n > 1000) sort(unique(c(which(flagged), sample(which(!flagged), 1000 - min(1000, n_flag))))) else seq_len(n)
map_df <- if (!is.null(pc)) data.frame(pc1 = round(pc$x[map_idx, 1], 3), pc2 = round(pc$x[map_idx, 2], 3), status = status[map_idx], stringsAsFactors = FALSE) else
data.frame(pc1 = round(Z[map_idx, 1], 3), pc2 = round(Z[map_idx, 2], 3), status = status[map_idx], stringsAsFactors = FALSE)
ord <- order(-score)
top <- ord[flagged[ord]][seq_len(min(n_flag, 15))]
combo <- abs(dom_z) < 3
top_df <- data.frame(
row = labels[top], score = tidy(score[top]),
stands_out_on = ifelse(combo[top], "the combination of measures", colnames(M)[dom_j[top]]),
value = ifelse(combo[top], NA_real_, tidy(M[cbind(top, dom_j[top])])),
typical = ifelse(combo[top], NA_real_, tidy(med[dom_j[top]])),
robust_z = round(dom_z[top], 1),
direction = ifelse(combo[top], "no single measure is extreme", ifelse(dom_z[top] > 0, "above typical", "below typical")),
stringsAsFactors = FALSE)
drivers_df <- if (n_flag > 0) {
who <- ifelse(combo[flagged], "Combination of measures", colnames(M)[dom_j[flagged]])
tb <- sort(table(who), decreasing = TRUE)
data.frame(stands_out_on = names(tb), flagged_rows = as.integer(tb), stringsAsFactors = FALSE)
} else NULL
prof_rows <- ord[flagged[ord]][seq_len(min(n_flag, 10))]
profile_df <- if (length(prof_rows)) data.frame(
row = rep(labels[prof_rows], times = k), measure = rep(colnames(M), each = length(prof_rows)),
robust_z = round(as.vector(Z[prof_rows, , drop = FALSE]), 1), stringsAsFactors = FALSE) else NULL
#' ## Assumption checks (LAT-3138)
#' Skewness on the rows WITHIN the usual range: the flagged rows are the tail by construction, so measuring skew with
#' them in made the check read violated on every dataset that had real anomalies (LAT-3187, first run).
usual <- M[!flagged, , drop = FALSE]
skew <- apply(if (nrow(usual) >= 10) usual else M, 2, function(x) { s <- stats::sd(x); if (s == 0) 0 else mean((x - mean(x))^3) / s^3 })
n_skewed <- sum(abs(skew) > 2)
flag_share <- n_flag / n
expected_false <- (1 - thr_p) * n
checks_df <- data.frame(
check = c("Enough rows per measure", "Robust covariance estimated", "Few filled cells", "Flag rate plausible", "Measures not heavily skewed"),
statistic = c(paste0(round(n / k), " rows per measure"), if (grepl("^robust", score_method)) "minimum covariance determinant" else score_method,
paste0(round(100 * n_filled / (n * k), 2), "% of cells filled with medians"),
paste0(round(100 * flag_share, 1), "% of rows flagged"),
paste0(n_skewed, " of ", k, " measures with skewness beyond 2 among usual rows")),
p_value = c("", "", "", "", ""),
verdict = c(if (n / k >= 20) "holds" else if (n / k >= 10) "strained" else "violated",
if (grepl("^robust", score_method)) "holds" else if (grepl("^classical", score_method)) "strained" else "violated",
if (n_filled / (n * k) < 0.01) "holds" else if (n_filled / (n * k) < 0.05) "strained" else "violated",
if (flag_share <= 0.05) "holds" else if (flag_share <= 0.15) "strained" else "violated",
if (n_skewed == 0) "holds" else if (n_skewed <= k / 2) "strained" else "violated"),
note = c("few rows per measure make the covariance, and every distance, unstable",
"without a robust covariance a cluster of outliers can widen the usual range and hide itself",
"filled cells pull those rows toward the centre, so a row with a blank is less likely to be flagged",
"a high flag rate usually means the data is several groups, not one usual pattern with exceptions",
"a long-tailed measure flags its tail as unusual even when the tail is ordinary for that measure"),
stringsAsFactors = FALSE)
#' ## Method, assumptions, answer
excluded <- c(if (length(excluded_cols)) paste0(excluded_cols, " (", why, ")"),
if (n_filled > 0) sprintf("%d blank cell%s filled with the measure's median", n_filled, if (n_filled > 1) "s" else ""))
method <- paste0(
"Anomaly detection on ", k, " measures (", paste(colnames(M), collapse = ", "), ") across ", n, " rows, scored by ",
score_method, if (!is.na(cutoff) && !grepl("z-score", score_method)) sprintf(", flagged above the %s chi-square quantile (%d degrees of freedom, distance %s)", format(thr_p), k, format(tidy(cutoff))) else ", flagged above 5",
if (length(excluded)) paste0("; excluded or filled: ", paste(excluded, collapse = "; ")) else "",
if (length(ignored_cols)) paste0("; not used: ", paste(utils::head(ignored_cols, 12), collapse = ", "), " (not mapped)") else "",
"; each flag names the measure with the largest robust z-score (median and MAD), or the combination when none reaches 3",
sprintf("; about %s ordinary rows pass by chance", format(tidy(expected_false))),
if (!is.null(pc)) sprintf("; the map shows the first two principal components of the robust z-scores (%s%% and %s%% of variance)", pc_var[1], pc_var[2]) else "", ".")
assumptions <- list(
"Usual rows form one broad pattern; data made of several distinct groups flags whole groups, which a segmentation reads better.",
"A flagged row is unusual, not wrong: check it against its source before correcting or removing it.",
"The threshold trades missed anomalies for false flags; about one ordinary row per thousand passes 0.999 by chance.",
"Measures are compared on a robust scale, so units do not matter, but a heavily skewed measure flags its own long tail.",
"Blank cells were filled with medians, which makes a row with blanks look more usual than it may be.")
answer <- list(n_flagged = n_flag, flagged_share_pct = round(100 * flag_share, 1), rows = n, measures = k,
threshold = thr_p, distance_cutoff = tidy(cutoff), expected_false_flags = tidy(expected_false),
most_unusual = if (n_flag) labels[ord[1]] else NULL, score_method = score_method, n = n)
results <- list()
#' The verdict and the headline are NOT places of a library tool (LAT-3130): the last mile writes them.
results$summary_metrics <- place_metric(list(n_flagged = n_flag, flagged_share_pct = round(100 * flag_share, 1),
rows = n, measures = k, distance_cutoff = tidy(cutoff), expected_false = tidy(expected_false)), lead = "n_flagged", place = "summary_metrics")
results$score_distribution <- place_distribution(dist_df, x = "score", series = "status", place = "score_distribution")
results$anomaly_map <- place_relationship(map_df, x = "pc1", y = "pc2", series = "status", place = "anomaly_map")
none_reason <- sprintf("no row passes the %s threshold, so there is no anomaly to explain; the score distribution shows how close the most unusual rows came", format(thr_p))
if (n_flag > 0) {
results$anomaly_drivers <- place_comparison(drivers_df, category = "stands_out_on", value = "flagged_rows", place = "anomaly_drivers")
results$anomaly_profile <- place_matrix(profile_df, x = "measure", y = "row", z = "robust_z", place = "anomaly_profile")
results$top_anomalies <- place_table(top_df, place = "top_anomalies")
} else {
results$anomaly_drivers <- place_dropped(none_reason, place = "anomaly_drivers")
results$anomaly_profile <- place_dropped(none_reason, place = "anomaly_profile")
results$top_anomalies <- place_dropped(none_reason, place = "top_anomalies")
}
results$assumption_checks <- place_table(checks_df, place = "assumption_checks")
results$detection_method <- list(kind = "metric", values = list(
method = method, n_in = n_in, n_used = n, excluded = as.list(excluded), assumptions = assumptions,
x_column = paste(colnames(M), collapse = ", "), y_column = sprintf("%d rows flagged", n_flag)),
value_order = list("n_used", "n_in"))
objects <- list() # filled by the object layer, not here
list(answer = answer, method = method, n = n, results = results, objects = objects,
json_output = list(answer = answer, method = method, n = n))
}