Naive Bayes: Probabilistic Classification

You have a dataset with a binary outcome (spam or not spam, churned or retained, approved or denied) and a set of features that might predict it. Naive Bayes gives you an interpretable classifier that estimates class probabilities, not just labels. Upload a CSV and get a classification report with ROC curves, confusion matrices, and feature profiles.

Worked Example: A Made-Up Test Set

Question: Can two number features help sort two classes in a small made-up test?

Inputs: Made-up data, seed 17. The script makes 240 rows with two number features and the same count for each class. It sets aside 60 rows for a test set. Class 0 is drawn from a normal distribution with mean 0 and class 1 from one with mean 1, in both features, and the two features are independent of each other. That is exactly what Naive Bayes assumes, so this example favors the model. It does not show what happens when features are linked.

Method: It fits Gaussian Naive Bayes on the rows used to train the model. It calls a row class 1 if its estimated chance is at least 0.5.

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import (
    accuracy_score,
    precision_score,
    recall_score,
    roc_auc_score,
    brier_score_loss,
    confusion_matrix,
)

# Simulated binary data with two numeric features.
seed = 17
rng = np.random.default_rng(seed)
n_per_class = 120
y = np.repeat([0, 1], n_per_class)
X = np.empty((2 * n_per_class, 2))
X[y == 0] = rng.normal(
    loc=[0.0, 0.0], scale=[1.0, 1.0], size=(n_per_class, 2)
)
X[y == 1] = rng.normal(
    loc=[1.0, 1.0], scale=[1.0, 1.0], size=(n_per_class, 2)
)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=seed, stratify=y
)
model = GaussianNB().fit(X_train, y_train)
prob = model.predict_proba(X_test)[:, 1]
pred = (prob >= 0.5).astype(int)
tn, fp, fn, tp = confusion_matrix(
    y_test, pred, labels=[0, 1]
).ravel()

print(f"Simulated data, seed {seed}")
print(f"Inputs: {len(y)} rows, 2 numeric features, balanced binary outcome")
print(f"Split: {len(y_train)} train rows, {len(y_test)} test rows")
print("Method: Gaussian Naive Bayes; class cutoff 0.5")
print(
    "Test confusion matrix (actual rows 0,1; predicted columns 0,1): "
    f"[[{tn}, {fp}], [{fn}, {tp}]]"
)
print(f"Accuracy: {accuracy_score(y_test, pred):.3f}")
print(f"Precision for class 1: {precision_score(y_test, pred):.3f}")
print(f"Recall for class 1: {recall_score(y_test, pred):.3f}")
print(f"ROC AUC: {roc_auc_score(y_test, prob):.3f}")
print(f"Brier score: {brier_score_loss(y_test, prob):.3f}")
edges = [0.0, 0.5, 1.0 + 1e-12]
for lo, hi in zip(edges[:-1], edges[1:]):
    mask = (prob >= lo) & (prob < hi)
    if mask.any():
        print(
            f"Calibration bin [{lo:.1f}, {min(hi, 1.0):.1f}]: "
            f"n={int(mask.sum())}, "
            f"mean estimate={prob[mask].mean():.3f}, "
            f"observed class-1 share={y_test[mask].mean():.3f}"
        )

Printed output (copied from a run with python3 -I):

Simulated data, seed 17
Inputs: 240 rows, 2 numeric features, balanced binary outcome
Split: 180 train rows, 60 test rows
Method: Gaussian Naive Bayes; class cutoff 0.5
Test confusion matrix (actual rows 0,1; predicted columns 0,1): [[23, 7], [4, 26]]
Accuracy: 0.817
Precision for class 1: 0.788
Recall for class 1: 0.867
ROC AUC: 0.904
Brier score: 0.126
Calibration bin [0.0, 0.5]: n=27, mean estimate=0.226, observed class-1 share=0.148
Calibration bin [0.5, 1.0]: n=33, mean estimate=0.767, observed class-1 share=0.788

Result: It got 49 of 60 test labels right. The count table shows 23 true negatives, 7 false positives, 4 false negatives, and 26 true positives. Accuracy was 0.817; class 1 precision was 0.788; class 1 recall was 0.867; and ROC AUC, a score for how well it ranks cases, was 0.904. In the low chance group, the mean estimate was 0.226 and the share in class 1 was 0.148. In the high chance group, the values were 0.767 and 0.788. This check compares estimated chances with observed shares. It does not show that the model's chances match real rates well.

Read these numbers with care: The accuracy of 0.817 and the AUC of 0.904 come from only 60 test rows, and they are a lucky split. For two classes this far apart, a typical accuracy would be about 0.76 and a typical AUC about 0.84. Do not read the 0.817 and 0.904 as the model's quality. The two groups of estimated chances are noisy for the same reason.

Limit: The data are made up, so these scores do not show how the model will do on real data.

What Is Naive Bayes?

Naive Bayes is a classification algorithm built on one of the most fundamental ideas in probability: Bayes' theorem. The theorem says that you can update your belief about something being true based on new evidence. In plain terms, if you already know that 30% of emails are spam, and you then observe the word "winner" in an email, Bayes' theorem tells you how to revise that 30% upward, because "winner" appears far more often in spam than in legitimate email. Stack enough pieces of evidence together and you get a strong prediction.

The "naive" part comes from one simplifying assumption: the algorithm treats each feature as independent of every other feature. In reality, the words "free" and "winner" probably do appear together in spam more often than chance would predict. Naive Bayes ignores that correlation and treats each word as an independent piece of evidence. This assumption is wrong in many real datasets. In the made-up example above it holds exactly, which is why that example favors the model. The classifier trains quickly and can be a useful baseline, especially when training data is limited.

See the made-up example above for the data, method, and test results.

When to Use Naive Bayes

Naive Bayes shines in several common scenarios. A common application is spam detection, where the algorithm learns which words and patterns are associated with spam versus legitimate email.

Sentiment analysis follows the same logic. Given a product review, which words predict a positive rating versus a negative one? "Excellent," "love," and "perfect" push toward positive; "broken," "terrible," and "waste" push toward negative. Naive Bayes handles this naturally because it builds a probability profile for each word under each class.

Document classification is another sweet spot: routing support tickets to the right department, tagging articles by topic, or classifying insurance claims by type. Anywhere you are sorting items into categories based on a set of attributes, Naive Bayes is worth trying as a baseline. It trains quickly, which suits rapid prototyping and situations where you need a working model soon.

Medical diagnosis is a more structured application. Given a set of symptoms, lab values, and patient demographics, Naive Bayes estimates the probability of each possible diagnosis. The probabilistic output is especially valuable in medicine because clinicians want to know how confident the model is, not just its best guess. A 0.51 probability of disease calls for different action than a 0.95 probability.

Use Naive Bayes when you want an interpretable model that gives you probability estimates. It is a reasonable first model for a classification problem. If it performs well enough on held-out data, you may not need anything more complex.

What Data Do You Need?

You need a CSV with at least two columns: an outcome column containing the binary class you want to predict (such as "spam"/"ham", "yes"/"no", "churned"/"retained", or 0/1), and at least one predictor column containing features that might help classify the outcome. You can include multiple predictor columns, the module accepts predictor_1 through predictor_[N], so you can map as many features as your data contains.

The module handles both numeric and categorical predictors. For numeric features, it fits a Gaussian distribution to estimate probabilities. For categorical features, it uses frequency-based estimates with Laplace smoothing to avoid zero-probability problems (a word that never appeared in the training set for one class would otherwise make that class impossible).

The key parameters you can configure include test_size (the fraction of data held out for evaluation, default is usually 0.2 or 20%), confidence_level (for the confidence intervals in the report), classification_threshold (the probability cutoff for assigning a class: default 0.5, but you can lower it to catch more positives at the cost of more false alarms), positive_class (which label is the "positive" class for metrics like precision and recall), and laplace (the smoothing parameter: higher values add more smoothing to prevent zero probabilities).

How to Read the Report

The report is organized into cards, each revealing a different facet of the classification results.

Naive Bayes Classifier (Overview). The opening card summarizes the model: what is being classified, how many observations were used for training versus testing, and the headline accuracy. This gives you a first answer to "does this model work?"

Data Preprocessing. Before fitting the model, the module inspects your data for missing values, class imbalance, and feature distributions. This card shows what was cleaned, transformed, or flagged. If your classes are heavily imbalanced (say 95% negative, 5% positive), this card will note it, because accuracy alone is misleading when one class dominates.

Executive Summary (TL;DR). A plain-language summary of the key findings: overall accuracy, which features matter most, and whether the model is trustworthy enough to act on. This is the card to share with stakeholders who will not read the full report.

ROC Curve. The Receiver Operating Characteristic curve plots the true positive rate against the false positive rate at every possible classification threshold. A model that predicts randomly traces a diagonal line; a perfect model hugs the top-left corner. The Area Under the Curve (AUC) condenses this into a single number between 0.5 (random) and 1.0 (perfect). The ROC curve is especially valuable when you need to choose your own threshold. You can see exactly how much false-positive rate you would accept to gain more true positives.

Confusion Matrix. This is the scorecard. It shows four numbers: true positives (correctly predicted positive), true negatives (correctly predicted negative), false positives (predicted positive but actually negative), and false negatives (predicted negative but actually positive). From these four numbers you can derive precision (of the items flagged as positive, how many actually were?), recall (of all actual positives, how many did the model catch?), and F1 score (the harmonic mean balancing both). The confusion matrix makes the model's mistakes concrete. You can see whether it is biased toward over-predicting or under-predicting the positive class.

Predicted Probability Distribution. This card shows the distribution of predicted probabilities for each class. Calibration asks if cases given a chance are positive about that often. Class gap asks if the model tends to give each true class a different score. If the distributions overlap heavily, the model is uncertain about many observations, which means your results will be sensitive to the threshold you choose. This visualization often reveals more about model quality than the headline accuracy.

Feature Profiles. Naive Bayes learns a probability profile for each feature under each class. This card shows those profiles: for numeric features, the mean and standard deviation within each class; for categorical features, the frequency distribution. This is where you see which features actually drive the classification. If the profiles for a feature look nearly identical across classes, that feature is not contributing much. If they look very different, it is a strong signal. This interpretability is one of Naive Bayes' key advantages over black-box models.

Performance Metrics. A comprehensive table of classification metrics: accuracy, precision, recall, F1 score, specificity, balanced accuracy, and AUC. These are computed on the held-out test set, so they reflect how the model performs on data it has not seen. The card also includes confidence intervals so you can assess the uncertainty around each metric. If your dataset is small, wide confidence intervals are a warning that the estimates may not be stable.

Model Parameters. The final card shows the fitted model parameters: the prior probabilities for each class and the conditional probabilities or distribution parameters for each feature. This is full transparency: you can see exactly what the model learned and verify it against your domain knowledge. If the model assigns high probability to a feature-class combination that makes no business sense, you have found either a data quality issue or a feature that should be excluded.

When to Use Something Else

If your features have strong interactions (where the effect of one feature depends on the value of another) the naive independence assumption will hurt performance. In that case, logistic regression is the natural next step. It is still interpretable and handles correlated features through its coefficient estimation. Logistic regression also gives you odds ratios, which are easier to explain in regulated industries like finance and healthcare.

If your dataset is large and you want maximum predictive accuracy without worrying about interpretability, random forest or XGBoost will usually outperform Naive Bayes. These ensemble methods capture complex interactions and nonlinear relationships that Naive Bayes cannot model. The tradeoff is that they are slower to train and harder to explain.

For high-dimensional data with clear linear boundaries, a support vector machine (SVM) can work well, particularly with text classification tasks where the feature space (word counts) is very large relative to the number of documents. Decision trees are another alternative when you need a model that non-technical stakeholders can follow step by step, though single decision trees tend to overfit without pruning or ensemble methods.

That said, Naive Bayes is worth running as a baseline. It gives you probability estimates quickly. If it meets your accuracy requirements, its speed and interpretability are a reason to keep it.

The R Code Behind the Analysis

Every report names its method and includes the R code that produced it, so you can reproduce, audit and cite it. The method code is fixed, not rewritten for each run, so the same data gives the same analysis every time.

The analysis uses an R function for Naive Bayes. The report also uses pROC for ROC curve computation and AUC estimation, and caret for confusion matrix metrics and cross-validation. Laplace smoothing and train/test splitting are handled in the code. Every step is visible in the code tab of your report, so you or a statistician can verify exactly what was done.