CatBoost: When to Use It and How to Run It on Your Data

Feature engineering and hyperparameter tuning are where much of the labor in a gradient boosting project goes. CatBoost accepts categorical features directly and has default hyperparameters you can start from. If you test it, measure the time on your data against a stated baseline.

CatBoost is one option for boosting models with categorical data. Compare its result with other tools on the same data and split.

Rather than a single forecast of "CatBoost is better," look at the range of possibilities and the assumptions behind each.

Worked Example (Simulated): Choosing a Contact Cut-Off

Question: How should a team pick a risk score cut-off for a contact offer?

Inputs: Simulated data, seed 17. The script makes 1,200 made-up risk scores and churn outcomes. The printed assumptions set the contact cost ($50), the value of a saved customer ($1,200) and the save rate (35%).

Method: The script picks a cut-off on 360 tuning rows from five candidates, then checks it once on 360 separate test rows. The test rows do not pick the cut-off. A contact pays when the saved value (churn chance x 0.35 x $1,200) is above the $50 cost, which is a churn chance above 50 / (0.35 x 1,200) = 0.119, or about 12%.

import numpy as np

# Simulated data, seed 17. Scores stand in for model predictions.
seed = 17
rng = np.random.default_rng(seed)
n = 1200
score = rng.uniform(0.05, 0.80, size=n)
churn_prob = np.clip(0.04 + 0.56 * score, 0.0, 1.0)
churn = rng.binomial(1, churn_prob)
order = rng.permutation(n)
tune_idx = order[:360]
test_idx = order[360:720]
candidates = [0.20, 0.30, 0.40, 0.50, 0.60]
campaign_cost = 50.0
revenue_at_risk = 1200.0
success_rate = 0.35

def net_value(indices, threshold):
    targeted = score[indices] >= threshold
    n_targeted = int(targeted.sum())
    n_churn = int(churn[indices][targeted].sum())
    retained = n_churn * success_rate
    spend = n_targeted * campaign_cost
    saved = retained * revenue_at_risk
    return saved - spend, n_targeted, n_churn, retained, spend, saved

best = max(candidates, key=lambda t: net_value(tune_idx, t)[0])
tune = net_value(tune_idx, best)
test = net_value(test_idx, best)
print(f"Seed: {seed}")
print(f"Simulated rows: {n}; tuning rows: {len(tune_idx)}; "
      f"test rows: {len(test_idx)}")
print(f"Assumptions: contact cost=${campaign_cost:,.2f}; "
      f"value per saved customer=${revenue_at_risk:,.2f}; "
      f"save rate={success_rate:.2f}")
print("Tuning thresholds: " + ", ".join(f"{t:.2f}" for t in candidates))
print(f"Selected on tuning set: threshold={best:.2f}; net=${tune[0]:,.2f}")
print(f"Untouched test: targeted={test[1]}; churned={test[2]}; "
      f"expected saved={test[3]:.2f}")
print(f"Test spend=${test[4]:,.2f}; expected saved value=${test[5]:,.2f}; "
      f"net=${test[0]:,.2f}")

Printed output:

Seed: 17
Simulated rows: 1200; tuning rows: 360; test rows: 360
Assumptions: contact cost=$50.00; value per saved customer=$1,200.00; save rate=0.35
Tuning thresholds: 0.20, 0.30, 0.40, 0.50, 0.60
Selected on tuning set: threshold=0.20; net=$24,360.00
Untouched test: targeted=289; churned=88; expected saved=30.80
Test spend=$14,450.00; expected saved value=$36,960.00; net=$22,510.00

Result: The tuning rows chose a score cut-off of 0.20. On the test rows, the script targeted 289 people, 88 of whom churned. At the assumed save rate it expected 30.80 saved people and a net value of $22,510.00. 0.20 was the lowest cut-off tried, so the grid did not bracket the best choice; widen it before trusting it.

Limit: This is a simulation, and the risk scores are made up. It does not fit or test CatBoost. The save rate and customer value are assumptions, so this result is not a forecast for a real campaign.

Where Feature Engineering Time Goes

A gradient boosting workflow usually runs: load the data, encode categorical features, tune hyperparameters, train the model, deploy. Encoding and tuning are two places where the time can add up.

Common encodings each carry a cost. One-hot encoding adds a column per category, which can make the feature matrix large. Target encoding can leak information from validation rows if it is not done inside the cross-validation folds. Hash encoding gives up some interpretability.

CatBoost takes a list of categorical columns through its cat_features argument, as in the code below. Whether that saves time on your project is a measurement: record the hours for each approach on your data and compare them with a stated baseline.

Building Models That Predict Business Outcomes (Not Just Metrics)

The business scenario below uses assumed inputs. The simulated example above shows a repeatable way to pick and assess a threshold.

Scenario: You're building a customer churn model for a subscription company, with features such as subscription tier, industry, company size bucket, feature usage patterns, support ticket categories, and acquisition channel.

The business question isn't "what's the best AUC?" It's "which customers should we target with retention offers, and what's the expected ROI?"

A retention campaign costs $50 per customer contacted (gift cards, discounts, account manager time). A churned customer represents $1,200 in lost annual revenue. Assume your intervention success rate is 35%: you retain 35% of customers you target who would have otherwise churned. That rate is an assumption here; in practice it should come from a controlled test (a held-out group that gets no offer), because the model predicts who churns, not who an offer would save.

The distribution of possible outcomes depends heavily on model calibration. If your model's predicted probabilities don't match actual churn rates, you'll target the wrong customers and waste campaign budget.

Implementation: A CatBoost Workflow

Here is what a CatBoost workflow could look like for this churn model. This sketch was not run for this page; it needs the catboost package and your own customer_data.csv.

import catboost as cb
import pandas as pd

# Load data
train_data = pd.read_csv('customer_data.csv')

# Define categorical features
cat_features = [
    'subscription_tier', 'industry', 'company_size',
    'acquisition_channel', 'primary_use_case', 'region'
]

# Parse the date (used only for the split, not as a feature)
train_data['signup_date'] = pd.to_datetime(train_data['signup_date'])

# Split features and target
X = train_data.drop(['customer_id', 'churned', 'signup_date'], axis=1)
y = train_data['churned']

# Create train/validation split with proper time-based separation
train_idx = train_data['signup_date'] < '2025-10-01'
val_idx = train_data['signup_date'] >= '2025-10-01'

X_train, X_val = X[train_idx], X[val_idx]
y_train, y_val = y[train_idx], y[val_idx]

# Train model with a few hyperparameters
model = cb.CatBoostClassifier(
    iterations=1000,
    learning_rate=0.03,
    eval_metric='AUC',
    cat_features=cat_features,
    random_seed=42,
    verbose=100
)

model.fit(
    X_train, y_train,
    eval_set=(X_val, y_val),
    early_stopping_rounds=50,
    plot=True
)

# Get predicted churn probabilities (uncalibrated; check calibration below)
predictions = model.predict_proba(X_val)[:, 1]

# Validation Pool, reused by the interpretation snippets below
pool_val = cb.Pool(X_val, y_val, cat_features=cat_features)

This sketch has no separate encoding pipeline and no hyperparameter search. Time how long each step takes on your own data.

Quantifying Business Impact: The Expected Value Framework

To decide at what predicted churn probability to intervene, pick the threshold using a separate tuning set. Then report the result once on a test set that did not help choose the threshold. The worked example above does this with simulated scores: it chose 0.20 on the tuning rows and reported a net value of $22,510.00 on the test rows.

With the assumptions above, contacting a customer pays whenever the churn chance is above about 12%. A grid that starts at 0.20 is above that line, so widen it downward and let the data show where net value peaks.

These are point estimates, and they do not include the chance that the model itself is wrong. The success rate should come from a controlled test.

Two Cost Traps Teams Fall Into

Even with a library that takes categorical columns directly, teams can waste time and money in predictable ways.

Cost Trap #1: Over-Tuning Hyperparameters

Start with the default CatBoost parameters. Data scientists can spend days running grid searches for a small gain in validation AUC. Weigh the hours spent against what that gain is worth to the business.

The rule: Start with defaults. Tune only if default performance is clearly inadequate for the business need. When you do tune, focus on three parameters:

• iterations - more trees usually helps, use early stopping
• learning_rate - lower rates with more iterations
• depth - try a few depths and keep what the held-out data supports

Other settings (subsample ratios, regularization terms, boosting types) are worth trying only after those three.

Cost Trap #2: Rebuilding Analysis Tools You May Not Need

CatBoost includes model interpretation tools: feature importance, SHAP values, prediction explanations, and object importance. Teams sometimes skip these and build custom analysis pipelines. Check what the library offers before you build your own.

# Get feature importance
feature_importance = model.get_feature_importance(prettified=True)

# Calculate SHAP values for top predictions
shap_values = model.get_feature_importance(
    data=pool_val,
    type='ShapValues'
)

# Explain one prediction: SHAP values for the first validation row
row_pool = cb.Pool(X_val.iloc[[0]], cat_features=cat_features)
explanation = model.get_feature_importance(
    data=row_pool,
    type='ShapValues'
)

Interpreting CatBoost Results: From Predictions to Decisions

A model that predicts churn probability of 0.67 for Customer A is useless unless you know how to act on that information. CatBoost provides tools to bridge the gap between predictions and decisions.

Feature Importance: Where Your Model Finds Signal

CatBoost calculates feature importance using "PredictionValuesChange": how much each feature contributes to changing prediction values across the entire dataset.

feature_importance = model.get_feature_importance(prettified=True)
print(feature_importance.head(10))

Output might look like this (made-up values, to show the format):

Importance shows how the model uses each feature, not what would reduce churn. It can suggest questions to test: if days_since_last_login weighs heavily in the predictions, a re-engagement offer is one idea worth testing in a controlled experiment before you act on it.

But feature importance only tells you what matters. To understand how features affect predictions, use SHAP values.

SHAP Values: Understanding Individual Predictions

SHAP (SHapley Additive exPlanations) values decompose each prediction into contributions from each feature. For Customer A with 0.67 churn probability:

# Calculate SHAP values for a specific customer
customer_idx = 0  # position of the customer in X_val
customer_pool = cb.Pool(
    data=X_val.iloc[[customer_idx]],
    cat_features=cat_features
)

shap_values = model.get_feature_importance(
    data=customer_pool,
    type='ShapValues'
)

# Base value + feature contributions = raw model output (log-odds scale)
base_value = shap_values[0, -1]  # Last column is base value
contributions = shap_values[0, :-1]  # Feature contributions

# Convert the raw sum to a churn probability with the logistic function
raw = base_value + contributions.sum()
churn_probability = 1 / (1 + np.exp(-raw))  # needs: import numpy as np

For CatBoost classifiers the SHAP values are on the raw log-odds scale, so the sum is converted to a probability as above. For readability, the made-up values below are shown as if already converted to probability points, so they add up directly:

• Base churn rate: 0.23 (population average)
• +0.18 from days_since_last_login = 47 (high risk)
• +0.12 from support_tickets_last_30d = 8 (frustrated user)
• +0.09 from feature_usage_score = 12 (low engagement)
• +0.05 from subscription_tier = 'basic' (less committed)
• Final prediction: 0.67

Now you can personalize the intervention. This customer hasn't logged in for 47 days and has submitted 8 support tickets. They're struggling with the product. Your retention offer should include dedicated onboarding support, not just a discount.

Calibration: Do Your Probabilities Mean What You Think?

A model is well calibrated when, among customers it scores near 0.40, about 40% actually churn. If 60% of them churn, your probabilities are miscalibrated.

Poor calibration destroys ROI calculations. If you think 0.40 means 40% churn but it actually means 60%, you'll under-invest in retention and lose customers.

Check calibration with a reliability diagram:

from sklearn.calibration import calibration_curve
import matplotlib.pyplot as plt

# Calculate calibration curve
prob_true, prob_pred = calibration_curve(
    y_val, predictions, n_bins=10, strategy='quantile'
)

# Plot
plt.figure(figsize=(8, 6))
plt.plot(prob_pred, prob_true, marker='o', label='CatBoost')
plt.plot([0, 1], [0, 1], linestyle='--', label='Perfect calibration')
plt.xlabel('Predicted Probability')
plt.ylabel('Observed Frequency')
plt.title('Model Calibration')
plt.legend()
plt.show()

Do not assume a model is well calibrated. Check it, especially if you use predictions for decision-making under uncertainty.

Cost-Optimized Hyperparameter Strategy

When default parameters aren't sufficient, tune strategically to maximize ROI per engineering hour invested.

Phase 1: Quick wins

• Increase iterations to 2000-3000 with early_stopping_rounds=50
• Try learning_rate values of 0.01, 0.03, 0.05
• Test depth values of 4, 6, 8

A small search like this is a low-cost way to see whether tuning is worth more time.

Phase 2: Diminishing returns

Only proceed if Phase 1 results are inadequate:

• l2_leaf_reg: Try 1, 3, 5, 10 (regularization strength)
• border_count: Try 128, 254 (numerical feature split points)
• bagging_temperature: Try 0, 1, 2 (Bayesian bootstrap intensity)

Phase 3: Specialized scenarios

Only for specialized problems:

• Imbalanced classes: class_weights or scale_pos_weight
• Uncertainty quantification: loss_function='RMSEWithUncertainty'
• Custom objectives: Define business-specific loss functions

Many teams can stop after Phase 1 if it meets the business need. Go further only for specific requirements or performance constraints.

Production Deployment Notes

Training is one part of the work. Deployment, monitoring, and maintenance add the rest. Measure each on your own setup.

Saving and Loading a Model

A CatBoost model can be saved to a file and loaded in production. Check the file size and load time for your model.

# Save model
model.save_model('churn_model.cbm', format='cbm')

# Load in production
loaded_model = cb.CatBoostClassifier()
loaded_model.load_model('churn_model.cbm', format='cbm')

Feature Drift and Model Degradation

Categories change over time. A new product category may appear after the model is trained. Test how your model behaves on categories it has not seen, and retrain on a planned schedule instead of waiting for an emergency.

Advanced Technique: Uncertainty Quantification for Risk-Aware Decisions

Most gradient boosting models output point predictions. "Customer A has 0.67 churn probability." But what's the uncertainty around that estimate?

When decisions involve financial risk, uncertainty matters as much as the prediction itself. CatBoost supports uncertainty estimation through the RMSEWithUncertainty loss function.

# Train model with uncertainty estimation
uncertainty_model = cb.CatBoostRegressor(
    iterations=1000,
    loss_function='RMSEWithUncertainty',
    cat_features=cat_features,
    random_seed=42
)

uncertainty_model.fit(X_train, y_train)

# Get predictions with uncertainty
# Returns: [prediction, uncertainty] for each sample
predictions_with_uncertainty = uncertainty_model.predict(X_val)

This outputs both a prediction and an uncertainty estimate (standard deviation). For a customer lifetime value prediction, with made-up numbers:

• Customer A: Predicted LTV = $2,400, Uncertainty = $180
• Customer B: Predicted LTV = $2,350, Uncertainty = $620

Both customers have similar predicted values, but Customer B's prediction has much higher uncertainty. Maybe they're in a rare segment with limited training data, or their feature values are far from typical patterns.

How does this change decisions? If you're deciding whether to offer a $200 retention incentive:

• Customer A: High confidence in $2,400 LTV. Clear ROI: spend $200 to protect $2,400.
• Customer B: Wide uncertainty band ($1,730-$2,970 at 1 std). The actual LTV might be $1,800, making a $200 incentive questionable ROI.

Rather than a single forecast, let's look at the range of possibilities. You might create a decision rule: "Offer incentives when predicted LTV exceeds $2,000 AND uncertainty is below $300." This risk-aware policy prevents overspending on uncertain predictions.

Choosing Between Gradient Boosting Libraries

To choose between CatBoost, XGBoost, LightGBM, or another method, fit each on the same data with the same split. Compare the metric that matters to the business, plus training time and serving cost, and pick on the measured results.

For a deeper comparison of gradient boosting techniques, see our comprehensive gradient boosting guide.

Best Practices: Maximizing ROI with CatBoost

These practices help keep cost and risk down:

1. Start with Defaults, Tune Only When Necessary

Premature hyperparameter optimization is a common way to waste engineering time. Before tuning anything:

• Train with defaults and evaluate validation performance
• Compare to a simple baseline (logistic regression, mean prediction)
• Only tune if default performance is clearly inadequate for business needs

If defaults achieve 0.78 AUC and you need 0.80 for business value, then tune. If defaults achieve 0.78 and a simple baseline gets 0.65, you're done, ship it.

2. Use Native Categorical Feature Handling

If you use CatBoost's categorical handling, pass the raw categorical columns and list them in cat_features instead of encoding them first.

# Encoding the columns yourself first
X_encoded = pd.get_dummies(X, columns=cat_features)
model.fit(X_encoded, y)

# Passing the raw columns to CatBoost
model.fit(X, y, cat_features=cat_features)

3. Leverage Built-In Model Analysis

Check CatBoost's native feature importance, SHAP values, and prediction explanations before building custom tools.

4. Use Time-Based Splits for Business Data

Random train/test splits can leak future information in temporal problems. Split by time for business forecasting:

# Random split
X_train, X_test = train_test_split(X, y, test_size=0.2)

# Split by time
cutoff_date = '2025-10-01'
train_mask = data['date'] < cutoff_date
X_train, X_test = X[train_mask], X[~train_mask]
y_train, y_test = y[train_mask], y[~train_mask]

5. Monitor Calibration in Production

Track how often your predictions match reality. If the model predicts 30% churn for a segment and actual churn is 45%, you have a calibration problem that's costing you money.

6. Version Models with Data Snapshots

Save not just the model but the training data statistics. This helps diagnose production issues and understand when retraining is needed.

# Save model with metadata
model.save_model('churn_v1.cbm')

# Save training metadata
metadata = {
    'training_date': '2026-03-01',
    'n_samples': len(X_train),
    'categorical_stats': {
        col: X_train[col].value_counts().to_dict()
        for col in cat_features
    },
    'target_mean': y_train.mean(),
    'validation_auc': 0.82
}

import json
with open('churn_v1_metadata.json', 'w') as f:
    json.dump(metadata, f)

The Probabilistic Perspective: Thinking in Distributions

Throughout this guide, we've focused on distributions rather than point estimates. This isn't just philosophical, it's financial.

When you predict "Customer A has 67% churn probability," you're not claiming certainty. You're describing a probability distribution over outcomes. That customer might churn (33% probability they don't), but your decision should account for this uncertainty.

CatBoost enables this probabilistic thinking through:

• Probability predictions you can check for calibration against observed frequencies
• Uncertainty quantification that tells you how confident to be in each prediction
• SHAP value distributions that show how different features contribute to prediction variance

Thinking in distributions helps with decisions under uncertainty. Don't just predict churn. They calculate expected value across the distribution of possible outcomes and choose actions that maximize ROI in expectation.

Uncertainty isn't the enemy, ignoring it is. CatBoost gives you the tools to quantify, understand, and act on uncertainty in ways that translate directly to better business outcomes and higher ROI.

Conclusion: The Cost-Benefit Reality of CatBoost

Whether CatBoost pays off depends on your data and your team.

CatBoost doesn't always produce the highest accuracy, and it is not always the fastest to train. Treat it as one option for boosting models with categorical data.

Is CatBoost worth a try for your project? Run this simple checklist:

1. Do you have categorical features? (If yes, a reason to try CatBoost)
2. Is development time expensive relative to compute? (If yes, a reason to try CatBoost)
3. Do you need fast production predictions? (Measure prediction time on your data)
4. Is this a one-off analysis or a production system? (A production system raises the value of measuring serving cost)

If you answered yes to two or more, try CatBoost and compare it with another tool on the same data and split. Choose on measured results, and count total cost across development, deployment, and maintenance.