Data Screening Before Analysis: The Checks That Save Your Conclusions

You run a correlation analysis on customer data. Find strong relationships between purchase frequency and lifetime value. Write the report. Present to stakeholders. Then someone asks: "Why do we have 847 duplicate customer IDs?" and "Is this person really 247 years old?" Your analysis is correct—for garbage data. Data screening catches these issues before they wreck your conclusions.

I've seen teams waste weeks on sophisticated analyses of fundamentally broken data. The fix takes 15 minutes: systematic data screening before you start. Not cleaning—screening. You're not fixing problems yet. You're diagnosing what's actually in your dataset so you can make informed decisions about what to do next.

Why Most Teams Skip Screening (And Regret It)

Data screening feels like busywork. You're eager to answer your research question. You've got a deadline. The CSV loaded without errors, so it must be fine, right?

Wrong. Here's what happens when you skip screening:

Your sample size is inflated. You think you have 5,000 customer records. Actually you have 4,200 unique customers and 800 duplicates. Your statistical tests are overconfident because they're treating the same person multiple times as independent observations. Every p-value is wrong.

Your outliers dominate your results. One customer with $4.8M in annual purchases pulls your correlation coefficient from 0.23 to 0.67. You conclude there's a strong relationship. Remove that one person and the relationship disappears. You've found one weird customer, not a general pattern.

Your missing data isn't random. You're missing 12% of income data. Seems manageable. But it's not random—low earners are three times more likely to skip that question. Your complete-case analysis excludes the people who matter most for your research question. Your conclusions apply to high earners only.

Your data contains impossible values. Someone has an age of 247. Another has -$50,000 in lifetime purchases. A third made 47 purchases in one day. These aren't insights—they're data entry errors, system bugs, or placeholder codes that should have been filtered out. They'll wreck any analysis that includes them.

Data screening catches all of this in the first 15 minutes. Before you've invested time in analysis. Before you've drawn conclusions. Before you've presented results to stakeholders.

The Five Quality Checks That Actually Matter

Forget the 37-step data quality framework. In practice, five checks catch 95% of problems:

1. Duplicate Records

Start here. Count unique identifiers versus total rows. If you have customer IDs, how many are unique? If you don't have explicit IDs, how many rows are exact duplicates across all columns?

Duplicates inflate sample size artificially. If you have 1,000 rows but only 800 unique customers, your effective sample size is 800, not 1,000. Every statistical test that uses sample size (which is all of them) will be wrong. You'll get significant results that disappear when you remove duplicates.

The Standard Data Screening tool flags exact duplicates automatically. Upload your CSV and it shows you duplicate counts and examples. From there, you decide: Are these legitimate repeated measures (same customer, multiple time points)? System errors (data export glitch)? Or something else?

2. Impossible and Implausible Values

Check the range of each variable. Look for:

• Physically impossible values: Age = 247, negative prices, purchase counts over 1000 per day
• Out-of-range codes: Gender coded as 1/2 but you find 9s (often "unknown" in source systems)
• Placeholder values: 999, -999, 0 used to represent missing data instead of actual NA
• Unit mismatches: Most revenues in thousands, but some in dollars (you'll see values 1000x larger)

Don't rely on summary statistics alone. Look at actual minimum and maximum values. Check a few random samples. One impossible value can tell you there's a systematic problem with data collection or export.

3. Outliers and Extreme Values

Outliers aren't automatically errors. A billionaire customer is an outlier, but they're real. The question is: Do these extreme values represent legitimate variation in your population, or do they reflect data problems?

Standard screening identifies outliers using multiple methods:

• Z-scores: Values more than 3 standard deviations from the mean
• IQR method: Values beyond 1.5 × interquartile range from Q1/Q3
• Visual inspection: Boxplots and histograms show the actual distribution

But the decision about what to do with outliers isn't statistical—it's substantive. You need domain knowledge. Is a customer with 200 purchases per year plausible in your business? If you sell industrial equipment to manufacturers, maybe. If you sell mattresses to consumers, definitely not.

4. Missing Data Patterns

Missing data is normal. The question is: Is it missing completely at random, or is there a pattern?

Calculate missingness for each variable. Then check if missingness correlates with other variables. For example:

• Income is missing for 15% of respondents
• But it's missing for 8% of high-education respondents and 23% of low-education respondents
• Missingness isn't random—it's related to education

This matters because different missing data patterns require different solutions. If data is missing completely at random (MCAR), you can often just drop those cases. If it's missing at random conditional on other variables (MAR), you might need imputation. If it's missing not at random (MNAR)—missingness is related to the unobserved value itself—you've got a serious problem that requires sensitivity analysis.

Data screening reveals the pattern. You decide how to handle it based on your research question and the severity of bias.

5. Distribution and Normality

Many statistical tests assume normally distributed data. Check that assumption before you run the test, not after.

Look at:

• Histograms: Is the distribution approximately bell-shaped, or heavily skewed?
• Q-Q plots: Do points follow the diagonal line (normal) or deviate systematically?
• Skewness and kurtosis: Quantify how far from normal your distribution is
• Formal tests: Shapiro-Wilk or Kolmogorov-Smirnov tests for normality

If your data is heavily skewed and you're planning parametric tests (t-tests, ANOVA, linear regression), you have three options: transform the data (log, square root), use non-parametric alternatives, or acknowledge the assumption violation and proceed with caution (often fine with large samples due to central limit theorem).

The key is knowing about the violation before you analyze, so you can make an informed choice.

What Screening Output Actually Looks Like

Theory is helpful. Implementation is better. Here's what you see when you run data screening on a real dataset.

Upload a CSV with customer transaction data to the Standard Data Screening tool. You get back a structured report with five sections:

Section 1: Dataset Overview

Dataset: customer_transactions.csv
Total rows: 5,247
Total columns: 12
Duplicate rows: 156 (3.0%)
Unique rows: 5,091

Variables:
  - customer_id (character)
  - age (numeric)
  - gender (character)
  - total_purchases (numeric)
  - lifetime_value (numeric)
  - signup_date (date)
  - last_purchase_date (date)
  - email_provided (logical)
  - income (numeric)
  - city (character)
  - zip_code (character)
  - referral_source (character)

First finding: You have 156 duplicate rows. That's 3% of your dataset. Not huge, but not negligible. You need to decide if these are legitimate repeated measures or data quality issues.

Section 2: Missing Data Summary

Missing Data by Variable:
  customer_id: 0 (0.0%)
  age: 89 (1.7%)
  gender: 12 (0.2%)
  total_purchases: 0 (0.0%)
  lifetime_value: 0 (0.0%)
  signup_date: 3 (0.1%)
  last_purchase_date: 247 (4.7%)
  email_provided: 0 (0.0%)
  income: 634 (12.1%)
  city: 45 (0.9%)
  zip_code: 52 (1.0%)
  referral_source: 891 (17.0%)

Patterns:
  - Income missing more often when age < 25 (18.3% vs 9.1%)
  - Referral source missing primarily for signups before 2024-01-01
  - Last purchase date missing for 247 customers (likely churned users)

Second finding: Missingness isn't random. Income is missing twice as often for young customers. Referral source is missing for old signups (probably wasn't tracked back then). Last purchase date is missing for customers who haven't bought recently—that's actually informative, not a problem.

Section 3: Outliers and Extreme Values

Outliers Detected (IQR method):

age:
  - 3 values > 100: [127, 138, 247]
  - LIKELY ERRORS: Age > 120 is implausible

total_purchases:
  - 7 values > 200: [234, 267, 289, 312, 334, 401, 523]
  - Review needed: Are these corporate/wholesale accounts?

lifetime_value:
  - 12 values > $50,000: ranging from $52,340 to $247,890
  - 2 values < $0: [-$2,400, -$850]
  - ERRORS: Negative lifetime value impossible

income:
  - 4 values > $500,000: [$523,000, $687,000, $890,000, $1,240,000]
  - Plausible but verify: Top 0.1% of dataset

Third finding: You've got clear errors (age = 247, negative lifetime value), questionable values (523 purchases—possible but unlikely), and legitimate outliers (high incomes). This needs manual review, not automatic filtering.

Section 4: Distribution Checks

Normality Assessment:

age:
  Skewness: 2.14 (right-skewed)
  Kurtosis: 8.92 (heavy-tailed)
  Shapiro-Wilk: p < 0.001 (not normal)

total_purchases:
  Skewness: 3.47 (heavily right-skewed)
  Kurtosis: 18.23 (very heavy-tailed)
  Shapiro-Wilk: p < 0.001 (not normal)

lifetime_value:
  Skewness: 2.89 (right-skewed)
  Kurtosis: 12.34 (heavy-tailed)
  Shapiro-Wilk: p < 0.001 (not normal)

Recommendation: Consider log transformation for purchase and value variables before parametric tests.

Fourth finding: Your key variables aren't normally distributed. If you're planning to run correlation, regression, or t-tests, you should either transform the data or use non-parametric alternatives.

Section 5: Recommendations

Data Quality Issues Requiring Attention:

HIGH PRIORITY:
  1. Remove or correct 3 impossible age values (> 120 years)
  2. Investigate 2 negative lifetime_value records (system error?)
  3. Address 156 duplicate rows (keep first occurrence?)

MEDIUM PRIORITY:
  4. Review 7 customers with > 200 purchases (corporate accounts?)
  5. Examine missingness pattern in income (related to age)
  6. Consider log transformation for skewed variables

LOW PRIORITY:
  7. Missing referral_source for old signups (expected)
  8. Missing last_purchase_date = churned customers (informative)

Suggested Next Steps:
  - Remove exact duplicates
  - Filter to age < 120
  - Investigate negative values
  - Run analysis with/without high-purchase customers (sensitivity check)
  - Use robust methods or transformations for skewed data

This is what actionable screening looks like. Not just "you have outliers" but specific guidance on what needs attention and what you can safely ignore.

The Hidden Patterns Data Screening Reveals

Beyond catching obvious errors, systematic screening surfaces patterns you wouldn't notice otherwise:

Systematic Bias in Data Collection

You're analyzing survey responses. Screening shows that income is missing for 18% of respondents under 25, but only 6% over 25. That's not random—young people are systematically skipping income questions. If you just drop incomplete cases, you're underrepresenting young low-earners in your analysis. That bias might be more important than anything you find in the complete data.

Seasonal or Temporal Patterns

Customer purchase data looks fine in aggregate. But when you screen by signup cohort, you notice that referral source is missing for 89% of pre-2024 signups and only 2% after. Your tracking system changed. If you ignore this, your "analysis of referral effectiveness" is actually comparing two different measurement systems. The pattern has nothing to do with customer behavior.

Data Export and Integration Issues

You receive data from a third-party vendor. Screening shows that values for "revenue" cluster into two distinct groups: some in the $10-$500 range, others in the $10,000-$500,000 range. The first group is exactly 1000x smaller. Unit mismatch. Some records are in dollars, others in thousands. If you just run your analysis, you'll conclude that two types of customers exist. Actually, it's one customer population with two data formats.

Placeholder Codes Masquerading as Data

Age values show a spike at exactly 99. Not 98, not 100—just 99, accounting for 147 records. That's not a real age distribution. It's a placeholder code for "unknown" that wasn't properly converted to NA during data export. If you include those 147 records in your age analysis, you'll think you have a lot of very old customers. You don't—you have a lot of unknown ages.

These patterns don't show up in standard summary statistics. You need systematic screening to catch them.

When to Screen vs. When to Clean

Data screening and data cleaning are different activities with different goals:

Data screening is diagnostic. You're assessing what's wrong. Output is a report: "You have 156 duplicates, 3 impossible ages, 634 missing income values, and heavy right skew in purchases." You don't change the data. You document issues.

Data cleaning is corrective. You're fixing problems. Output is a modified dataset: duplicates removed, impossible values replaced with NA, variables transformed. You change the data based on documented decisions.

Always screen before cleaning. Why?

First, screening gives you the scope of problems. If you have 3 impossible ages in 5,000 records, you can safely drop them. If you have 847 impossible ages, you need to investigate why—there's probably a systematic issue with data collection.

Second, screening reveals patterns that change your cleaning strategy. If missing income is random, you might drop incomplete cases. If missingness correlates with age and education, dropping cases introduces bias. You need different solutions (imputation, sensitivity analysis, or keeping the variable out of your model entirely).

Third, screening creates an audit trail. When someone asks "Why did you exclude these 23 records?" you can point to your screening report: "They had age > 120, which is impossible. Here's the distribution showing three extreme outliers. We verified against source data and confirmed they're data entry errors."

Screen to diagnose. Clean based on what you find. Document both steps.

Common Screening Mistakes and How to Avoid Them

Even teams that do screening make predictable mistakes:

Mistake 1: Automatic Outlier Removal

Don't write code that automatically removes anything beyond 3 standard deviations. That's a statistical threshold, not a data quality decision.

An outlier might be:

• A legitimate extreme value (billionaire customer, viral product)
• An error (age = 247, negative revenue)
• A different population (wholesale customer in retail dataset)
• A rare but important case (fraud event, system failure)

You need to look at the actual values and use domain knowledge. Is a $4.8M customer plausible in your business? If yes, keep them. If no, investigate why that value exists. Don't let an algorithm decide.

Mistake 2: Ignoring the Pattern of Missingness

Teams count missing data but don't check if it's random. "We're missing 12% of income values, that's fine" ignores the critical question: Which 12%?

If missingness is random, dropping cases is usually safe (though you lose statistical power). If missingness correlates with other variables—especially your outcome—dropping cases introduces bias. You're analyzing a non-representative subset.

Always cross-tabulate missingness with other variables. Is income missing equally across education levels? Age groups? Geographic regions? If not, you have a pattern that needs addressing.

Mistake 3: Screening Only Numeric Variables

Categorical and text variables need screening too:

• Check category counts: Is "Gender" supposed to be M/F but you find 37 other values?
• Look for typos: "New York", "new york", "NY", "New York" (double space) are all different to a computer
• Check encoding: Special characters (é, ñ, ü) might display as gibberish if encoding is wrong
• Validate codes: State abbreviations should be two uppercase letters—if you find "Ca", "ca", "calif", they're all California but won't match

Run frequency tables for all categorical variables. Look at the long tail of rare categories. Many are data entry variations that need standardization.

Mistake 4: Screening Just Once

Data quality issues appear at different stages:

• Initial load: duplicates, impossible values, missing data
• After merging datasets: new missingness from non-matches, unit mismatches, ID conflicts
• After creating derived variables: outliers in calculated fields, division by zero errors
• After filtering: reduced sample size might reveal patterns hidden in full data

Screen at each stage. A dataset that looks clean initially might have problems after you join it with another source or subset to your analysis sample.

Mistake 5: Not Documenting Decisions

You screen the data, find issues, make decisions, and analyze. Two months later, a reviewer asks: "Why were these 47 records excluded?"

Without documentation, you can't answer. You don't remember. You have to re-screen, re-investigate, and hope you make the same decisions.

Document everything:

• What screening checks you ran
• What issues you found
• What decisions you made (keep, drop, transform, impute)
• Why you made those decisions

The MCP Analytics platform automatically generates screening reports with embedded R code and timestamps. You get a permanent record of what was checked, what was found, and when. That audit trail answers reviewer questions and lets you reproduce your decisions.

Building Data Screening Into Your Workflow

Data screening should be automatic, not optional. Here's how to make it a standard part of every analysis:

Step 1: Screen Immediately After Data Load

Before you write any analysis code, run screening. Upload your CSV to the screening tool, get the quality report, and review it. This takes 5 minutes and prevents days of wasted work analyzing bad data.

Make this non-negotiable. No analysis starts until you've screened and documented data quality.

Step 2: Create a Quality Checklist

For every dataset, check:

1. Duplicate records (exact and ID-based)
2. Impossible values (domain-specific rules)
3. Outliers (statistical thresholds + manual review)
4. Missing data (counts and patterns)
5. Distributions (normality, skewness for key variables)

Use the same checklist every time. Consistency helps you spot unusual patterns—if a dataset screens differently than similar data you've seen, that's a red flag.

Step 3: Separate Screening From Cleaning

Keep your screening script separate from cleaning. Screening is read-only—it assesses but doesn't modify. Cleaning modifies based on screening findings.

This separation creates an audit trail. Screening report documents issues as found. Cleaning script documents decisions and transformations. Anyone can see what was wrong and what you did about it.

Step 4: Screen Again After Major Transformations

Data quality can degrade during processing:

• Merging introduces missingness from non-matches
• Filtering changes distributions and might reveal hidden patterns
• Derived variables might have their own outliers or impossible values

Re-screen after major steps. Quick checks (row counts, range checks, missing data summaries) catch issues early.

Step 5: Include Screening in Analysis Reports

Your analysis report should include a data quality section:

• Original sample size and final analysis sample size
• What records were excluded and why
• What transformations were applied
• What assumptions were checked and results

This isn't methodological overkill—it's transparency. Readers need to know if your analysis is based on clean, representative data or if you had to make significant exclusions or adjustments.

The MCP Analytics platform makes this easy. Every analysis module starts with automated data screening. The output report includes a data quality section showing what was checked and what was found. You can review and approve before the main analysis runs. The full workflow—screening, cleaning decisions, and analysis—is documented in one reproducible report with embedded R code.

What Good Screening Enables

Systematic data screening doesn't just prevent errors—it unlocks better science:

Defensible Exclusions

You'll need to exclude some data. Reviewers will question those exclusions. Screening provides evidence.

"We excluded 23 records" is questionable. "We excluded 23 records with age > 120, verified as data entry errors via source data cross-check, representing 0.4% of sample" is defensible.

Appropriate Methods

Knowing your data's distribution lets you choose correct methods. If screening shows heavy right skew, you know to use non-parametric tests, transformations, or robust estimators rather than assuming normality and getting invalid p-values.

Sensitivity Analysis

When screening reveals ambiguous cases—outliers that might be real, missing data that might not be random—you can run analyses both ways. Report results with and without outliers. Show how conclusions change if you drop incomplete cases versus impute. That sensitivity analysis strengthens your conclusions.

Stakeholder Confidence

When you present results, stakeholders ask about data quality. "Did you check for duplicates?" "What about outliers?" "How much data was missing?"

If you screened systematically, you have answers. Show the screening report. Walk through what you checked, what you found, what you decided. That transparency builds confidence in your conclusions.

Reproducible Research

Documented screening means others can verify your decisions. They can see what quality issues existed, review your handling of them, and assess whether different decisions would change conclusions. That's how science works.

Limitations: What Screening Won't Catch

Be realistic about what data screening can and can't do:

Screening Won't Catch Systematic Measurement Error

If your survey question is poorly worded and everyone misunderstands it, screening won't help. The data will look fine statistically—no outliers, no missing values, normal distribution. But you're measuring the wrong thing. Screening checks data quality, not measurement validity.

Screening Won't Fix Selection Bias

If your sample is non-representative of your population, screening won't reveal that. You might have perfect data quality on a biased sample. Screening checks internal data quality, not external validity.

Screening Won't Identify All Impossible Values

Automated screening catches statistically unusual values. It won't catch domain-specific impossibilities unless you program explicit rules. Age = 247 is obvious. But "purchased enterprise software on free trial" might be impossible in your business, and screening won't know that without custom logic.

Screening Won't Make Decisions for You

Screening identifies issues. You still have to decide what to do. Keep outliers or remove them? Impute missing data or drop cases? Transform variables or use robust methods? Those are judgment calls that depend on your research question and domain knowledge.

Use screening as a diagnostic tool, not a decision engine.

Frequently Asked Questions

What's the difference between data screening and data cleaning?

Data screening is diagnostic—it tells you what's wrong with your data. Data cleaning is corrective—it fixes those issues. Think of screening as the quality inspection that happens before you decide what to repair. You screen first to understand the scope of problems, then clean based on what you find.

How do I know if an outlier is real or an error?

Check three things: (1) Is it physically possible? Age = 247 is an error. Revenue = $2.4M might be real. (2) Does it match source data? Cross-reference your original records. (3) Is there a pattern? One person with age 247 is probably a typo. Seven people all with age 99 might be a data entry convention for 'unknown.' Context matters more than statistical thresholds.

Should I remove outliers before running my analysis?

Not automatically. First, distinguish errors from legitimate extreme values. Remove clear errors (impossible values, data entry mistakes). For legitimate outliers, your decision depends on your research question. If you're studying typical customers, excluding extreme cases makes sense. If you're trying to understand the full range of behavior, you need to keep them. Document whatever you decide.

How much missing data is too much?

There's no universal threshold, but here's a practical framework: Under 5% missing and random pattern? Usually safe to proceed. 5-15% missing? Check if missingness relates to other variables. Over 15%? You need a formal plan for handling it (imputation, sensitivity analysis, or dropping the variable). More important than the percentage: Is the data missing completely at random, or does missingness correlate with your outcome?

Can I automate data screening?

You can automate detection, but you can't automate decisions. Automated checks will flag duplicates, impossible values, outliers, and missing data patterns. But deciding what to do about them requires domain knowledge. Is this outlier a billionaire customer or a data entry error? Is this missingness random or systematic? Use tools to surface issues, but apply human judgment to resolve them.

Start Screening Before Your Next Analysis

Data screening isn't optional preparation—it's essential quality control. Fifteen minutes of systematic screening prevents days of rework when you discover quality issues after analysis. More importantly, it ensures your conclusions rest on solid data, not garbage dressed up with sophisticated statistics.

Here's your action plan:

1. Screen your current dataset. Upload it to the Standard Data Screening tool right now. Get the quality report. See what issues exist that you haven't noticed.
2. Review the findings. Look at duplicates, outliers, missing data patterns. Don't just read the summary—look at the actual flagged values. Use domain knowledge to assess what's an error versus what's real.
3. Make documented decisions. Write down what you found and what you're doing about it. "Removed 3 records with age > 120 (data entry errors). Kept 7 customers with purchases > 200 (verified as corporate accounts). Used log transformation for lifetime_value due to right skew."
4. Make screening automatic. Add it to your workflow. Every new dataset gets screened before analysis starts. No exceptions. Build the habit now.

Quality conclusions require quality data. Screening is how you verify that foundation before you build on it.