Image by AuthorThrough years of mentoring data science students, I’ve noticed missing data handling is where the worst statistical errors happen. Unlike coding mistakes, missing data problems are silent. Your analysis runs successfully, but your conclusions are wrong because you mishandled missingness.
I’ve reviewed analyses where students concluded treatments were effective when they weren’t, simply because they deleted missing values incorrectly. This guide shares the most common missing data mistakes I encounter, each one capable of throwing off your entire analysis.
Key Terms: Missing Data Mechanisms
This article uses three acronyms that describe different patterns of missing data. Here’s what they mean:
MCAR (Missing Completely At Random): The probability of data being missing is unrelated to any variables in your dataset, observed or unobserved. For example, a sensor randomly fails 2% of the time regardless of what it’s measuring. This is the best-case scenario and the rarest pattern in real data.
MAR (Missing At Random): The probability of data being missing depends on other observed variables but not on the missing value itself. For example, younger people are less likely to report their income, but among people of the same age, income level doesn’t affect whether they report it. This is the most common pattern you’ll encounter.
MNAR (Missing Not At Random): The probability of data being missing depends on the unobserved value itself. For example, people with very high incomes are less likely to report their income. This is the most problematic pattern because standard methods can’t fully correct for it.
Before you can identify which mechanism you’re dealing with, ask a more basic question: why is this data missing? Sometimes what looks like missing data isn’t actually missing. In housing datasets, a blank “Pool Quality” field might mean “this house has no pool” rather than “we forgot to measure pool quality.” This is structural missingness, not statistical missingness, and the solution is encoding it as its own category (“No_Pool”) rather than imputation or deletion. Once you’ve ruled out structural missingness, you can focus on identifying whether remaining gaps are MCAR, MAR, or MNAR.
Quick Context: What Makes Missing Data So Dangerous?
Missing data appears in almost every real dataset. Survey respondents skip questions, patients miss appointments, sensors fail. The problem isn’t that data is missing but how missingness relates to your variables.
When data is missing completely at random (MCAR), you can delete those cases safely. But usually, missingness is systematic. People with depression skip mental health questions. Low-income respondents don’t report income. The missingness itself contains information, and ignoring this introduces bias.
Common Mistake #1: Deleting All Missing Cases Without Checking the Pattern
The most frequent error? Discovering missing values and immediately removing every row containing them. This feels logical but often backfires badly.
Recently, a student analyzing income and health outcomes had 8% missing income data and deleted those 800 cases. When we investigated, missing income cases were disproportionately from lower-income neighborhoods. Deletion artificially inflated average income and missed the strongest part of the income-health relationship.
Always check whether data is MCAR before deletion. Run tests comparing characteristics of cases with and without missing data. If systematic differences exist, listwise deletion will bias your results.
Common Mistake #2: Replacing Missing Values With Mean or Median by Default
Once students learn deletion can bias results, many discover mean imputation and think they’ve found the solution. Just replace missing values with the mean. This preserves sample size but creates serious problems.
A student once analyzed salary data with 15% missing values, replacing them with mean salary of $75,000. Their regression showed education-salary relationships much weaker than published literature. Why? Mean imputation reduced salary variance, artificially compressing all predictor relationships.
Use mean imputation only when you have strong evidence values are MCAR and missingness is very low (under 5%). Otherwise, you’ll create artificial precision that misleads readers.
Common Mistake #3: Ignoring Whether Missingness Is Related to Unobserved Values
This is the most conceptually challenging mistake. Missing not at random (MNAR) occurs when missingness probability depends on the missing value itself. People with very high incomes don’t report income. Patients with severe depression don’t complete depression questionnaires.
You can’t definitively test for MNAR using your data alone because you don’t observe the missing values. Yet students confidently apply techniques that assume MAR when the pattern suggests MNAR.
When you suspect MNAR, conduct sensitivity analyses showing how conclusions change under different assumptions. This doesn’t solve the problem, but it prevents presenting biased results as certain truth.
Common Mistake #4: Using Complete Case Analysis When Data Is MAR
Many students assume that if they can’t prove data is MCAR, they should use complete case analysis as the “conservative” choice. This is backward. When data is MAR, complete case analysis is often more biased than modern methods.
A student once analyzed customer satisfaction data where older customers were less likely to respond to online surveys, creating 25% missing data. Using complete case analysis effectively analyzed only younger customers, then made conclusions about all customers. Multiple imputation with age as predictor changed satisfaction estimates by 15 percentage points.
When data is MAR, multiple imputation typically outperforms complete case analysis.
Common Mistake #5: Failing to Report Missing Data Handling in Methods
The most overlooked mistake isn’t statistical but methodological: students don’t document how they handled missing data. I’ve reviewed projects where methods sections mention original and final sample sizes but say nothing about thousands of cases that vanished between them.
I’ve seen students present regression results from 5,000 observations but actually use 3,200. When asked about the 1,800 missing cases, they say they “cleaned the data” but can’t remember specifics.
Your methods section should state: missing data percentages for key variables, the missingness pattern, the technique you used, and any sensitivity analyses you conducted.
How to Avoid These Mistakes: Quick Checklist
Before analyzing data with missing values:
- Examine missing data percentage for each variable and look for patterns
- Test whether data is MCAR using statistical tests and visual inspection
- Consider whether missingness might relate to unobserved values based on domain knowledge
- Choose your method based on mechanism: deletion only for MCAR with low missingness, multiple imputation for MAR
- Conduct sensitivity analyses when you suspect MNAR
- Document every decision about missing data handling in methods
- Report sample sizes at each analysis stage
Conclusion
These mistakes represent the most common missing data errors I’ve observed in mentoring sessions. All are preventable with careful attention to missing data mechanisms.
Students who invest time understanding these patterns produce far more trustworthy analyses than those treating missing data as a quick nuisance to fix.
