In the world of statistics and data science, it is easy to believe that numbers tell a straightforward story. Sometimes, though, certain ways of looking at data can obscure the truth. Occasionally, in a phenomenon known as Simpson’s Paradox, aggregated data can lead to conclusions that contradict underlying trends. Understanding this paradox and why it happens is a key reminder that context matters in any data analysis process.  

What is Simpson’s Paradox?

Simpson’s Paradox occurs when a trend that appears in aggregated data reverses when the data is divided into subgroups. It demonstrates that relationships between variables can change depending on how the data is partitioned. This specifically occurs when there is a confounding variable that influences the variables of interest and creates a misleading aggregation.  

The phenomena was first formally described by Edward Simpson, a British statistician, in 1951. However, it had been observed as far back is 1899 by British mathematician Karl Pearson, the same Pearson of the Pearson Correlation coefficient. He presented many examples of how relationships between different variables could be misinterpreted upon aggregation when certain hidden factors were ignored.  

A Classic Example of Simpson’s Paradox

A classic example of Simpson’s paradox is a case of university admissions. Consider a university with only two departments, Humanities and Science, with the following acceptance rates for men and women: 

Humanities

• 150 men apply, 75 are admitted (50% acceptance rate).
• 50 women apply, 25 are admitted (50% acceptance rate).

Science

• 8 men apply, 2 are admitted (25% acceptance rate).
• 80 women apply, 32 are admitted (40% acceptance rate).

In aggregate, the university accepted about 49% of male applicants and 44% of female applicants, which may suggest a gender bias in favor of male applicants. However, if we consider departmental data, men and women are accepted at equal rates in the humanities and a higher percentage of women are actually accepted in science.  

The apparent disparity in the aggregated data arises because women disproportionately apply to the science department, which is more competitive and harder to get into overall. There is a confounding variable of department competitiveness, which skews the aggregated data. When the data is properly stratified, it becomes clear that there is no systematic bias present.  

More Examples of Simpson’s Paradox

Simpson’s Paradox can appear anywhere where aggregated data exists. Here are three more examples to demonstrate the paradox.  

• Voting Patterns. When looking at the aggregated data, Candidate A may have won overall. However, Candidate B may have received a greater percentage of votes among both urban or rural voters. The paradox arises if the proportion of urban and rural voters in the overall population is heavily skewed, disproportionately affecting the aggregated results.
• Sales Performance. Overall, Bob may have more sales close than Alice, but Alice could have a higher rate of success within each product category. The discrepancy comes when there is a vastly different distribution in the number of sales within each category made by each employee.
• Hospital Survival Rates. County Hospital may have a higher overall survival rate than State Hospital, but if rates are stratified by patient severity, State Hospital may have higher rates in all categories. This can happen if State Hospital has a much higher proportion of critically ill patients.

Why Does Simpson’s Paradox Happen?

Fundamentally, Simpson’s Paradox occurs when there is an interplay between confounding variables and aggregation of data. Hidden factors influence the relationship between the variables of interest. For example, in healthcare studies, patient severity might confound the relationship between treatment type and recovery rate.  

There can also be aggregation bias where the simple act of aggregating data can obscure subgroup-specific relationships, leading to trends that are opposite to those seen within the groups. This is made more problematic when subgroup sizes differ significantly.  

Implications of Simpson’s Paradox

A key implication of Simpson’s Paradox is that it can lead to poor policy decisions when data is misinterpreted. It is easy to draw incorrect conclusions when overlooking subgroup differences, leading to flawed studies or models.  

Ignoring subgroup trends can exacerbate inequalities or biases, particularly in areas like healthcare or education, where decisions have profound impacts. Even in business contexts, aggregated customer satisfaction ratings might suggest that a company’s overall performance is strong, but subgroup analysis could reveal significant dissatisfaction among key customer segments. Acting on the aggregated data alone might result in missed opportunities to address specific issues.  

How to Avoid Being Mislead by Simpson’s Paradox

In cases where prominent subgroups exist in the data, it is best to always investigate data at this granular level. This will help quickly identify trends that may deviate from what the larger story says. Identification of potential confounders through statistical methods like stratification, multivariate regression, or propensity score matching can help find these subgroups to stratify by.  

Graphical representations, such as stratified bar charts or scatter plots, can reveal trends within subgroups that aggregated data might hide. Full exploration of the raw data is a critical part of the analytics cycle and can help find where Simpson’s Paradox may arise.  

Summary

Simpson’s Paradox is a staple example in statistics education, illustrating the critical need for understanding confounding variables and stratification in data analysis. While aggregated data may suggest one trend, subgroup analysis often tells a different—and more accurate—story. Ultimately, Simpson’s Paradox is a valuable lesson in the importance of context, critical thinking, and the nuanced interpretation of data.