While Python offers several ways to calculate correlations, Pingouin stands out by providing comprehensive statistical information in an easy-to-use package. In this article, we’ll explore how to perform correlation analysis using Pingouin to understand relationships between study habits, test performance, and stress levels.
Creating Our Dataset
Let’s create a realistic dataset that simulates the relationship between study habits, test performance, and stress levels for 100 students. We’ll generate:
- Study hours ranging from 1 to 8 hours
- Test scores from 0 to 100, influenced by study time
- Stress levels on a scale of 1-10, with a slight negative relationship to study hours
import numpy as np
import pandas as pd
import pingouin as pg
# Set random seed for reproducibility
np.random.seed(123)
# Generate realistic student data
n = 100 # Number of students
# Study hours (between 1 and 8 hours)
study_hours = np.random.uniform(1, 8, n)
# Test scores (influenced by study hours, between 0 and 100)
test_scores = study_hours * 8 + np.random.normal(50, 10, n)
test_scores = np.clip(test_scores, 0, 100)
# Stress levels (1-10 scale, slightly negatively related to study hours)
stress_levels = np.random.normal(7, 2, n) - (study_hours * 0.3)
stress_levels = np.clip(stress_levels, 1, 10)
# Create DataFrame
data = pd.DataFrame({
'Study_Hours': np.round(study_hours, 1),
'Test_Score': np.round(test_scores, 1),
'Stress_Level': np.round(stress_levels, 1)
})
print("First few students' data:")
print(data.head())
Our initial data shows a good spread of values. We can see students studying anywhere from about 2.6 to 6 hours, with corresponding test scores ranging from the mid-70s to 100. Stress levels vary between 4.5 and 6.9, giving us a realistic snapshot of student experiences.
First few students' data: Study_Hours Test_Score Stress_Level 0 5.9 100.0 4.5 1 3.0 75.7 6.9 2 2.6 82.2 6.6 3 4.9 76.2 5.5 4 6.0 100.0 5.6
Basic Correlation Analysis
One of Pingouin’s key strengths is its ability to provide comprehensive statistical information with minimal code. With just a single line of code (pg.corr()), we get a wealth of statistical metrics that would typically require multiple calculations using other Python packages. Let’s analyze how study time relates to test performance.
result = pg.corr(data['Study_Hours'], data['Test_Score'])
print("\nCorrelation between Study Hours and Test Scores:")
print(result)
Correlation between Study Hours and Test Scores:
n r CI95% p-val BF10 power
pearson 100 0.782493 [0.69, 0.85] 7.007341e-22 7.261e+18 1.0
These results provide a comprehensive view of the relationship:
1. Correlation Strength (r = 0.782)
– A strong positive correlation
– As study hours increase, test scores tend to increase reliably
– The relationship explains about 61% of the variation in test scores (r² = 0.61)
2. Confidence Interval ([0.69, 0.85])
– We can be 95% confident the true correlation lies between these values
– Even at its lowest estimate (0.69), the relationship is still strong
3. Statistical Significance (p-val = 7.01e-22)
– The extremely small p-value indicates this relationship is not due to chance
– Far below the conventional 0.05 threshold
4. Bayes Factor (BF10 = 7.261e+18)
– An extremely large value indicating overwhelming evidence for the relationship
– The data is over 7 quintillion times more likely under the hypothesis that there is a relationship than under the null hypothesis
5. Statistical Power (1.0)
– Perfect power (maximum possible value)
– Indicates we had more than enough data to detect this relationship
– Gives us high confidence in the reliability of our findings
Examining All Relationships
Let’s look at how all our variables relate to each other. Once again Pingouin provides a quick results with one line of code.
matrix = pg.pairwise_corr(data=data)
print("\nAll Relationships:")
print(matrix)
All Relationships:
X Y method alternative n r CI95% p-unc BF10 power
0 Study_Hours Test_Score pearson two-sided 100 0.782493 [0.69, 0.85] 7.007341e-22 7.261e+18 1.000000
1 Study_Hours Stress_Level pearson two-sided 100 -0.238508 [-0.42, -0.04] 1.686286e-02 2.084 0.672096
2 Test_Score Stress_Level pearson two-sided 100 -0.133314 [-0.32, 0.06] 1.860704e-01 0.296 0.263596
This matrix reveals several interesting patterns:
1. Study Hours and Test Scores
– Strong positive correlation (r = 0.782)
– Extremely significant (p = 7.01e-22)
– Very strong evidence (BF10 = 7.261e+18)
– Perfect statistical power
2. Study Hours and Stress Levels
– Weak negative correlation (r = -0.239)
– Statistically significant (p = 0.017, below 0.05)
– Moderate evidence (BF10 = 2.084)
– Decent but not optimal power (0.672)
– Suggests studying more might slightly reduce stress
3. Test Scores and Stress Levels
– Very weak negative correlation (r = -0.133)
– Not statistically significant (p = 0.186, above 0.05)
– Evidence actually favors no relationship (BF10 < 1)
– Low statistical power (0.264)
– Suggests test performance and stress aren’t meaningfully related
These varying p-values and Bayes Factors help us distinguish between reliable findings and potential noise in our data, with only the first two relationships showing evidence of genuine effects.
Understanding Correlation Methods
While Pingouin defaults to the Pearson correlation method, it offers several alternatives. Here’s when to use each:
1. Pearson (default)
– Best for linear relationships
– When you expect a unit change in X to relate to a consistent change in Y
– Most common in research settings
2. Spearman
– When you care about rank order rather than linear change
– When the relationship might be monotonic but not linear
– When extreme values might distort the relationship
3. Robust Methods
– Options like ‘shepherd’, ‘bicor’, ‘percbend’
– When outliers are a concern
Best Practices & Conclusion
When conducting correlation analyses, it’s good to follow some recommended steps. Before diving into the analysis, always examine your data carefully for outliers and consider the nature of the relationships you’re investigating. Ensure your sample size is adequate – generally at least 30 observations – to draw reliable conclusions. When reporting results, be thorough in including the correlation coefficient (r), sample size (n), and statistical significance (p-value), as these metrics together tell the complete story of your findings.
One of Pingouin’s greatest strengths is its ability to provide comprehensive statistical information with remarkably concise code. What would typically require multiple calculations and function calls in other Python packages can be achieved with just a single line using Pingouin. Our example demonstrated this efficiency, revealing not just basic correlations but also confidence intervals, Bayes factors, and statistical power in one straightforward output.
Through our analysis of student data, we discovered that study time strongly predicts test performance, explaining 61% of the variation in test scores. We also found subtle insights, such as the slight negative relationship between study hours and stress levels. These findings highlight how Pingouin’s rich statistical output helps researchers and analysts understand not just whether relationships exist, but also their strength, reliability, and practical significance. This combination of efficiency and thoroughness makes Pingouin a great tool for correlation analysis in Python.
