How to Run Paired T-Tests in Python with Pingouin

A paired t-test, also known as a dependent t-test, is commonly used when comparing two related groups—such as measuring the same participants before and after an intervention. While SciPy provides basic functionality to perform this analysis, Pingouin stands out by delivering extensive statistical details with minimal coding effort. In this article, we’ll explore how to use Pingouin for paired t-tests and compare its rich output to the standard results from SciPy.

Creating Sample Data for Paired Analysis

Let’s simulate a realistic scenario where we measure students’ test scores before and after implementing a new study technique. We’ll create data for 25 students:

import pandas as pd
import numpy as np

# Set pandas display options for single line
pd.set_option('display.max_columns', None)
pd.set_option('expand_frame_repr', False)

# Set random seed for reproducibility
np.random.seed(456)

# Generate pre-test scores (mean = 70, sd = 10)
pre_test = np.random.normal(loc=70, scale=10, size=25)

# Generate post-test scores (mean = 75, sd = 10) with correlation to pre-test
noise = np.random.normal(loc=0, scale=5, size=25)
post_test = pre_test + 5 + noise

# Create a DataFrame for better organization
data = pd.DataFrame({
    'Student_ID': range(1, 26),
    'Pre_Test': pre_test,
    'Post_Test': post_test
})

print("First few rows of our data:")
print(data.head())

Our sample data shows test scores for five students, with Pre_Test scores ranging from about 63 to 83 and Post_Test scores showing consistent improvement.

First few rows of our data:
   Student_ID   Pre_Test  Post_Test
0           1  63.318715  69.000574
1           2  65.017905  68.183164
2           3  76.185758  79.680331
3           4  75.686922  81.225121
4           5  83.505095  94.213269

Looking at just these first few rows, we can see improvements ranging from about 3 to 11 points, suggesting the intervention might have had a positive effect. However, we’ll need statistical analysis to properly quantify and validate this improvement.

Comprehensive Analysis with Pingouin

The Pingouin library provides a sophisticated approach to paired t-test analysis, offering a wealth of statistical information in a single function call. This makes it particularly valuable for researchers who need detailed statistical reporting without writing extensive code.

import pingouin as pg

# Perform paired t-test
results = pg.ttest(data['Pre_Test'], data['Post_Test'], paired=True)
print("\nPingouin Paired T-Test Results:")
print(results)

Pingouin Paired T-Test Results:
              T  dof alternative         p-val           CI95%   cohen-d       BF10     power
T-test -6.74933   24   two-sided  5.565409e-07  [-6.82, -3.63]  0.493282  3.069e+04  0.657966

Standard Approach Using SciPy

While SciPy provides a more streamlined approach to conducting paired t-tests, its output is notably more basic. This approach might be sufficient for simple analyses where only the fundamental test statistics are needed, but it lacks the comprehensive statistical metrics that might be required for thorough research reporting.

from scipy import stats

# Perform paired t-test with scipy
t_stat, p_val = stats.ttest_rel(data['Pre_Test'], data['Post_Test'])
print("\nScipy Results:")
print(f"T-statistic: {t_stat:.6f}")
print(f"P-value: {p_val:.6f}")

Scipy Results:
T-statistic: -6.749330
P-value: 0.000001

Deep Dive into Pingouin’s Rich Output

1. Basic Test Statistics

T-statistic (-6.749):
- The negative sign appears because we’re computing pre-test minus post-test
- The large magnitude indicates a substantial increase from pre-test to post-test
- Shows post-test scores were consistently higher than pre-test scores
- Matches perfectly with scipy’s output (to six decimal places), validating the calculation
Degrees of freedom (24):
- Reflects our sample size (n-1)
- Important for determining critical values and p-values
- Used in calculating confidence intervals and other metrics
P-value (5.57e-07):
- Extremely small (much less than conventional 0.05 threshold)
- Indicates the improvement is highly unlikely to be due to chance
- Matches scipy’s output (shown as 0.000001)
- Provides very strong evidence against the null hypothesis of no difference

2. Confidence Interval Analysis

CI95% [-6.82, -3.63]:
- Provides the range where we expect the true average improvement to lie
- The interval represents the mean difference calculated as “Pre-Test” – “Post-Test”
- Both bounds are negative, indicating consistent improvement
- We can be 95% confident that the true average improvement is between 3.63 and 6.82 points
- Narrow range (about 3.2 points) indicates good precision in our estimate
- Useful for practical decision-making (e.g., determining if improvement meets educational standards)

3. Effect Size Interpretation

Cohen’s d (0.493):
- Represents a medium effect size
- Falls just below the conventional benchmark of 0.5 for medium effect
- In educational contexts, this suggests a meaningful improvement
- Useful for:
  - Comparing results across different studies
  - Determining practical significance
  - Meta-analyses
- Not available in basic scipy output

4. Bayesian Analysis

Bayes Factor (BF10 = 30,690):
- Extremely strong evidence favoring the alternative hypothesis
- Indicates the data is over 30,000 times more likely under the alternative hypothesis
- Provides a different perspective than traditional null hypothesis testing
- Particularly useful when:
  - Making decisions about educational interventions
  - Comparing different teaching methods
  - Justifying continued use of an intervention
- Not available in basic scipy output

5. Statistical Power Assessment

Power (0.658):
- Our value (65.8%) is below the conventional 80% threshold
- Suggests potential improvements for future studies:
  - Increasing sample size beyond 25 students
  - Reducing measurement variability
  - More precise intervention implementation
- Important for:
  - Planning future studies
  - Interpreting null results
  - Understanding study limitations
- Not available in basic scipy output

Key Advantages of Using Pingouin

1. Comprehensive Statistical Package

One line of code provides six important statistical measures
Saves time compared to calculating each measure separately
Reduces chances of computational errors

2. Modern Statistical Approaches

Combines traditional (p-values) and modern (Bayes Factors) methods
Provides effect sizes for practical significance
Includes power analysis for study quality assessment

3. Clear and Accessible Output

Well-organized results in a single DataFrame
Easy to extract specific metrics for reporting
Facilitates thorough statistical reporting

Conclusion

The paired t-test implementation in Pingouin offers significant advantages over basic statistical packages. While SciPy provides just the t-statistic and p-value, Pingouin’s comprehensive output delivers deeper insights with minimal coding effort. In our educational example, the analysis revealed:

Strong statistical significance (p = 5.57e-07)
Practical significance (medium effect size, d = 0.493)
Very strong Bayesian evidence (BF10 = 30,690)
Average improvement between 3.63 and 6.82 points (95% CI)

These additional metrics make Pingouin particularly valuable for researchers and practitioners who need to provide thorough statistical reporting and make evidence-based decisions. The power analysis (65.8%) also provides valuable guidance for future research design, suggesting that larger sample sizes might be beneficial for similar studies.