How to Perform Independent T-Tests in Python Using Pingouin

Looking for a more efficient way to conduct t-tests in Python? Pingouin might be the solution you’ve been seeking.

An independent t-test (also called two-sample t-test) is used to determine whether there is a significant difference between the means of two independent groups. For example, comparing test scores between two different teaching methods or comparing heights between two populations. While Python offers several packages for statistical testing, Pingouin stands out for its simplicity and comprehensive output.

The Setup

First, let’s generate some sample data and settings for our output. In this example, we’ll work with two groups of 30 samples each, where group2 has a slightly higher mean (5.5 vs 5) than group1. This mimics a typical experimental setup where we’re testing for group differences.

import pandas as pd
import numpy as np

# Set pandas display options for single line
pd.set_option('display.max_columns', None)
pd.set_option('expand_frame_repr', False)

# Generate sample data
np.random.seed(123)
group1 = np.random.normal(loc=5, scale=1, size=30)
group2 = np.random.normal(loc=5.5, scale=1, size=30)

Approach 1: Using Pingouin

Let’s see how Pingouin can handle this analysis with remarkable efficiency:

import pingouin as pg
# Perform t-test
results = pg.ttest(group1, group2, paired=False)
print(results)

Output:

               T  dof alternative     p-val          CI95%   cohen-d   BF10     power
T-test -1.909932   58   two-sided  0.061091  [-1.22, 0.03]  0.493142  1.189  0.467509

As we can see, Pingouin provides a wealth of information in a single output. Beyond the basic t-test statistics, we get effect size (cohen-d), confidence intervals (CI95%), Bayes Factor (BF10), and statistical power – all from a single line of code. This comprehensive output follows current best practices in statistical reporting, saving researchers significant time in calculating these additional metrics.

Approach 2: Using Statsmodels

The basic statsmodels approach provides core statistics:

from statsmodels.stats import weightstats as stm
# Perform t-test with statsmodels
t_stat, p_val, dof = stm.ttest_ind(group1, group2)
print("\nStatsmodels Results:")
print(f"T-statistic: {t_stat:.6f}")
print(f"P-value: {p_val:.6f}")
print(f"Degrees of freedom: {dof}")

Output:

Statsmodels Results:
T-statistic: -1.909932
P-value: 0.061091
Degrees of freedom: 58.0

The statsmodels output provides the fundamental t-test statistics. While these core statistics are accurate and match pingouin’s results, researchers would need additional code to calculate other important metrics like effect size, confidence intervals, or power analysis. This makes statsmodels a solid choice for basic analyses but requires more work for comprehensive statistical reporting. This highlights a key advantage of Pingouin: it combines what would typically require multiple statistical functions and calculations into a single, comprehensive output.

Interpreting the Results

Let’s break down all the statistics provided by Pingouin:

1. Basic Test Statistics (available in both packages):

The t-statistic (-1.91) and its 58 degrees of freedom (dof) represent the standardized difference between groups
The p-value (0.061) being greater than 0.05 suggests the difference is not statistically significant

2. Confidence Interval (CI95%):

The interval [-1.22, 0.03] represents the range where we can be 95% confident the true population difference lies
Since this interval contains zero, it supports our non-significant finding
The width of the interval gives us information about the precision of our estimate

3. Effect Size (cohen-d):

Cohen’s d = 0.493 indicates a medium effect size
This standardized measure helps us understand the practical significance, regardless of sample size
Guidelines: 0.2 = small, 0.5 = medium, 0.8 = large effect
Even though our result wasn’t statistically significant, the medium effect size suggests the difference might be practically meaningful

4. Bayes Factor (BF10):

BF10 = 1.189 represents the odds ratio for the alternative hypothesis versus the null hypothesis
Values between 1-3 suggest anecdotal evidence for the alternative hypothesis
This provides an alternative to traditional null hypothesis testing
Helps quantify the evidence for/against an effect, rather than just reject/fail to reject

5. Statistical Power:

Power = 0.467 (or 46.7%) indicates the probability of detecting a true effect if one exists
This is below the conventional 80% threshold
Suggests our study might be underpowered
Could guide future research by indicating a need for larger sample sizes

Conclusion

While both packages can perform the basic t-test calculations with similar accuracy, Pingouin provides a much more comprehensive statistical output in a single line of code. The additional metrics (effect size, confidence intervals, Bayes Factor, and power) offer valuable insights for more thorough statistical reporting and interpretation. This makes Pingouin particularly valuable for researchers who need to report multiple aspects of their statistical analyses or make informed decisions about their findings. For future analyses, consider starting with Pingouin when you need quick access to comprehensive statistical metrics, especially when reporting requirements include effect sizes and power analyses.