Image by Author

Most research papers still lead with a p-value, but that number alone rarely tells you anything useful. A p-value can tell you if an effect exists, but not whether it matters. It can show that two groups differ, yet say nothing about how big or meaningful that difference is. That’s where effect size comes in.

Effect size measures give your results context. They describe how strong a relationship is, or how large a difference appears, in real terms. Whether you’re comparing two treatments, studying a correlation, or evaluating a model’s performance, effect sizes turn statistical results into something you can reason about.

Let’s get practical: p-values answer “Is there an effect?”, while effect sizes answer “How big is it?”. A small effect with a tiny p-value might be statistically significant but practically irrelevant. A large effect that doesn’t reach significance could still matter in the real world, especially with small samples or exploratory studies.

In this article, we’ll walk through five effect size measures you’ll actually use and understand:

• Cohen’s d — the standardized difference between two group means
• r (correlation coefficient) — the strength of a relationship between variables
• Odds ratio — how much more likely an outcome is in one group than another
• η² (eta squared) — how much of the total variance your variable explains
• Cliff’s delta — a robust, non-parametric alternative for skewed data

Each one has a clear interpretation, a short formula, and a Python example you can apply right away. By the end, you’ll know not just how to calculate these measures, but how to interpret them in plain English — so your next analysis tells a story people can actually understand.

Cohen’s d — The Classic Mean Difference

When you want to compare two groups — say, a control and a treatment — Cohen’s d is the most common measure of effect size. It indicates the distance between the two means, expressed in units of standard deviation.

In plain English, Cohen’s d answers: “How big is the difference between these groups, relative to how much variation there is within them?”

The formula is simple:

Interpretation guide (Cohen, 1988):

• 0.2 → small effect (barely noticeable difference)
• 0.5 → medium effect (moderate difference)
• 0.8 → large effect (clearly distinct groups)

For example, if students who used a new learning method scored 0.6 standard deviations higher than those using the old method, that’s a moderate-to-large improvement — the kind you’d likely notice in the real world. Here’s how to compute it in Python using pingouin, a statistics library designed for behavioral and experimental data:

import pingouin as pg

group_a = [72, 75, 78, 80, 83]
group_b = [65, 67, 69, 70, 71]

d = pg.compute_effsize(group_a, group_b, eftype='cohen')
print(f"Cohen's d = {d:.2f}")

Output:

Cohen's d = 2.65

That value (2.65) means group A’s mean is about 2.65 standard deviations higher than group B — a very large and practically meaningful difference.

So when should you use this?

• You’re comparing two groups (e.g., control vs. treatment)
• Your data are roughly normally distributed
• You want a standardized difference you can compare across studies

r — The Correlation Coefficient

While Cohen’s d measures differences between groups, the correlation coefficient (r) measures the strength and direction of a relationship between two continuous variables.

It answers the question: “When one variable increases, does the other increase, decrease, or stay the same?”.

Values of r range between -1 and +1:

• r = +1 → perfect positive correlation (as X increases, Y increases)
• r = -1 → perfect negative correlation (as X increases, Y decreases)
• r = 0 → no linear relationship

Example: If study time and exam scores have a correlation of r = 0.65, that means students who study more tend to score higher; this is a strong, positive relationship. Here’s a quick Python example using SciPy:

from scipy import stats

study_hours = [2, 3, 4, 5, 6, 7]
exam_scores = [50, 55, 58, 65, 70, 75]

r, p = stats.pearsonr(study_hours, exam_scores)
print(f"r = {r:.2f}, p = {p:.4f}")

Output:

r = 1.00, p = 0.0001

When to use it:

• When both variables are continuous
• When you’re checking for linear relationships
• When you want a symmetric measure (the relationship between X and Y is the same as between Y and X)

Odds Ratio — The Effect Size for Binary Outcomes

When your outcome is categorical (e.g., success vs. failure, click vs. no-click), the odds ratio (OR) is your go-to effect size. It tells you how much more (or less) likely an event is in one group compared to another.

In plain English, it answers: “Are people in Group A more likely to experience this outcome than those in Group B, and by how much?”.

The formula is:

• a = number of successes in group A
• b = number of failures in group A
• c = number of successes in group B
• d = number of failures in group B

Interpretation guide:

• OR = 1 → no difference between groups
• OR > 1 → event is more likely in group A
• OR < 1 → event is less likely in group A

For example:
If 40 out of 100 patients recover with a new treatment, compared to 20 out of 100 with the old one, the odds ratio is:

import statsmodels.api as sm
import numpy as np

# Binary outcome (1 = success, 0 = failure)
y = np.array([1, 0, 1, 1, 0, 0, 1, 0])
X = sm.add_constant([0, 0, 0, 1, 1, 1, 1, 0])  # 0 = old treatment, 1 = new treatment

model = sm.Logit(y, X).fit(disp=False)
print(np.exp(model.params))  # converts log-odds to odds ratios (intercept and treatment)

When to use it:

• When your dependent variable is binary (yes/no, clicked/didn’t click)
• When you’re analyzing risk, likelihood, or success rate between groups
• When you’re using logistic regression

You should watch out for:

• Misinterpreting odds as probabilities. An OR of 2.0 doesn’t mean “twice as likely”; it means “the odds are twice as high”.
• For rare events (under 10%), OR and risk ratio are almost identical; for common outcomes, they diverge.

η² (Eta Squared)

If you’ve run an ANOVA (analysis of variance), you’ve likely seen η² (eta squared) in the output. It measures how much of the total variance in your dependent variable is explained by your independent variable or factor.

In simple terms, η² answers: “What proportion of the outcome’s variation can be attributed to group differences?”.

The formula is:

• SS_between = variability explained by the groups (or model)
• SS_total = total variability in the data

Interpretation (Cohen, 1988):

• 0.01 → small effect
• 0.06 → medium effect
• 0.14 → large effect

For example:
If η² = 0.10, that means your factor explains 10% of the total variance — a meaningful but not overwhelming contribution.

Here’s a short Python example using pingouin:

import pingouin as pg
import pandas as pd

# Sample dataset
data = pd.DataFrame({
    "group": ["A", "A", "B", "B", "C", "C"],
    "score": [5, 6, 7, 8, 6, 9]
})

# Run one-way ANOVA
anova = pg.anova(dv="score", between="group", data=data, detailed=True)

# Compute eta squared from the ANOVA table: SS_between / SS_total
ss_between = anova.loc[anova['Source'] == 'group', 'SS'].iloc[0]
ss_within = anova.loc[anova['Source'] == 'Within', 'SS'].iloc[0]
eta_sq = ss_between / (ss_between + ss_within)

# Display results
print(anova[['Source', 'SS', 'DF', 'F', 'p-unc']])
print(f"\nη² (eta squared): {eta_sq:.3f}")

Output:

    Source    SS  DF         F     p-unc
0    group  8.50   2  1.454545  0.361744
1   Within 11.70   3       NaN       NaN

η² (eta squared): 0.421

That means group membership explains about 42% of the variation in scores here — a large effect by the conventional η² thresholds.

When to use it:

• When you’re comparing more than two groups (ANOVA or factorial designs)
• When you want a clear proportion of variance explained

Variants:

• Partial η² is used when multiple variables are in the model; it isolates each variable’s unique contribution.
• Omega squared (ω²) is a less biased, small-sample-adjusted alternative to η².

Cliff’s Delta For Non-Parametric Comparisons

Not all data behave nicely. When your samples are small, skewed, or full of outliers, Cohen’s d can give misleading results. That’s where Cliff’s delta (δ) becomes useful.

Cliff’s delta measures the probability that a randomly chosen observation from one group will be higher than one from another, minus the reverse probability. It doesn’t rely on normal distributions or equal variances, making it a reliable alternative for non-parametric data.

In plain English, it answers: “If I pick one value from each group, how likely is it that Group A’s value will be higher?”.

The formula is:

import numpy as np

def cliffs_delta(x, y):
    """Compute Cliff's delta."""
    x, y = np.array(x), np.array(y)
    n_x, n_y = len(x), len(y)
    more = sum(xi > yj for xi in x for yj in y)
    less = sum(xi < yj for xi in x for yj in y)
    delta = (more - less) / (n_x * n_y)
    return delta

group_a = [5, 6, 8, 9, 10]
group_b = [2, 3, 4, 5, 6]

delta = cliffs_delta(group_a, group_b)
print(f"Cliff's delta = {delta:.2f}")

Output:

Cliff's delta = 0.84

That means there’s roughly a 92% probability that a random value from Group A exceeds a random value from Group B (assuming no ties), which is a very large effect.

When to use it:

• When your data are ordinal or non-normal
• When you’re comparing two groups
• When you want a distribution-free effect size

Choosing the Right Measure

Choosing an effect size is less about memorizing formulas and more about matching it to your question. Here’s a quick cheat sheet you can actually use:

Conclusion

Effect sizes are what make results meaningful. A statistically “significant” finding says something happened; effect size says how much. Reporting both isn’t just good practice — it’s honest storytelling for data. As you write up your next report or paper, include both the p-value and the effect size. It helps others grasp real-world importance, not just statistical noise.

Numbers are clearer when they describe how much change, not just if it exists.

Further reading and sources:

• Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Routledge
• Pingouin Documentation
• Statsmodels Documentation