In statistics, we often use p-values to determine if there is a statistically significant difference between the mean of two groups.
However, while a p-value can tell us whether or not there is a statistically significant difference between two groups, an effect size can tell us how large this difference actually is.
One of the most common measurements of effect size is Cohen’s d, which is calculated as:
Cohen’s d = (x1 – x2) / √(s12 + s22) / 2
where:
- x1 , x2: mean of sample 1 and sample 2, respectively
- s12, s22: variance of sample 1 and sample 2, respectively
Using this formula, here is how we interpret Cohen’s d:
- A d of 0.5 indicates that the two group means differ by 0.5 standard deviations.
- A d of 1 indicates that the group means differ by 1 standard deviation.
- A d of 2 indicates that the group means differ by 2 standard deviations.
And so on.
Here’s another way to interpret cohen’s d: An effect size of 0.5 means the value of the average person in group 1 is 0.5 standard deviations above the average person in group 2.
We often use the following rule of thumb when interpreting Cohen’s d:
- A value of 0.2 represents a small effect size.
- A value of 0.5 represents a medium effect size.
- A value of 0.8 represents a large effect size.
The following example shows how to calculate Cohen’s d in R.
Example: How to Calculate Cohen’s d in R
Suppose a botanist applies two different fertilizers to plants to determine if there is a significant difference in average plant growth (in inches) after one month.
There are two methods we can use to quickly calculate Cohen’s d in R:
Method 1: Use lsr Package
library(lsr) #define plant growth values for each group group1 <- c(8, 9, 11, 11, 12, 14, 15, 16, 16, 18, 20, 21) group2 <- c(7, 9, 10, 10, 11, 11, 12, 14, 14, 16, 20, 23) #calculate Cohen's d cohensD(group1, group2) [1] 0.2635333
Method 2: Use effsize Package
library(effsize) #define plant growth values for each group group1 <- c(8, 9, 11, 11, 12, 14, 15, 16, 16, 18, 20, 21) group2 <- c(7, 9, 10, 10, 11, 11, 12, 14, 14, 16, 20, 23) #calculate Cohen's d cohen.d(group1, group2) Cohen's d d estimate: 0.2635333 (small) 95 percent confidence interval: lower upper -0.5867889 1.1138555
Notice that both methods produce the same result: Cohen’s d is 0.2635.
We interpret this to mean that the average height of plants that received fertilizer #1 is 0.2635 standard deviations greater than the average height of plants that received fertilizer #2.
Using the rule of thumb mentioned earlier, we would interpret this to be a small effect size.
In other words, whether or not there is a statistically significant difference in the mean plant growth between the two fertilizers, the actual difference between the group means is trivial.
Additional Resources
The following tutorials offer additional information on effect size and Cohen’s d:
Effect Size: What It Is and Why It Matters
How to Calculate Cohen’s d in Excel