A one-way ANOVA is used to determine whether or not there is a statistically significant difference between the means of three or more independent groups.
If the overall p-value from the ANOVA table is less than some significance level, then we have sufficient evidence to say that at least one of the means of the groups is different from the others.
However, this doesn’t tell us which groups are different from each other. It simply tells us that not all of the group means are equal. In order to find out exactly which groups are different from each other, we must conduct a post hoc test.
One of the most commonly used post hoc tests is Tukey’s Test, which allows us to make pairwise comparisons between the means of each group while controlling for the family-wise error rate.
This tutorial explains how to perform Tukey’s Test in R.
Note: If one of the groups in your study is considered a control group, you should instead use Dunnett’s Test as the post-hoc test.
Example: Tukey’s Test in R
Step 1: Fit the ANOVA Model.
The following code shows how to create a fake dataset with three groups (A, B, and C) and fit a one-way ANOVA model to the data to determine if the mean values for each group are equal:
#make this example reproducible set.seed(0) #create data data <- data.frame(group = rep(c("A", "B", "C"), each = 30), values = c(runif(30, 0, 3), runif(30, 0, 5), runif(30, 1, 7))) #view first six rows of data head(data) group values 1 A 2.6900916 2 A 0.7965260 3 A 1.1163717 4 A 1.7185601 5 A 2.7246234 6 A 0.6050458 #fit one-way ANOVA model model <- aov(values~group, data=data) #view the model output summary(model) Df Sum Sq Mean Sq F value Pr(>F) group 2 98.93 49.46 30.83 7.55e-11 *** Residuals 87 139.57 1.60 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
We can see that the overall p-value from the ANOVA table is 7.55e-11. Since this is less than .05, we have sufficient evidence to say that the mean values across each group are not equal. Thus, we can proceed to perform Tukey’s Test to determine exactly which group means are different.
Step 2: Perform Tukey’s Test.
The following code shows how to use the TukeyHSD() function to perform Tukey’s Test:
#perform Tukey's Test TukeyHSD(model, conf.level=.95) Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = values ~ group, data = data) $group diff lwr upr p adj B-A 0.9777414 0.1979466 1.757536 0.0100545 C-A 2.5454024 1.7656076 3.325197 0.0000000 C-B 1.5676610 0.7878662 2.347456 0.0000199
The p-value indicates whether or not there is a statistically significant difference between each program. We can see from the output that there is a statistically significant difference between the mean weight loss of each program at the 0.05 significance level.
In particular:
- P-value for the difference in means between B and A: .0100545
- P-value for the difference in means between C and A: .0000000
- P-value for the difference in means between C and B: .0000199
Step 3: Visualize the results.
We can use the plot(TukeyHSD()) function to visualize the confidence intervals as well:
#plot confidence intervals plot(TukeyHSD(model, conf.level=.95), las = 2)
Note: The las argument specifies that the tick mark labels should be perpendicular (las=2) to the axis.
We can see that none of the confidence intervals for the mean value between groups contain the value zero, which indicates that there is a statistically significant difference in mean loss between all three groups. This is consistent with the fact that all of the p-values from our hypothesis tests are below 0.05.
For this particular example, we can conclude the following:
- The mean values of group C are significantly higher than the mean values of both group A and B.
- The mean values of group B are significantly higher than the mean values of group A.
Additional Resources
A Guide to Using Post Hoc Tests with ANOVA
How to Conduct a One-Way ANOVA in R
How to Conduct a Two-Way ANOVA in R
Can you specify the results of the Tukey test to only show the rows that have significance (p < 0.05)? On the chance that the Tukey test has too many interactions, such as a three-way interaction, and therefore has too many rows.
Hi Kristi…To filter the results of Tukey’s test in R to show only the rows with a significance level \( p < 0.05 \), you can process the output of the `TukeyHSD` function. Here's how you can do it: ### Steps: 1. Perform your Tukey test using the `TukeyHSD` function. 2. Extract the results as a data frame. 3. Filter the rows based on the \( p \)-value threshold (\( p < 0.05 \)). Here's a code example: ```R # Example ANOVA model data(iris) model <- aov(Sepal.Length ~ Species, data = iris) # Perform Tukey's Test tukey_results <- TukeyHSD(model) # Extract the results for a specific factor (e.g., 'Species') tukey_species <- as.data.frame(tukey_results$Species) # Add row names as a column for clarity tukey_species <- cbind(Comparison = rownames(tukey_species), tukey_species) # Filter for significant comparisons (p-value < 0.05) significant_results <- tukey_species[tukey_species$`p adj` < 0.05, ] # View significant results print(significant_results) ``` ### Explanation: 1. **Extracting Results**: - The `TukeyHSD` function returns a list where each element corresponds to a factor in the model. - Each element contains a matrix with comparisons and their statistics. 2. **Converting to a Data Frame**: - The specific component (e.g., `tukey_results$Species`) is converted to a data frame for easier manipulation. 3. **Filtering**: - The `p adj` column represents the adjusted \( p \)-values. - Rows with \( p adj < 0.05 \) are retained in the filtered results. 4. **Adding Readability**: - Adding a `Comparison` column makes it clear which pairs the rows correspond to. ### Output: The final `significant_results` will contain only the rows where the \( p \)-value is below the significance threshold. This approach works even if your Tukey test has multiple factors or interactions, as you can repeat the extraction and filtering for each component in the `TukeyHSD` output.