How to Perform an ANOVA with Unequal Sample Sizes


One question students often have in statistics is:

Is it possible to perform a one-way ANOVA when the sample sizes of each group are not equal?

The short answer:

Yes, you can perform a one-way ANOVA when the sample sizes are not equal. Equal sample sizes is not one of the assumptions made in an ANOVA.

However, there are two potential issues to be aware of when performing a one-way ANOVA with unequal sample sizes:

(1) Reduced statistical power.

(2) Reduced robustness to unequal variance.

The following sections explain both of these potential issues in detail.

Issue #1: Reduced Statistical Power

When we use any type of statistical test to compare groups, the statistical power of the test is highest when each group has an equal sample size.

Recall that statistical power refers to the probability that a test will detect some effect when there actually is one.

It can be shown that the greater the differences in sample sizes between the groups, the lower the statistical power of an ANOVA.

This is why researchers typically want equal sample sizes so that they have higher power and thus a greater probability of detecting true differences.

It’s certainly possible to perform a one-way ANOVA with unequal sample sizes, but you should be aware that the power of the one-way ANOVA will be reduced.

Issue #2: Reduced Robustness to Unequal Variance

One of the assumptions of a one-way ANOVA is that the variance between each group is equal.

In general, a one-way ANOVA is considered to be robust against violations of the equal variances assumption, but only if each group has the same sample size.

Thus, if you have unequal sample sizes and unequal variances between the groups, then the results of the one-way ANOVA can be hard to trust.

How to Decide to Use a One-Way ANOVA with Unequal Sample Sizes

If you have unequal sample sizes and you’d like to perform a one-way ANOVA to test for differences between group means, you can use the following flow chart to decide how to proceed:

Flow chart for deciding to use one-way ANOVA with unequal sample sizes

Here’s a brief explanation of the flow chart:

Step 1: Determine if each group has the same variance.

To determine if each group has the same variance, you can use one of two approaches:

  • Create boxplots for each group and see if the spread of values in each group is roughly equal.
  • Perform a formal statistical test for equal variances like Bartlett’s Test.

If the variances are not equal, perform a Kruskal-Wallis Test, which is considered the non-parametric equivalent to a one-way ANOVA.

If the variances are equal, proceed to the next step.

Step 2: Determine if each group is normally distributed.

To determine if the values in each group are roughly normally distributed, you can use one of two approaches:

  • Create histograms or Q-Q plots for each group.
  • Perform formal statistical tests like Shapiro-Wilk, Kolmogorov-Smironov, Jarque-Barre, or D’Agostino-Pearson.

If each group is normally distributed, you can proceed to perform a one-way ANOVA and interpret the results as you would with any ordinary one-way ANOVA.

If each group is not normally distributed, perform a Kruskal-Wallis Test instead.

Additional Resources

An Introduction to the One-Way ANOVA
How to Check ANOVA Assumptions
An Introduction to the Kruskal-Wallis Test
How to Interpret the F-Value and P-Value in ANOVA

Leave a Reply

Your email address will not be published.