One question students often have in statistics is:
Is it possible to perform a t-test when the sample sizes of each group are not equal?
The short answer:
Yes, you can perform a t-test when the sample sizes are not equal. Equal sample sizes is not one of the assumptions made in a t-test.
The real issues arise when the two samples do not have equal variances, which is one of the assumptions made in a t-test.
When this occurs, it’s recommended that you use Welch’s t-test instead, which does not make the assumption of equal variances.
The following examples demonstrate how to perform t-tests with unequal sample sizes when the variances are equal and when they’re not equal.
Example 1: Unequal Sample Sizes and Equal Variances
Suppose we administer two programs designed to help students score higher on some exam.
The results are as follows:
Program 1:
- n (sample size): 500
- x (sample mean): 80
- s (sample standard deviation): 5
Program 2:
- n (sample size): 20
- x (sample mean): 85
- s (sample standard deviation): 5
The following code shows how to create a boxplot in R to visualize the distribution of exam scores for each program:
#make this example reproducible set.seed(1) #create vectors to hold exam scores program1 <- rnorm(500, mean=80, sd=5) program2 <- rnorm(20, mean=85, sd=5) #create boxplots to visualize distribution of exam scores boxplot(program1, program2, names=c("Program 1","Program 2"))
The mean exam score for Program 2 appears to be higher, but the variance of exam scores between the two programs is roughly equal.
The following code shows how to perform an independent samples t-test along with a Welch’s t-test:
#perform independent samples t-test t.test(program1, program2, var.equal=TRUE) Two Sample t-test data: program1 and program2 t = -3.3348, df = 518, p-value = 0.0009148 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -6.111504 -1.580245 sample estimates: mean of x mean of y 80.11322 83.95910 #perform Welch's two sample t-test t.test(program1, program2, var.equal=FALSE) Welch Two Sample t-test data: program1 and program2 t = -3.3735, df = 20.589, p-value = 0.00293 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -6.219551 -1.472199 sample estimates: mean of x mean of y 80.11322 83.95910
The independent samples t-test returns a p-value of .0009 and Welch’s t-test returns a p-value of .0029.
Since the p-value of each test is less than .05, we would reject the null hypothesis in each test and conclude that there is a statistically significant difference in mean exam scores between the two programs.
Even though the sample sizes are unequal, the independent samples t-test and Welch’s t-test both return similar results since the two samples had equal variances.
Example 2: Unequal Sample Sizes and Unequal Variances
Suppose we administer two programs designed to help students score higher on some exam.
The results are as follows:
Program 1:
- n (sample size): 500
- x (sample mean): 80
- s (sample standard deviation): 25
Program 2:
- n (sample size): 20
- x (sample mean): 85
- s (sample standard deviation): 5
The following code shows how to create a boxplot in R to visualize the distribution of exam scores for each program:
#make this example reproducible set.seed(1) #create vectors to hold exam scores program1 <- rnorm(500, mean=80, sd=25) program2 <- rnorm(20, mean=85, sd=5) #create boxplots to visualize distribution of exam scores boxplot(program1, program2, names=c("Program 1","Program 2"))
The mean exam score for Program 2 appears to be higher, but the variance of exam scores for Program 1 is much higher than Program 2.
The following code shows how to perform an independent samples t-test along with a Welch’s t-test:
#perform independent samples t-test t.test(program1, program2, var.equal=TRUE) Two Sample t-test data: program1 and program2 t = -0.5988, df = 518, p-value = 0.5496 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -14.52474 7.73875 sample estimates: mean of x mean of y 80.5661 83.9591 #perform Welch's two sample t-test t.test(program1, program2, var.equal=FALSE) Welch Two Sample t-test data: program1 and program2 t = -2.1338, df = 74.934, p-value = 0.03613 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -6.560690 -0.225296 sample estimates: mean of x mean of y 80.5661 83.9591
The independent samples t-test returns a p-value of .5496 and Welch’s t-test returns a p-value of .0361.
The independent samples t-test is not able to detect a difference in mean exam scores, but the Welch’s t-test is able to detect a statistically significant difference.
Since the two samples had unequal variances, only Welch’s t-test was able to detect the statistically significant difference in mean exam scores since this test does not make the assumption of equal variances between samples.
Additional Resources
The following tutorials provide additional information about t-tests:
Introduction to the One Sample t-test
Introduction to the Two Sample t-test
Introduction to the Paired Samples t-test
I don’t pretend to understand any of the above but may I ask for guidance?
I am a member of a golf group of 18 players every Friday. We usually play in groups of four. Obviously, there are times when the number of players don’t split into exact fours and so we have various combinations of threes and fours. The perennial problem is how to create a level playing field to balance the threes against the fours. Our current technique is, after all cards are submitted, to select a player at random from the fourball card and use that score to enhance the three ball result. This ‘ghost’ player seems to work but is difficult for some members to grasp. Another suggestion is that the three ball group divides their score by 3 and multiplies by 4 – is this statistically fair as it doesn’t represent the larger playing group? Happy to receive any guidance but remember I’m not maths savvy.
Hi Tony…I think the simplest way to look at it is this: the “ghost player” idea is trying to give the three-ball group a fourth score so they can be compared with the four-ball groups. The problem is that using a random player from another card adds a bit of luck into it, because the three-ball could benefit from a very good or very poor randomly selected score.
Dividing the three-ball score by 3 and multiplying by 4 is probably easier for everyone to understand, and in many cases it is a fairer approach because it simply says, “What was the average score per player in this group, and what would that look like for four players?” It is not perfect, but it is consistent and transparent.
So, for example, if the three players score 30, 32 and 34 points, their total is 96. Divide by 3 and the average is 32. Multiply by 4 and their adjusted team score becomes 128.
That said, the fairest method depends a bit on the format being played. If you are simply adding everyone’s points together, then average × 4 is a very reasonable method. If you are using best-ball scoring, where only the best score on each hole counts, then it gets trickier because a four-ball has more chances to produce a good score on each hole.
My personal view would be to use the average × 4 method because it is easy to explain, applies the same rule every time, and avoids the confusion of borrowing a random “ghost” player from another card. It may not be mathematically perfect, but for a Friday golf group it is probably fair enough and much easier for everyone to accept.