Cochran’s Q test is a statistical test that is used to determine whether the proportion of “successes” is equal across three or more groups in which the same individuals appear in each group.
For example, we may use Cochran’s Q test to determine if the proportion of students who pass a test is equal when using three different studying techniques.
Steps to Perform Cochran’s Q Test
Cochran’s Q test uses the following null and alternative hypotheses:
Null Hypothesis (H0): The proportion of “successes” is the same in all groups
Alternatve Hypothesis(HA): The proportion of “successes” is different in at least one of the groups
The test statistic is calculated as:
- k: The number of treaments (or “groups”)
- X.j: The column total for the jth treatment
- b: The number of blocks
- Xi.: The row total for the ith block
- N: The grand total
The test statistic T follows a Chi-Square distribution with k-1 degrees of freedom.
If the p-value associated with the test statistic is less than a certain significance level (like α = .05), we can reject the null hypothesis and conclude that we have sufficient evidence to say the proportion of “successes” is different in at least one of the groups.
Example: Cochran’s Q Test
Suppose a researcher wants to know if three different studying techniques lead to different proportions of pass rates among students.
To test this, she recruits 20 students to each take an exam of equal difficulty using three different studying techniques. The results are shown below:
To perform Cochran’s Q test, we can use statistical software since it can be cumbersome to perform by hand.
Here’s the code we can use to create this dataset and perform Cochran’s Q test in the statistical programming language R:
#load DescTools package library(DescTools) #create dataset df <- data.frame(student=rep(1:20, each=3), technique=rep(c('A', 'B', 'C'), times=20), outcome=c(1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1)) #perform Cochran's Q test CochranQTest(outcome ~ technique| student, data=df) Cochran's Q test data: outcome and technique and student Q = 0.33333, df = 2, p-value = 0.8465
From the output of the test we can observe the following:
- The test statistic is 0.333
- The corresponding p-value is 0.8465
Since this p-value is not less than .05, we fail to reject the null hypothesis.
This means we do not have sufficient evidence to say that the studying technique used by students leads to different proportions of passing rates.