Image by Editor | Midjourney
Introduction
The Analysis of Variance (ANOVA) is a statistical method capable of determining whether significant differences exist between the means of three or more groups. It therefore overcomes the limitations of other tests like t-tests that are limited to comparing two groups only.
A cornerstone element to ANOVA is the F-statistic: named after Ronald A. Fisher (1890-1962), the F-statistic or F-value is a measure that quantifies the ratio of variability between group means to the variability within groups. A higher F-value suggests that the inter-group variability is greater than the intra-group variability, indicating significant differences among group means.
This idea is represented in the two diagrams below:
F-value and its ties to variability within and between groups
The F-Value in Different ANOVA Tests
Let’s now examine the specific role, utilization, and interpretation of the F-value in three common types of ANOVA tests.
F-Value in One-Way ANOVA
One-way ANOVA tests evaluate the effect of a single categorical independent variable (groups) on a continuous dependent variable, for instance, testing whether average crop yield differs between three fertilizer types applied to green tea plantations. The F-value in a one-way ANOVA assesses whether there are statistically significant differences between the means of multiple unrelated groups, in our example, fertilizer types.
This Python example shows how to programmatically calculate the F-value in one-way ANOVA for comparing the mean performance of students under three independent teaching methods, along with the p-value for joint analysis and interpretation of results.
import numpy as np
from scipy import stats
# Test scores for three different teaching methods
method_A = [88, 92, 85, 91, 87]
method_B = [78, 85, 80, 83, 82]
method_C = [90, 93, 88, 92, 91]
# Perform one-way ANOVA
f_statistic, p_value = stats.f_oneway(method_A, method_B, method_C)
print(f'F-Statistic: {f_statistic}')
print(f'P-Value: {p_value}')
The F-value obtained is 17.937823834196895 and the p-value is 0.00024797020611308447. When both the p-value is lower than 0.05 and the F-value is large, there is a significant difference among the group means, so in this example the teaching method clearly influences student performance. In most scenarios, when the p-value is low the F-value is high, or vice versa, although there are rare exceptions to this norm, which may signify anomalies in the data.
F-Value in Two-Way ANOVA
In two-way ANOVA, the influence of two distinct categorical independent variables on one continuous dependent variable is tested. The interaction effect between the two independent categorical variables is also evaluated. In this slightly more complex scenario, the F-value can for instance help examine how both temperature (hot/cold) and humidity (high/low) affect battery life (measured as a continuous variable) in smartphones, and the interaction between temperature and humidity categories.
Consider this example where we analyze the effect of both teaching method and gender on student scores. In this case, each score is simultaneously falling under two groups: one associated with each categorical variable, for instance, the first score ’88’ is associated with a male (‘M’) student subject to teaching method ‘A’.
import pandas as pd
import statsmodels.api as sm
from statsmodels.formula.api import ols
# Sample data
data = {
'score': [88, 92, 85, 91, 87, 78, 85, 80, 83, 82, 90, 93, 88, 92, 91],
'method': ['A', 'A', 'A', 'A', 'A', 'B', 'B', 'B', 'B', 'B', 'C', 'C', 'C', 'C', 'C'],
'gender': ['M', 'F', 'M', 'F', 'M', 'F', 'M', 'F', 'M', 'F', 'M', 'F', 'M', 'F', 'M']
}
df = pd.DataFrame(data)
# Fit the two-way ANOVA model
model = ols('score ~ C(method) + C(gender) + C(method):C(gender)', data=df).fit()
anova_table = sm.stats.anova_lm(model, typ=2)
print(anova_table)
Output:
sum_sq df F PR(>F)
C(method) 235.909921 2.0 52.209573 0.000011
C(gender) 5.377778 1.0 2.380328 0.157265
C(method):C(gender) 51.488889 2.0 11.395082 0.003418
Residual 20.333333 9.0 NaN NaN
Two-way ANOVA calculates one F-value for each factor (method and gender), plus one F-value for the interaction between both: three in total. Once again, F-values are analyzed jointly with p-values (last column in the ANOVA table above). with a high F-value and p-value << 0.05, the teaching method again is proved to affect student scores, whereas the small F-value and p > 0.05 for gender indicates that this is not an influencing factor. Finally, the moderately high F-value and p < 0.05 for method-gender interaction, indicates that teaching method effectiveness subtly varies by gender.
F-Value in Repeated Measures ANOVA
This ANOVA test is utilized when the same subjects (e.g. students) are used for each treatment (e.g. experiencing every teaching method). The F-value in this context assesses whether there are statistically significant differences in means across the different conditions subjects are exposed.
Let’s delight ourselves once again with our starring statistic in action, through a final example:
import pandas as pd
import statsmodels.api as sm
from statsmodels.stats.anova import AnovaRM
# Sample data: Student scores under three teaching methods
data = {
'Student': [1, 2, 3, 4, 5],
'Method_A': [85, 88, 86, 90, 89],
'Method_B': [78, 82, 80, 79, 81],
'Method_C': [88, 90, 85, 87, 89]
}
# Convert data to long format for repeated measures ANOVA
df = pd.DataFrame(data)
df_long = pd.melt(df, id_vars=['Student'],
value_vars=['Method_A', 'Method_B', 'Method_C'],
var_name='Method',
value_name='Score')
# Perform repeated measures ANOVA
anova = AnovaRM(df_long, 'Score', 'Student', within=['Method'])
results = anova.fit()
print(results)
Output:
Anova
===================================
F Value Num DF Den DF Pr > F
-----------------------------------
Method 44.9394 2.0000 8.0000 0.0000
===================================
The first and last outputs in the printed table are the F-value and p-value respectively. Clearly, a very high F-value along with a p-value equal to zero indicates that the teaching method has a statistically significant impact on student performance.
Wrapping Up
In this article, we took a deep dive into a fascinating statistical measure used in ANOVA tests: the F-measure, which in broad terms measures the relationship between inter-group and intra-group variability for several groups of data. Through practical Python-based examples, we learned how to apply and interpret this value for several data analysis scenarios.
