Image by Editor (Kanwal Mehreen) | Canva
Multicollinearity is a statistical phenomenon in which two or more independent variables in a regression model are highly correlated. This correlation undermines the statistical significance of an independent variable and can lead to misleading interpretations of the model’s coefficients.
In this article, we will explore how to detect multicollinearity using Python’s statsmodels library.
Why Multicollinearity Matters
Multicollinearity undermines the reliability and interpretability of regression analysis in several ways:
- Unstable Coefficient Estimates: In the presence of multicollinearity, small changes in the data can lead to large swings in the estimates of the regression coefficients.
- Inflated Standard Errors: High multicollinearity inflates the standard errors of the coefficients, which reduces the statistical significance of the independent variables.
- Misleading Model Interpretation: When independent variables are strongly related to each other, it becomes hard to tell which one is affecting the outcome.
Prerequisites
To begin, install the necessary Python packages:
pip install statsmodels pandas numpy
Prepare Your Data
We’ll use a sample dataset to demonstrate. You can use your own dataset in place of this.
import pandas as pd
import numpy as np
import statsmodels.api as sm
from statsmodels.tools.tools import add_constant
# Simulate a regression dataset
np.random.seed(0)
X1 = np.random.rand(100)
X2 = 0.8 * X1 + 0.2 * np.random.rand(100) # Highly correlated with X1
X3 = np.random.rand(100)
y = 3*X1 + 2*X2 + 1.5*X3 + np.random.rand(100)
# Create DataFrame
df = pd.DataFrame({'X1': X1, 'X2': X2, 'X3': X3, 'y': y})
# Build the OLS regression
X = df[['X1', 'X2', 'X3']] # Features
X = add_constant(X) # Add intercept term
y = df['y'] # Target
# Fit the model
model = sm.OLS(y, X).fit()
# Print the summary
print(model.summary())
OLS Regression Results ============================================================================== Dep. Variable: y R-squared: 0.960 Model: OLS Adj. R-squared: 0.958 Method: Least Squares F-statistic: 759.0 Date: Tue, 22 Jul 2025 Prob (F-statistic): 1.05e-66 Time: 15:22:15 Log-Likelihood: -15.266 No. Observations: 100 AIC: 38.53 Df Residuals: 96 BIC: 48.95 Df Model: 3 Covariance Type: nonrobust ============================================================================== coef std err t P>|t| [0.025 0.975] ------------------------------------------------------------------------------ const 0.5683 0.099 5.734 0.000 0.372 0.765 X1 3.3272 0.425 7.830 0.000 2.484 4.171 X2 1.6061 0.525 3.058 0.003 0.564 2.649 X3 1.3828 0.096 14.374 0.000 1.192 1.574 ============================================================================== Omnibus: 25.984 Durbin-Watson: 1.916 Prob(Omnibus): 0.000 Jarque-Bera (JB): 5.643 Skew: 0.113 Prob(JB): 0.0595 Kurtosis: 1.859 Cond. No. 31.1 ==============================================================================
Detecting Multicollinearity
There are several ways to detect multicollinearity, including:
Variance Inflation Factor (VIF)
The Variance Inflation Factor measures how much the variance of an estimated regression coefficient increases due to multicollinearity. A VIF value greater than 5 or 10 is considered problematic.
Common thresholds:
- VIF < 5: generally safe
- VIF between 5–10: moderate risk
- VIF > 10: high risk
To calculate VIF for each variable in your model, you can use the variance_inflation_factor function from statsmodels.stats.outliers_influence:
from statsmodels.stats.outliers_influence import variance_inflation_factor # Compute VIF vif_data = pd.DataFrame() vif_data["feature"] = X.columns vif_data["VIF"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])] print(vif_data)
This will output a table showing each variable along with its corresponding VIF score.
feature VIF 0 const 11.867552 1 X1 18.134743 2 X2 18.188449 3 X3 1.017704
Condition Number
The Condition Number measures how sensitive a system of linear equations is to small changes in the input data. In regression analysis, it helps you assess the degree of multicollinearity among the independent variables.
Here’s how to interpret the condition number in practice:
- < 30: Little to no multicollinearity
- 30–100: Moderate multicollinearity
- > 100: Strong multicollinearity
print("Condition Number:", model.condition_number)
Condition Number: 31.135318191338968
Tips for Handling Multicollinearity
When multicollinearity is detected, there are several strategies you can use to mitigate its effects:
- Remove One of the Correlated Variables: If two variables are highly correlated, consider removing one of them, preferably the one that is less relevant to the outcome.
- Combine Variables: You can combine correlated variables into a single feature using techniques like Principal Component Analysis (PCA) or by creating an index.
- Use Regularization Techniques: Ridge and Lasso regression add penalties to the regression coefficients, which can help reduce the impact of multicollinearity.
Conclusion
Multicollinearity can distort regression results and lead to misleading interpretations. You can detect it using tools such as the Variance Inflation Factor (VIF) and the condition number. To fix multicollinearity, drop a correlated variable, combine them, or use regularization.
