Image by Editor (Kanwal Mehreen) | Canva
Regression analysis is a statistical technique to model the relationship between independent and dependent variables. The main goal is to predict or explain the changes in the independent variables’ effect on the dependent variables.
When we already have our regression model, we want to assess how well our model fits the data. There are many methodologies to evaluate them, but most of them rely on the residual concept.
Residual can be defined as the differences between the actual data and the prediction. For example, the model provides output 10 while the actual data is 15. This means that the residual will be different between the actual and prediction, which is 5 (15 – 10).
Many of the metrics used to evaluate the model are based on the residual, but the residual plot is a unique tool for regression analysis as it offers visual representation.
So, what are residual plots? Let’s explore them together.
Residual Plots
As mentioned above, residual plots are a visual representation tool used to evaluate the regression analysis. It provides a way to reveal any patterns and anomalies that might be present in the model.
Residual plots are created by plotting the predicted values on the x-axis and the residuals on the y-axis. You can standardize the residual if you are comparing residual results from multiple models. Otherwise, you can leave it as it is.
In general, an excellent residual plot will look like the image below.
We can interpret that the residual positive values in the y-axis mean that the prediction was too low compared to the actual values. On the other hand, the negative values mean the prediction was too high. The residual value precisely at 0 indicates the guess was exactly correct.
Ideally, we want our residual plots to be symmetrically distributed where they cluster toward the middle and close to 0 as much as possible. Moreover, the plot should not show any clear patterns.
For example, a residual plot that shows that our model has problems is shown in the image below.
The residual shows that our model cannot capture the non-linear pattern as the residual plot shows a non-linear pattern. The prediction is terrible, and the model is unable to capture the true data relationship.
By understanding the patterns shown by the residual plots, we can act upon them to fix our model; we may need to transform our data, choose another model, clear the outlier, and many others. It depends on what the residual plots are showing.
In the end, the ideal residual plots are where no patterns are seen from the plots, and they tend to be clustered in the middle. If there is any violation of this, assess if you can accept the result or do something to fix it.
That’s all you need to know about residual plots!
