Linear regression is one of the most fundamental statistical techniques. It is used to model the relationship between a continuous dependent variable and one or more independent variables. This guide will walk you through all the steps to perform a linear regression analysis in R, including data preparation, model construction, validation, and making predictions.  

Introduction to Linear Regression

A linear regression model defines the relationship between a continuous dependent variable and one or more independent variables, otherwise referred to as predictors. The goal is to fit a straight line that best describes the relationship between all the variables.  

The equation for a simple linear regression model is:  

Y = β₀ + β₁X + ε  

Where:  

• Y is the dependent variable (response).
• X is the independent variable (predictor).
• β₀ is the intercept.
• β₁ is the coefficient of the independent variable.
• ε is the error term.

The coefficient describes the linear relationship between the independent variable and the dependent variable. For example, if the coefficient is 13, this means that for every one unit increase in X, then Y will increase by 13. If you have more than one independent variable, you can add more X terms with additional coefficients associated with each of them.  

Preparation

The ability to run a linear regression is pre-built into R. However, there are some additional packages used for creating data visualizations and additional model tests that can be helpful to install and load into your environment.  

# Install necessary packages (only needed first time)
install.packages(“ggplot2”)	# for data visualization
install.packages(“lmtest”)	# for additional model tests

# Load packages into your R environment
library(ggplot2)
library(lmtest)

Another critical part of the setup process is loading in and cleaning the dataset you will use for the model. In this tutorial, the built-in mtcars dataset will be used.  

Data Exploration

Before fitting a model, it is essential to understand the dataset structure. Some helpful methods are printing the first few rows and getting summary statistics. These can be done using built-in functions in R.  

# Print the first few rows
head(mtcars)

# Summary statistics
summary(mtcars)

For a simple linear regression, we can look at the relationship between miles per gallon and car weight.  

Fitting a Linear Regression Model

To fit a simple linear regression model in R, you can use the lm() function. The dependent variable is listed first, followed by a ~ and the list of independent variables. The second part of the function specifies the dataset to be used to build the model.  

model <- lm(mpg ~ wt, data = mtcars)

summary(model)

The built-in summary() function returns a lot of important information about the model. The coefficients of the model equation are given. The coefficient for weight tells us the impact of the car’s weight on miles per gallon while the intercept is the estimated fuel efficiency when the car’s weight is zero.  

P-values of less than 0.05 typically mean that an independent variable is statistically significant in predicting the dependent variable. The R-squared value of the model is also provided. The closer this value is to 1, the better the overall model fit.  

Model Assumptions

In order for the results of a linear regression to be valid, there are 4 key assumptions that must be met. There is a built-in function that will automatically generate all the plots needed to visually assess these assumptions.  

# set up a 2x2 plot area
par(mfrow = c(2,2))

# generate diagnostic plots
plot(model)

1. Independence: Each observation must be independent of each other. There is no way to check this; it must be verified when the data is collected. For example, data from the same observation should not be duplicated and there should not be any expectation that the data from one observation has a direct impact on another observation.
2. Linearity: . The relationship between the independent and dependent variable must be linear. This is assessed using the residuals vs. fitted plot and there should be no clear pattern in the scatterplot.
3. Homoscedasticity: The variance of the residuals of the linear regression model should be constant. This is checked using the Scale-Location plot wherein the residuals should be equally spread out with no clear pattern.
4. Normality: The residuals of the linear regression model should be normally distributed. This is verified using the normal Q-Q plot and the data should fall approximately along the 45-degree reference line.

If one or more assumptions are violated, data transformation or more advanced techniques should be considered.  

Model Evaluation

Any model should also be evaluated before its results are used for predictions. There are three main ways to assess a linear regression model, all using the summary() function demonstrated above.  

1. R-Squared: This metric defines how much of the variance in the dependent variable is explained by the independent variables. The greater the R-Squared, the more predictive power a model has. The minimum value is 0 and the maximum value is 1.
2. Coefficient significance: Each coefficient in the model will have its own p-value. A significant p-value, typically 0.05 or smaller, indicates that the independent variable is a strong predictor of the dependent variable. If all the coefficients are non-significant, the overall model is probably not very good for predicting values.
3. Residual standard error (RSE): This value indicates the average amount that the data points deviate from the regression line. The less deviation there is, the more closely the model matches the data.

Predicting Values

Once a model has been fit and evaluated, it can be used to make predictions on new data. For example, we can use the model fitted above to predict what the MPG will be for a car with a weight of 3.0 thousand pounds.  

# create a dataset for the new value
new_data <- data.frame(wt = 3.0)

# create a prediction
mpg_prediction <- predict(model, newdata = new_data)

# show the new value
mpg_prediction

You can also get predicted values for a list of values or even the column of another dataset using the same technique.  

Visualizations

Finally, it can be helpful to visualize the relationship between your predictor and output variable alongside the fitted model regression line. This can be done using ggplot2, adding in the data points and the linear regression line of best fit.  

ggplot(mtcars, aes(x = wt, y = mpg)) 
+ geom_point() 
+ geom_smooth(method = "lm", col = "blue") 
+ labs(title = "Linear Regression of MPG on Weight", x = "Weight (1000 lbs)", y = "Miles per Gallon")

Enhancements and Next Steps

There are many ways to enhance the basic linear regression model developed in this example. For example, you can create a multiple linear regression model by adding in more independent variables. Doing so can improve the model’s explanatory power and increase accuracy in predictions.  

If you believe two variables are strongly interrelated, such as the number of cylinders changing the relationship between weight and MPG, you can add an interaction term. This will allow you to model how the relationship between two variables changes under different conditions of a third variable.  

Other, more advanced techniques include cross-validation to see how well your model predicts new datasets, stepwise regression to optimize the best subset of independent variables to include in a model, and regularization to address multicollinearity between two or more independent variables.  

Summary

This guide provided a high level overview for how to perform a linear regression in R. The major steps of this process are: data exploration, fitting a model, checking assumptions, evaluating the model, and making predictions. Linear regressions are a foundational technique in statistics and data analytics and are the gateway to more advanced predictive techniques, so understanding the basics is essential to developing strong analytic skills.