How to Perform Linear Regression in R

Linear regression stands as a foundational statistical modeling technique serving as a cornerstone in predictive analytics and empirical research methodology. This mathematical framework enables researchers to quantify relationships between variables by establishing a linear relationship between predictors and response variables. This helps provide both explanatory and predictive capabilities that are essential in fields ranging from economics to social sciences

Through the implementation of linear regression in R, practitioners can leverage robust statistical computing capabilities to perform sophisticated analyses, enabling data-driven decision-making through the examination of variable relationships and the construction of predictive models that balance complexity with interpretability.

In this tutorial, let’s have a look at how to implement linear regression in R. First we will load the data and explore it, and then we will create a model to determine how accurately it predicts values.

Loading the Data

First, we need to load the Boston housing dataset into R.

# Install and load the necessary libraries
install.packages("MASS")  
library(MASS)             

# Load the dataset
data("Boston")

# View the first few rows of the dataset
head(Boston)

Data Exploration and Visualization

Before building the model, it’s important to explore the data and gain insights into the relationships between variables.

hist(Boston$medv, 
    col = "lightblue", 
    border = "black", 
    main = "Distribution of Median Value", 
    xlab = "Median Value of Homes (medv)", 
    breaks = 20)

# Scatterplot of 'rm' vs. 'medv'
plot(Boston$rm, Boston$medv, 
    main = "'rm' vs. 'medv'", 
    xlab = "Average Number of Rooms per Dwelling (rm)", 
    ylab = "Median Value of Homes (medv)", 
    col = "blue", pch = 16)

# Scatterplot of 'lstat' vs. 'medv'
plot(Boston$lstat, Boston$medv, 
    main = "'lstat' vs. 'medv'", 
    xlab = "Lower Status of the Population (lstat)", 
    ylab = "Median Value of Homes (medv)", 
    col = "red", pch = 16)

Building the Linear Regression Model

Next we will split the dataset into training and testing sets. After this, you can build the linear regression model using the training dataset. The lm() function in R is used to create the model. We’ll forecast the median price of homes that people own using other characteristics.

# Set seed for reproducibility
set.seed(123)

# Split the data into training (70%) and testing (30%) sets
sample_index <- sample(1:nrow(Boston), 0.7 * nrow(Boston))
train_data <- Boston[sample_index, ]
test_data <- Boston[-sample_index, ]

# Build the linear regression model
model <- lm(medv ~ ., data = train_data)

# View the model summary
summary(model)

The model summary provides key insights into the linear regression model. Here’s a breakdown of the main components:

Residuals: These are the differences between what we see and what the model predicts. They show how well the model works.
Coefficients: Each coefficient tells us how much the median home value (medv) changes when a predictor goes up by one unit.
Standard Error: This number shows how much the coefficient estimates can change. Smaller numbers mean the estimates are more reliable.
t-value: This value helps us check if a coefficient is different from zero. Higher values mean the predictor is more important.
Pr(>|t|): The p-value shows the chance that a coefficient is zero. Lower values (less than 0.05) mean the predictor is strong.
Significance Codes: Stars next to the p-values show how important they are. More stars mean more importance.
R-squared: This number tells us how much of the change in medv the model explains. Numbers close to 1 show a good fit.
Adjusted R-squared: his value changes the R-squared based on how many predictors are used. It helps us compare models better.
F-statistic: This tests if at least one predictor is related to medv. A high value and low p-value mean the model is useful.
Residual Standard Error: This measures how much the predictions can be off. Lower numbers mean the predictions are more accurate.

Visualizing the Training Set

After building the linear regression model, you can plot the actual vs. predicted values on the training set.

# Predict values on the training set
train_predictions <- predict(model, newdata = train_data)

# Plot actual vs. predicted values for the training set
plot(train_data$medv, train_predictions, 
    main = "Training Set Visualization",
    xlab = "Actual Values (medv)", 
    ylab = "Predicted Values (medv)", 
    col = "blue", pch = 16)
abline(a = 0, b = 1, col = "red", lwd = 3)

Making Predictions

After building the model, you can use it to predict the testing set. This step will help us see how well our model works with new data.

# Make predictions on the testing set
predictions <- predict(model, newdata = test_data)

# View the first few predictions
head(predictions)

  1     3     6     8     9    12
30.02799 29.65668 18.33281 25.04430 10.03691 20.83256

Visualizing the Test Set

Once you have made predictions on the test set, you can visualize the actual vs. predicted values for the test set.

# Plot actual vs. predicted values for the test set
plot(test_data$medv, predictions, 
    main = "Test Set Visualization",
    xlab = "Actual Values (medv)", 
    ylab = "Predicted Values (medv)", 
    col = "purple", pch = 16)
abline(a = 0, b = 1, col = "red", lwd = 3)

Conclusion

In this tutorial, we learn how to use R to create linear regression models. The model we created can demonstrate how different factors of a dataset relate to each other. The model, if given enough data to become robust, is then able to help predict new data. This method is useful for understanding both relationships and trends in data.