How to Perform Logistic Regression in R

Logistic regression represents a probabilistic classification algorithm that extends the principles of linear regression into the domain of binary outcome prediction through the application of the logistic function. As a cornerstone of statistical learning theory, this logistic regression employs maximum likelihood estimation to model the relationship between predictor variables and dichotomous outcomes, transforming linear predictions into probability estimates bounded between 0 and 1.

Through its implementation in R’s statistical computing environment, logistic regression provides practitioners with a powerful and interpretable framework for binary classification tasks. It allows us to combine mathematical rigor with practical utility in fields ranging from medical diagnosis to marketing analytics, while offering robust statistical inference capabilities through its well-established theoretical foundations.

In this article, we will learn how to perform logistic regression using R.

Loading Data

First we will start with loading our data. For this example, we will use the built-in mtcars dataset, which contains data about various car models. We will predict whether a car has an automatic or manual transmission based on several predictors.

# Load necessary library for data manipulation
library(dplyr)

# Load the dataset
data(mtcars)

# View the first few rows of the dataset
head(mtcars)

Converting the Target Variable

Logistic regression requires the target variable to be a factor. In the mtcars dataset, we will predict the transmission type (am), which we will convert to a factor.

# Convert the 'am' column to a factor
mtcars$am <- factor(mtcars$am, levels = c(0, 1))

Fitting the Logistic Regression Model

Use the glm() function to fit the logistic regression model. The syntax for glm() is similar to lm() for linear regression, but with the addition of the family argument set to binomial to specify logistic regression.

# Fit logistic regression model
model <- glm(am ~ mpg + hp + wt, data = mtcars, family = "binomial")

# Display model summary
summary(model)

Call: This shows the formula used to create the logistic regression model.
Deviance Residuals: These numbers tell us how far off the model’s predictions are from the actual data we observed. They show the smallest difference, the first quarter of differences, the middle value, the third quarter, and the largest difference.
Signifcance Codes: These symbols (like ***, **, *, and .) tell us how important each predictor is based on its p-value.
Null Deviance: This shows how well the model fits when only the intercept is included, without any predictors.
Residual Deviance: This indicates how well the model fits with the predictors included, compared to the null model.
AIC (Akaike Information Criterion): This number helps us understand how good the model is. Lower AIC values mean the model is better.
Number of Fisher Scoring Iterations: This tells us how many times the model needed to go through calculations to get a steady result.

Evaluating the Model

You can predict probabilities using the predict() function. Convert the probabilities to 0 or 1 by setting a threshold, typically 0.5.

predicted_probs <- predict(model, type = "response")
predicted_classes <- ifelse(predicted_probs > 0.5, 1, 0)

# Confusion matrix
table(Predicted = predicted_classes, Actual = mtcars$am)

             Actual
Predicted     0  1
          0  18  1
          1   1 12

Calculating Model Accuracy

To check the accuracy of your model, compare the predicted and actual values.

# Calculate accuracy
accuracy <- mean(predicted_classes == mtcars$am)
accuracy

[1] 0.9375

Making Predictions on New Data

You can use the fitted model to predict outcomes on new data. For demonstration, let’s use a subset of the original dataset to simulate new data:

# Example new data
new_data <- mtcars[1:5, ]

# Predict probabilities for new data
new_predictions <- predict(model, newdata = new_data, type = "response")
new_predictions

Mazda RX4         Mazda RX4 Wag      Datsun 710         Hornet 4 Drive     Hornet Sportabout
0.9176753         0.8671651          0.0726885         0.2026027         0.6372011

To convert these probabilities into binary predictions:

# Convert probabilities to predicted classes
new_predicted_classes <- ifelse(new_predictions > 0.5, 1, 0)
new_predicted_classes

Mazda RX4         Mazda RX4 Wag      Datsun 710         Hornet 4 Drive     Hornet Sportabout
1                  1                  0                  0                  1

ROC Curve

The ROC (receiver operating characteristic) curve is a crucial tool for assessing the performance of a logistic regression model. The roc() function computes the ROC curve based on actual class labels and predicted probabilities. The auc() function calculates the area under the curve (AUC) for the ROC curve.

# Load necessary library for ROC curve
library(pROC)

# Generate predicted probabilities from the model
predicted_probs <- predict(model, type = "response")

# Create the ROC curve
roc_curve <- roc(mtcars$am, predicted_probs)

# Plot the ROC curve
plot(roc_curve, main = "ROC Curve", col = "blue")

# Calculate and display AUC
auc_value <- auc(roc_curve)
legend("bottomright", legend = paste("AUC =", round(auc_value, 2)), bty = "n")

Conclusion

Logistic regression’s implementation in R provides practitioners with a computational framework that balances mathematical sophistication with practical utility. This enables both predictive modeling and statistical inference through logistic regression’s well-established theoretical foundations. Through careful model specification, validation through metrics like ROC curves and confusion matrices, and thoughtful interpretation of coefficients, logistic regression continues to serve as an essential tool in the modern data scientist’s analytical arsenal.