A Beginner’s Guide to Generalized Linear Models (GLMs)

by Published on

A Beginner’s Guide to Generalized Linear Models (GLMs)
Image by Author | Midjourney & Canva

Let’s be honest. You’ve already scratched the surface of what generalized linear models are meant to address if you’ve ever constructed a linear regression model in Python and wondered, “This works great, but what if my data isn’t so… linear?“.

In essence, linear regression develops into a generalized linear model (GLM). Even if your data doesn’t match the assumptions of a traditional straight-line model, you can still use this adaptable framework to describe relationships between variables. Consider it a powerful extension that allows you greater flexibility while maintaining interpretability.

Now, why should you care about GLMs?

Because real-world data is messy. Sometimes your target variable is binary (yes/no), sometimes it’s a count (like the number of clicks), and sometimes it’s highly skewed (like insurance claims). A standard linear regression assumes the outcome is continuous and normally distributed, which just doesn’t hold up in many of these cases. That’s where GLMs come in.

These models give you the tools to work with all sorts of outcome variables, using the right mathematical assumptions behind the scenes. And the best part? They still give you those nice, clean coefficients you can interpret and explain to your team or client.

Here are just a few problems GLMs are made for:

  • Will a customer click on an ad? (binary → logistic regression)
  • How many purchases does a user make per month? (count → Poisson regression)
  • What is the expected size of an insurance claim? (positive, skewed → Gamma regression)

Linear Regression Recap (The Starting Point)

Before we get too deep into GLMs, let’s take a moment to appreciate where it all begins: linear regression. It’s probably the first statistical model you learned, and for good reason. It’s simple, intuitive, and often good enough for a quick analysis. But it does have limitations.

At its core, linear regression tries to draw a straight line through your data. It predicts a continuous outcome based on one or more input features. The relationship is expressed in this form:

$Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + … + \beta_nX_n + \epsilon$

Where:

  • Y is the target variable (e.g., price, temperature)
  • $\beta_0$ is the intercept
  • $\beta_1, \beta_2,…,\beta_n$ are the coefficients
  • $X_1, X_2,…,X_n$ are your input variables
  • $\epsilon$ is the error term

Linear regression makes a few assumptions that are often too strict for real-world data:

  • The outcome variable is continuous and normally distributed
  • The relationship between the variables is linear
  • The variance of the residuals is constant (homoscedasticity)
  • Observations are independent

If any of those assumptions are broken, and they often are, your model’s results could be misleading.
Let’s say you want to predict whether a customer will churn (yes or no). Linear regression can’t handle that properly because:

  • The output isn’t continuous
  • Predictions could fall outside the range of 0 and 1
  • It doesn’t model probability directly

That’s where Generalized Linear Models come in.

What Makes a Model “Generalized”?

Alright, now we’re getting to the heart of it.

A Generalized Linear Model (GLM) builds on top of linear regression but offers more flexibility. Think of it like this: instead of forcing your data to follow a straight line and assuming everything is normally distributed, GLMs let you customize how the outcome is modeled.

There are three key components that make a model “generalized”.

1. Random Component: The Distribution of the Response Variable

In regular linear regression, the response variable (Y) is assumed to be normally distributed. But GLMs say, “Let’s keep our options open.” Depending on your data, GLMs allow the response variable to come from other distributions in the exponential family including:

  • Binomial – for binary outcomes like yes/no or success/failure
  • Poisson – for count data, like the number of visits or clicks
  • Gamma – for continuous, positive, skewed data like time or cost

This means GLMs can adapt to different shapes of the data without breaking the math.

2. Systematic Component: The Linear Predictor

This part stays familiar. It’s the same linear combination of predictors that you use in standard regression:

$\eta = \beta_0 + \beta_1X_1 + \beta_2X_2 + … + \beta_nX_n$

This is where your independent variables come into play. No change here; it’s still the same design matrix you’re used to.

3. Link Function: Connecting Linear Predictors to the Mean of the Distribution

Here’s the twist: in GLMs, the predicted outcome doesn’t have to be modeled directly. Instead, we use a link function to connect the linear predictor ($\eta$) to the expected value of Y.

In formula terms: $g(\mu) = \eta$

Where:

  • $\mu$ is the expected value of Y (mean of the response)
  • g(·) is the link function (like log, logit, or identity)

Different GLMs use different link functions depending on the problem:

Different GLMs use different link functions depending on the problem

3 core components of a GLM3 core components of a GLM

Most Common Types of GLMs

Now that we understand what makes a model “generalized,” let’s break down some of the most useful types of GLMs. These are the workhorses of applied statistics and data science, especially when you’re dealing with binary outcomes, count data, or skewed continuous variables.

We’ll explain how each one works and what problems it solves, then walk through real Python code examples.

1. Logistic Regression

This model is used for making predictions about binary outcomes, such as whether a customer will sign up for a service. It estimates the probability that a given input belongs to a specific class. Under the hood, its linear component models the log-odds.

Formula: $log(\frac{p}{1 – p}) = \beta_0 + \beta_1X_1 + … + \beta_nX_n$

import pandas as pd
import statsmodels.api as sm
from sklearn.preprocessing import LabelEncoder

# Load dataset
df = pd.read_csv("bank-full.csv", sep=';')

# Encode target and one feature
df['subscribed'] = LabelEncoder().fit_transform(df['y'])
df['job_encoded'] = LabelEncoder().fit_transform(df['job'])

X = df[['job_encoded']]
y = df['subscribed']

# Add constant for intercept
X = sm.add_constant(X)

# Fit logistic regression model
logit_model = sm.Logit(y, X).fit()
print(logit_model.summary())

2. Poisson Regression

This is for modeling count data, such as the number of clicks, visitors, or transactions. It’s perfect for modeling events that occur independently over time or space, as it assumes the mean and variance of the response are equal.

Formula: $log(\lambda) = \beta_0 + \beta_1X_1 + … + \beta_nX_n$

import numpy as np
import statsmodels.api as sm

# Simulate count data
np.random.seed(0)
X_sim = np.random.poisson(3, 100)
y = 2 + 0.3 * X_sim + np.random.poisson(1, 100)

X = sm.add_constant(X_sim)
poisson_model = sm.GLM(y, X, family=sm.families.Poisson()).fit()
print(poisson_model.summary())

3. Gamma Regression

Predicting skewed, positive outcomes, like waiting time or insurance costs. Gamma models are great when you want to model things that can’t be zero or negative and are heavily skewed.

Formula (log link): $log(\mu) = \beta_0 + \beta_1X_1 + … + \beta_nX_n$

import numpy as np
import statsmodels.api as sm

# Simulated cost data
np.random.seed(42)
X_sim = np.random.normal(1000, 250, 200)
y = 100 + 0.05 * X_sim + np.random.gamma(2, 100, 200)

X = sm.add_constant(X_sim)
gamma_model = sm.GLM(y, X, family=sm.families.Gamma(link=sm.families.links.log())).fit()
print(gamma_model.summary())

Logistic Regression with GLM

Now that we’ve covered the theory, let’s see a GLM in action. We’ll work with the Bank Marketing dataset, which tracks whether a customer subscribed to a term deposit based on their profile and interaction history.
We’ll use statsmodels.api.GLM to fit a logistic regression, since the target variable y is binary (yes/no).

Step 1: Import the required libraries

import pandas as pd
import statsmodels.api as sm
import numpy as np

Step 2: Load the dataset

# Load the CSV file
df = pd.read_csv("bank-full.csv", sep=';')

# Quick check
df["y"].value_counts()

This shows us how many people said “yes” or “no” to the term deposit offer. We’ll convert this into a binary variable.

Step 3: Prepare the data for modeling

We’ll use just a few columns to keep things readable: age, balance, duration of the last call, and marital status.

# Create a copy of the data
df_model = df[["age", "balance", "duration", "marital", "y"]].copy()

# Convert the target variable to binary: 1 for 'yes', 0 for 'no'
df_model["subscribed"] = df_model["y"].map({"yes": 1, "no": 0})
df_model.drop("y", axis=1, inplace=True)

# One-hot encode the 'marital' column
df_model = pd.get_dummies(df_model, columns=["marital"], drop_first=True)

What we’ve done here:

  • Focused on relevant variables for clarity
  • Converted the outcome to binary
  • Used one-hot encoding so that the categorical variable marital becomes numeric (GLMs need numeric inputs)

Step 4: Fit the GLM (Logistic Regression)

# Define X and y
X = df_model.drop("subscribed", axis=1)
y = df_model["subscribed"]

# Add an intercept
X = sm.add_constant(X)

# Fit the logistic regression model using GLM
model = sm.GLM(y, X, family=sm.families.Binomial()).fit()

# Show summary
print(model.summary())

What to Look for in the Output

When you run this, the model summary will include:

  • Coefficients (coef): These tell you the log-odds change for a one-unit increase in that variable
  • p-values: Indicate how significant the variable is (values below 0.05 are usually considered meaningful)
  • Confidence Intervals: Help you understand the range in which the true coefficient likely lies

Final Thoughts

Generalized Linear Models might sound intimidating at first, but once you understand the logic behind them, they become an indispensable tool in your data science projects.

If you’ve followed this guide up to this point, you’ve seen how GLMs work conceptually, how they differ from ordinary linear regression, and how to apply them in Python using real-world data like the Bank Marketing dataset. You’ve also explored how different distributions and link functions let you model everything from binary outcomes to counts and costs.
But there’s still more to explore. You can go deeper by learning about:

Leave a Reply

Your email address will not be published. Required fields are marked *