Image by Author
Linear regression draws a straight line through your data, and that works well when two variables rise and fall together at a constant rate. But a lot of relationships in the real world don’t behave that way. Plant growth accelerates early in the season then slows down. Drug concentration in the bloodstream rises quickly after dosing and then declines. The relationship between speed and fuel consumption curves upward as speed increases. In every one of these cases, a straight line would miss the pattern entirely.
Polynomial regression extends linear regression by adding curved terms to the model. Instead of fitting a line, you fit a curve. The underlying math stays familiar, the interpretation gets a little more involved, and the payoff is a model that can describe relationships a straight line can’t capture. Here’s what polynomial regression is, how it works, when to use it, and how to interpret what your model is telling you.
Key Definitions
Before getting into mechanics, it helps to lock down the vocabulary. A polynomial is a mathematical expression built from a variable raised to one or more powers. The expression x² + 3x + 5 is a polynomial. The highest power present is called the degree. A degree-1 polynomial is a straight line, a degree-2 polynomial is a parabola, and a degree-3 polynomial can form a curve that rises, dips, and rises again.
In polynomial regression, the predictor variable (your x) gets raised to increasing powers, and each power enters the model as a separate term. A degree-2 model looks like this:
\[
y = \beta_0 + \beta_1 x + \beta_2 x^2
\]
The β (beta) values are the coefficients. They tell the model how much weight to assign to each term. β₀ is the intercept, β₁ controls the linear component, and β₂ controls the curvature.
One thing worth clarifying right away: polynomial regression is still a form of linear regression. The word “linear” refers to the relationship between the coefficients and the outcome, not between x and y. The model stays linear in its parameters even when it curves through your data.
How Polynomial Regression Works
Step 1: Recognize a Curved Pattern
Everything starts with looking at your data. Plot the predictor against the outcome and check the shape. If the points follow a curved path rather than a straight one, a polynomial model is likely the better fit.
Residual plots from a linear model also reveal curvature. When you fit a straight line and then plot the residuals (the differences between predicted and actual values), a U-shape or arch in the residuals is a direct signal that a curved term belongs in the model.
Step 2: Choose a Degree
The degree you choose controls how many bends your curve can make.
A degree-2 (quadratic) model adds one bend. It fits U-shaped or hump-shaped relationships, think of the arc a ball follows when thrown, or the relationship between age and some health biomarker that peaks in midlife.
A degree-3 (cubic) model adds two bends. It can follow more complex patterns where the slope changes direction twice, like growth curves in biology or the trajectory of an economic cycle. Note that this is different from a true S-curve (a sigmoid), which doesn’t reverse direction at all, it just changes how sharply it’s curving. A cubic can approximate the shape of an S-curve over a limited range, but it isn’t the same thing mathematically.
Higher degrees are possible, but they bring their own risks, which we’ll get to in the section on overfitting below.
As a starting point, look at your scatterplot. If you see one clear curve with a single turning point, start with degree 2. If the pattern changes direction more than once, consider degree 3.
Step 3: Create the Polynomial Terms
To fit the model, you create new columns in your dataset. If x is your predictor, you calculate x² and add it as a new variable. For a cubic model, you also calculate x³. These derived columns let you run standard linear regression on what is now a set of transformed features.
The math works out cleanly: the same ordinary least squares estimation that fits a straight line also fits these higher-degree models, because you’re still minimizing the sum of squared differences between observed and predicted values.
One thing to watch for: x, x², and x³ tend to be highly correlated with each other, which can inflate the standard errors on your coefficients and make the model numerically unstable, especially at higher degrees. Centering x before raising it to a power, or using orthogonal polynomials, helps with this.
Step 4: Fit the Model
With the polynomial terms in place, you run regression the same way you would for any linear model. The software estimates a coefficient for each term, and the result is a set of numbers that, when plugged back into the polynomial equation, trace a curve through your data.
Step 5: Check the Fit
Fit quality matters as much as the coefficient values themselves. Here’s what to look at:
R-squared: The proportion of variance in the outcome explained by the model. Higher values indicate a better fit, though R-squared never decreases when you add terms, so use it carefully.
Adjusted R-squared: Penalizes unnecessary complexity. If adding a degree-3 term barely changes the adjusted R-squared, the extra complexity probably isn’t worth it.
Residual plots: After fitting the polynomial model, plot residuals again. If the curved pattern has disappeared and the residuals scatter randomly around zero, the model has captured the nonlinearity.
Practical Examples
Example 1: Employee Productivity and Experience
A company wants to understand how productivity changes with years of experience. When they plot the data, productivity rises steeply in the first few years, then levels off, and eventually dips slightly for very long-tenured employees, maybe due to complacency or skill obsolescence.
A straight line would show a weak positive trend and miss the arc entirely. A quadratic model, with its single curve, captures the rise and decline well. The negative coefficient on the x² term confirms the downward curvature: as experience increases, the rate of productivity growth slows and eventually reverses.
The model might produce something like:
\[
\text{Productivity} = 45 + 8.2(\text{Experience}) – 0.4(\text{Experience}^2)
\]
Here’s where it’s easy to misread the coefficients. β₁ = 8.2 is not a constant “each year adds 8.2 units” effect, once there’s an x² term in the model, the effect of experience changes at every point along the curve. The actual marginal effect of experience is given by the derivative, 8.2 − 0.8(Experience). Near the start of a career, an extra year of experience adds close to 8.2 units of productivity. That benefit then shrinks by 0.8 units for every additional year, hitting zero at around 10 years (8.2 / 0.8 = 10.25). Past that point, more experience predicts lower productivity. You get this by taking the derivative and setting it to zero, or you can just note that the curve turns downward somewhere around 10 years of experience.
Example 2: Temperature and Ice Cream Sales
A retailer tracks daily ice cream sales against temperature. Sales increase as temperature rises from cool to warm, peak around the mid-80s (Fahrenheit), and then decline slightly on very hot days as people stay indoors.
Again, a straight line can’t capture a relationship with a peak. A quadratic model fits the arch, and the coefficients tell you both where sales are rising fastest and where the curve turns over.
Example 3: Drug Dosage and Response
A pharmacologist studies how patient response changes with drug dosage. At low doses, response barely changes. At moderate doses, response climbs sharply. At high doses, the curve flattens out as receptors saturate.
This is a classic saturating dose-response curve, and it’s worth being precise about what’s modeling it. A true saturation curve like this is usually fit with a sigmoid or logistic form (the Hill equation is the standard choice in pharmacology), not a polynomial, because polynomial terms grow without bound and can’t actually level off the way receptor saturation does. A cubic can approximate the slow start and steep middle reasonably well over the observed dose range, but treat any flattening at the high end as a local approximation rather than a true plateau, and be cautious about extrapolating beyond the doses you’ve actually measured.
Common Misconceptions
Higher Degree Always Means Better Fit
This is the most frequent mistake. Adding more polynomial terms will always improve fit on the data you have, but the model increasingly memorizes noise rather than learning signal. A degree-10 polynomial might snake through every point in your training dataset and then produce wildly wrong predictions for new data. That’s overfitting.
The goal is finding the degree that captures the real pattern without chasing noise. In most applied work, degree 2 or degree 3 covers the vast majority of curved relationships.
Polynomial Regression Is Not Linear Regression
This one trips up students. Polynomial regression is a special case of multiple linear regression. The features are nonlinear transformations of a single variable, but the model itself is still linear in its parameters. That matters because all the standard diagnostics and assumptions of linear regression still apply.
A Low R-squared Means the Model Is Wrong
A polynomial model with a low R-squared might still be fitting the correct curve. There may simply be a lot of scatter around the curve that no model can eliminate. Check the residual plots, not just the R-squared.
Implementation Guide
When to Use Polynomial Regression
Polynomial regression fits best when:
- The scatterplot shows a clear curve
- Residuals from a linear model show a systematic pattern
- The relationship has a known or expected turning point
- You want an interpretable model rather than a black-box prediction
Software Considerations
Polynomial regression is available in pretty much every statistical software package. In Python, scikit-learn provides PolynomialFeatures to generate the transformed terms before passing them to a linear regression model. In R, the poly() function handles the transformation directly inside a formula, but it defaults to orthogonal polynomials, so the coefficients won’t match the raw x, x², x³ interpretation used in the examples above. If you want coefficients you can read directly as “units per year” the way this article does, use poly(x, degree, raw = TRUE). Stata, SAS, and SPSS all support polynomial terms through standard regression interfaces.
If you want to see hands-on code examples across these approaches, this guide to polynomial regression on Machine Learning Mastery walks through implementation with full Python examples.
Connection to Other Methods
Polynomial regression sits within a broader family of approaches for modeling nonlinear relationships. Spline regression breaks the curve into segments, fitting polynomials within each segment for more local flexibility. Generalized additive models take this further, allowing smooth functions of multiple predictors. Polynomial regression is the natural entry point into all of these methods.
Conclusion
Polynomial regression gives you a direct, interpretable way to model curved relationships. You start by identifying a nonlinear pattern in your data, choose a degree that matches the complexity of that pattern, create the corresponding transformed terms, fit the model, and then interpret the coefficients with an eye on both direction and curvature. The key discipline is resisting the pull of higher-degree models, the simplest curve that captures the real pattern is almost always the right one.
For further reading, see related Statology guides on linear regression assumptions, residual diagnostics, and model selection criteria.
