How to Calculate Correlation Between Two Columns in Pandas

You can use the following syntax to calculate the correlation between two columns in a pandas DataFrame:

df['column1'].corr(df['column2'])

The following examples show how to use this syntax in practice.

Example 1: Calculate Correlation Between Two Columns

The following code shows how to calculate the correlation between columns in a pandas DataFrame:

import pandas as pd

#create DataFrame
df = pd.DataFrame({'points': [25, 12, 15, 14, 19, 23, 25, 29],
                   'assists': [5, 7, 7, 9, 12, 9, 9, 4],
                   'rebounds': [11, 8, 10, 6, 6, 5, 9, 12]})

#view first five rows of DataFrame
df.head()

        points	assists  rebounds
0	25	5	 11
1	12	7	 8
2	15	7	 10
3	14	9	 6
4	19	12	 6

#calculate correlation between points and assists
df['points'].corr(df['assists'])

-0.359384

The correlation coefficient is -0.359. Since this correlation is negative, it tells us that points and assists are negatively correlated.

In other words, as values in the points column increase, the values in the assists column tend to decrease.

Example 2: Calculate Significance of Correlation

To determine whether or not a correlation coefficient is statistically significant, you can use the pearsonr(x, y) function from the SciPy library.

The following code shows how to use this function in practice:

import pandas as pd
from scipy.stats import pearsonr

#create DataFrame
df = pd.DataFrame({'points': [25, 12, 15, 14, 19, 23, 25, 29],
                   'assists': [5, 7, 7, 9, 12, 9, 9, 4],
                   'rebounds': [11, 8, 10, 6, 6, 5, 9, 12]})

#calculate p-value of correlation coefficient between points and assists
pearsonr(df['points'], df['assists'])

(-0.359384, 0.38192)

The first value in the output displays the correlation coefficient (-0.359384) and the second value displays the p-value (0.38192) associated with this correlation coefficient.

Since the p-value is not less than α = 0.05, we would conclude that the correlation between points and assists is not statistically significant.