Image by Editor

In a recent article, we took a gentle tour of strategies to deal with mixed data. In real-world problems, it is common to have datasets containing a mix of data types. This typically requires particular approaches beyond classical ones that, in most cases, suit homogeneous, numerical data almost exclusively.

Now, it’s time to dive deeper into one of these strategies, concretely for clustering mixed data objects into subgroups based on similarity: the K-prototypes clustering algorithm.

K-prototypes in a Nutshell

What makes K-prototypes different from other well-known approaches like K-means, and why does it work on mixed data?

K-prototypes works by using a hybrid cost function that combines squared Euclidean distances for numerical variables with simple matching or Hamming distances for categorical variables. It also uses both the mean on numerical variables and the mode on categorical ones to update cluster centers (prototypes).

Let’s see how to use it in practice!

K-prototypes in action

We will illustrate the practical use of K-prototypes through its scikit-learn implementation in Python.
You’ll need to !pip install kmodes in your notebook or IDE first. This library provides an implementation of the K-prototypes algorithm.

import pandas as pd
import seaborn as sns
from sklearn.preprocessing import StandardScaler
from kmodes.kprototypes import KPrototypes

# Loading the penguins dataset and dropping missing values
df = sns.load_dataset('penguins').dropna()

# Previewing the mixed data
display(df.head())

Data preview:

Before clustering, we remove the species attribute, which is too informative for our clustering purposes. We also scale numerical attributes to yield better results and identify the indices of categorical ones (island and sex).

# Drop 'species' to try cluster penguins naturally without knowing their species
X = df.drop(columns=['species'])

# Column types
cat_cols = ['island', 'sex']
num_cols = ['bill_length_mm', 'bill_depth_mm', 'flipper_length_mm', 'body_mass_g']

# Feature scaling: recommended before clustering
scaler = StandardScaler()
X_scaled = X.copy()
X_scaled[num_cols] = scaler.fit_transform(X[num_cols])

# K-Prototypes requires the indices of categorical columns
cat_indices = [X_scaled.columns.get_loc(col) for col in cat_cols]

print(f"Categorical indices: {cat_indices}")

Next, we apply K-prototypes, with k=3 clusters to be found (we inevitably know there might be three logical groups of similar data based on penguin species):

# Convert to numpy array: expected kmodes input
X_matrix = X_scaled.values

# Apply K-Prototypes
kproto = KPrototypes(n_clusters=3, init='Cao', n_init=5, random_state=42)
clusters = kproto.fit_predict(X_matrix, categorical=cat_indices)

# Append the predicted clusters to the original dataframe
df['cluster'] = clusters

To evaluate cluster quality, since we have ground-truth information about penguins’ species, we can use cross-tabulation to assess how well the algorithm separated the penguins correctly into species:

# Comparing real species with clusters found
crosstab_result = pd.crosstab(df['species'], df['cluster'])
print(crosstab_result)

Result:

cluster      0   1    2
species                
Adelie       0  30  116
Chinstrap    0  60    8
Gentoo     119   0    0

Somewhat similar to a confusion matrix for classification models, we can observe that Gentoo penguins were perfectly separated from the rest of the species. Meanwhile, Chinstrap and Adelie got recent results, but some specimens were mistakenly assigned to the wrong cluster.

Well done! You just learned how K-prototypes works and how to use it in Python.