Image by Editor
In a recent article, we took a gentle tour of strategies for handling mixed data. In real-world problems such as clustering similar data, it is common to have datasets that contain a mix of data types. This typically requires particular approaches beyond classical ones that, in most cases, suit homogeneous, numerical data almost exclusively.
While specialized clustering algorithms for mixed data, such as k-prototypes, can deal with various data types natively, most of the widely used ones — k-means, hierarchical clustering, and DBSCAN, to name a few — are strictly limited to numerical features and reliant on mathematical distance metrics.
This article shows how to adequately encode and prepare a mixed dataset and make it numerically uniform and compatible with any popular clustering technique.
Step-by-Step Process
Let’s assume we want every feature in our mixed dataset to be treated equally in terms of measuring distances. Importantly, if we consider the Euclidean distance, which assumes equal contribution by all features to the distance computation:
$$d(p, q) = \sqrt{\sum_{i=1}^{n} (q_i – p_i)^2}$$
We need not only to convert non-numerical variables like categorical ones into a numerical format, but also to scale them properly, so that they have a similar magnitude, thereby keeping the overall distance calculation balanced.
To illustrate the approach, we’ll load the Palmer Archipelago Penguins dataset, which combines numerical and categorical features.
import pandas as pd
import seaborn as sns
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
# Loading the dataset and dropping missing values
penguins = sns.load_dataset('penguins').dropna()
print(penguins.head())
Separating features by type will prevent accidentally scaling text-based, categorical features: make sure you do this before conducting transformations:
categorical_cols = ['species', 'island', 'sex'] numerical_cols = ['bill_length_mm', 'bill_depth_mm', 'flipper_length_mm', 'body_mass_g']
Aided by a ColumnTransformer object, we will build a unified data transformation pipeline. Here we specify “what to do with which columns”, depending on their type. Standardization of numerical features will yield a mean of 0 and a standard deviation of 1. Meanwhile, one-hot encoding creates a set of binary, 0/1 features, one per category:
preprocessor = ColumnTransformer(
transformers=[
('num', StandardScaler(), numerical_cols),
('cat', OneHotEncoder(), categorical_cols)
])
Time to feed our raw dataset to the preprocessor we just created.
encoded_data = preprocessor.fit_transform(penguins) # Extracting new column names resulting from the OneHotEncoder cat_feature_names = preprocessor.named_transformers_['cat'].get_feature_names_out(categorical_cols) all_feature_names = numerical_cols + list(cat_feature_names) # Rebuilding a Pandas DataFrame with uniform data encoded_df = pd.DataFrame(encoded_data, columns=all_feature_names) print(encoded_df.head())
Output (single DataFrame shown along multiple lines for the sake of clarity):
bill_length_mm bill_depth_mm flipper_length_mm body_mass_g \ 0 -0.896042 0.780732 -1.426752 -0.568475 1 -0.822788 0.119584 -1.069474 -0.506286 2 -0.676280 0.424729 -0.426373 -1.190361 3 -1.335566 1.085877 -0.569284 -0.941606 4 -0.859415 1.747026 -0.783651 -0.692852 species_Adelie species_Chinstrap species_Gentoo island_Biscoe \ 0 1.0 0.0 0.0 0.0 1 1.0 0.0 0.0 0.0 2 1.0 0.0 0.0 0.0 3 1.0 0.0 0.0 0.0 4 1.0 0.0 0.0 0.0 island_Dream island_Torgersen sex_Female sex_Male 0 0.0 1.0 0.0 1.0 1 0.0 1.0 1.0 0.0 2 0.0 1.0 1.0 0.0 3 0.0 1.0 1.0 0.0 4 0.0 1.0 0.0 1.0
Now you have a perfectly uniform dataset, ready to feed to algorithms like K-means! Let’s give it a quick try:
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
import seaborn as sns
# K-Means with k=3
kmeans = KMeans(n_clusters=3, random_state=42, n_init='auto')
encoded_df['cluster'] = kmeans.fit_predict(encoded_df.drop('cluster', axis=1, errors='ignore'))
And to finalize, this is a visualization of clusters using two selected features among the existing ones:
plt.figure(figsize=(10, 7))
sns.scatterplot(x='bill_length_mm', y='flipper_length_mm', hue='cluster', data=encoded_df, palette='viridis', legend='full')
plt.title('K-Means Clusters (k=3) visualized with Scaled Bill Length and Flipper Length')
plt.xlabel('Scaled Bill Length (mm)')
plt.ylabel('Scaled Flipper Length (mm)')
plt.grid(True)
plt.show()
If you are interested in learning or freshening up on analyzing clustering results, check out this article. Good job!
