Image by Editor | FLUX.1 Schnell

Cluster analysis is a type of unsupervised learning task commonly needed in many sectors: from finance and marketing to segment shop or bank customers based on their behavior, to healthcare for identifying patient subgroups, to biology to cluster gene expressions, to name a few. These example problems share one aspect in common: the aim is to group data points based on similarities, without having any predefined labels associated with data observations.

From classical methods inherited from statistics like k-means to more advanced solutions involving hierarchical clustering or complex density-based algorithms, there is a wealth of approaches with varying degrees of complexity, interpretability, and data requirements. Therefore, a question that often comes up is: which type of clustering strategy should I select based on my problem, data, and needs?

This article provides a decision tree-based taxonomy of cluster analysis methods to guide you in identifying the most suitable approach to apply among the diverse landscape of options available, depending on the problem you are addressing and the characteristics of your data.

Note: while in previous articles of the author, the difference between cluster analysis and clustering techniques has been underlined, since the key choice to make for a suitable clustering analysis strategy pertains to its underlying clustering technique, this guide uses both terms indistinctly, placing the focus of the decision-tree guide on clustering techniques.

A Decision Tree Approach to Identify the Best Cluster Analysis Strategy

The following diagram illustrates our proposed decision tree approach for navigating you towards choosing the right cluster analysis approach.

Let’s further describe this visual guide along the following key points and elements to consider:

A decision tree approach to select the most suitable cluster analysis strategy (click to enlarge)

The first criterion to look at is typically the type of data we have, namely whether features are numerical or categorical. If not all features are numerical, with some or all of them being categorical, choosing a clustering technique largely depends on how many groups we are expecting to find in the data. If few groups, typically two to five, are expected to exist in the data, the K-mode (a categorical counterpart to the popular K-means algorithm) and hierarchical clustering algorithms are a good approach. Meanwhile, if we expect finding more than five groups in the data, techniques like DBSCAN or Gaussian Mixture Models (GMM) become more appropriate.

Now let’s look at the most frequent cluster scenario, in which all data variables are numerical. In this situation, the next thing to look at is the number of existing data points or observations. For smaller datasets with less than 1000 points, K-means or K-medoids are good choices, combined with the elbow method to determine the optimal number of clusters. For larger datasets comprising over 1000 instances, the choice will depend on whether the data is linearly separable. For linearly separable data, we can again rely on k-means algorithms alongside a strategy to find the optimal number of clusters to find. For non-linearly separable data, the type of clustering technique to choose will depend on another factor: having an a priori clue about the number of clusters we are looking for.

If we have a solid idea of the number of clusters to find, a variety of clustering methods can be tried: from K-means or K-medoids, to GMMs to spectral clustering, the latter of which makes use of similarity matrices to find clusters.

Otherwise, in the absence of a hint about the number of clusters to find, better options would be the DBSCAN algorithm, agglomerative hierarchical clustering, or the OPTICS algorithm, the latter of which is particularly useful in the presence of spatial data.

The above guide is useful in most situations. Additional considerations can be made in more specific situations. For instance, when the data contains noise or outliers, it is in most cases better to directly resort to algorithms like DBSCAN or OPTICS. In addition, if the interpretability of clusters is more important than achieving high-quality clusters, the K-means, K-medoids, and hierarchical clustering algorithms attain more importance.

Wrapping Up

This article provided a structured taxonomy, given by a decision tree, to help you identify the right cluster analysis method depending on the properties and size of your data, and the complexity and nature of the problem to address. Selecting the right clustering and cluster analysis approach is a key step to developing an effective and successful data segmentation or labeling solutions that yield high impact and provide real value in the scenario targeted.