When working with large datasets, understanding your data’s distribution is essential. Histograms are one of the most powerful and intuitive visualization tools for this purpose, designed specifically for numerical data measured on an interval scale.
However, creating an effective histogram requires careful consideration of the bin size. The number of bars in your histogram, determined by the bin size, can significantly impact the insights you gain from your data. Too few bins might hide important details, while too many can create a cluttered and confusing picture. The key is to remember that there is no universally “perfect” bin size – the optimal choice depends on your specific dataset, its characteristics, and the story you want to tell with your visualization.
In this tutorial, we’ll explore the art and science of histogram binning. We’ll start by understanding histograms and their use cases, then dive into a practical example using a synthetic dataset. We’ll compare the default binning behaviors of popular Python libraries, investigate Seaborn’s “auto” binning method, and explore additional binning methods like the square root rule, Rice rule, Scott’s rule, and Doane’s rule. By the end, you’ll have a solid understanding of the different binning methods available and when to use them, along with practical tips for experimenting with bin sizes to create histograms that effectively communicate your data’s story.
What is a Histogram?
A histogram is a graphical representation that organizes data into continuous, adjacent bins and displays the frequency of observations within each bin. Two key aspects make histograms unique:
- Data Organization: The x-axis shows the range of values divided into intervals (bins), while the y-axis represents the frequency density.
- Area Representation: The area of each bar represents the frequency of data points falling within that bin, making the total area proportional to the total frequency.
Understanding Our Example Data
Before diving into binning methods, let’s understand the type of data we’ll be working with. For this guide, we’ll use a synthetic dataset that represents a common real-world scenario: a bimodal distribution (data with two peaks).
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
# Create our sample data
np.random.seed(42)
data = np.concatenate([
np.random.normal(0, 1, 1000), # Main group: 1000 points centered at 0
np.random.normal(4, 1.5, 500) # Secondary group: 500 points centered at 4
])
Default Behaviors in Popular Libraries
When you first create a histogram, you’ll likely use either Matplotlib or Seaborn. Let’s understand their default behaviors.
Matplotlib’s Default: 10 Bins
Matplotlib takes a simple approach by default: it divides your data into 10 equal-width bins. This is straightforward but can be too simplistic for complex distributions.
Seaborn’s Default: auto
Seaborn uses NumPy’s auto bin selection method, which selects the maximum bin count between sturges and fd (Freedman-Diaconis) rules. This often provides a better starting point for exploration by ensuring sufficient detail in the visualization.
Let’s visualize how these default behaviors compare:
# Create a figure with two subplots side by side
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 5))
# Matplotlib default
ax1.hist(data) # No bins specified - will use default of 10
ax1.set_title("Matplotlib Default\n(10 bins)", fontsize=20)
ax1.set_xlabel("Values", fontsize=18)
ax1.set_ylabel("Frequency", fontsize=18)
# Seaborn default
sns.histplot(data=data, ax=ax2) # Using seaborn's histplot
ax2.set_title("Seaborn Default\n(auto)", fontsize=20)
ax2.set_xlabel("Values", fontsize=18)
ax2.set_ylabel("Frequency", fontsize=18)
plt.tight_layout()
Using our sample data, the default methods for Matplotlib and Seaborn will result in the following histograms:
Understanding Seaborn’s Auto Bin Selection
Let’s dive deeper into how Seaborn’s auto setting works. According to NumPy’s documentation, the auto method chooses the (minimum bin width or) maximum bin count between sturges and fd (Freedman-Diaconis) rules. We can examine this directly:
# Calculate number of bins using each method
n_bins_sturges = np.ceil(np.log2(len(data)) + 1) # Sturges' formula
q75, q25 = np.percentile(data, [75, 25])
iqr = q75 - q25
h = 2 * iqr / (len(data) ** (1/3)) # FD rule bin width
range_fd = np.max(data) - np.min(data)
n_bins_fd = np.ceil(range_fd / h)
# Create figure with three subplots
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(20, 5))
# Sturges' rule
sns.histplot(data=data, bins=int(n_bins_sturges), ax=ax1)
ax1.set_title(f"Sturges' Rule\n({int(n_bins_sturges)} bins)", fontsize=20)
# Freedman-Diaconis rule
sns.histplot(data=data, bins=int(n_bins_fd), ax=ax2)
ax2.set_title(f"Freedman-Diaconis Rule\n({int(n_bins_fd)} bins)", fontsize=20)
# Seaborn auto (maximum of the two)
sns.histplot(data=data, bins='auto', ax=ax3)
ax3.set_title(f"Seaborn Auto\n(max({int(n_bins_sturges)}, {int(n_bins_fd)}) bins)", fontsize=20)
plt.tight_layout()
print(f"Number of bins using Sturges' rule: {int(n_bins_sturges)}")
print(f"Number of bins using FD rule: {int(n_bins_fd)}")
print(f"Seaborn auto uses: {max(int(n_bins_sturges), int(n_bins_fd))} bins")
This comparison reveals why Seaborn’s auto setting works so well:
- Sturges’ Rule: Calculates bins based on sample size using log2(n) + 1
- Simple and works well for normally distributed data
- Can underestimate needed bins for complex distributions
- Best for: Small to medium normally distributed datasets
- Freedman-Diaconis Rule: Uses data spread (IQR) and sample size
- More sophisticated, adapting to data variability
- Generally more bins for complex distributions
- More robust to outliers due to using IQR
- Best for: Data with outliers and complex distributions
- Seaborn’s Auto: Takes the maximum bin count
- Provides finer granularity by choosing the larger bin count
- Ensures sufficient detail in the visualization
- Particularly useful for complex or multimodal distributions
Exploring Additional Binning Methods
Seaborn’s auto binning method provides a solid foundation for creating informative histograms by combining the strengths of Sturges’ and Freedman-Diaconis rules. However, there are several other binning methods available, each with its own unique characteristics and use cases.
Before we dive into these additional methods, let’s introduce a more efficient way to calculate the number of bins using NumPy. This approach will allow us to quickly compare the bin counts across different methods and understand their impact on our histogram’s appearance.
# Compare all available binning methods
bin_methods = ['auto', 'sqrt', 'sturges', 'fd', 'rice', 'doane', 'scott']
for method in bin_methods:
bins = np.histogram_bin_edges(data, bins=method)
n_bins = len(bins) - 1
print(f"Number of bins using {method:8} rule: {n_bins}")
This code snippet iterates through a list of binning methods and uses NumPy’s histogram_bin_edges function to calculate the bin edges for each method. We then count the number of bins by subtracting 1 from the length of the bins array (since the number of edges is always one more than the number of bins). The above block of code will output the below for our sample dataset.
Number of bins using auto rule: 19 Number of bins using sturges rule: 12 Number of bins using fd rule: 19 Number of bins using doane rule: 16 Number of bins using scott rule: 16 Number of bins using rice rule: 23 Number of bins using sqrt rule: 39
Now that we’ve introduced the additional binning methods and demonstrated how to calculate the bin counts using NumPy, let’s visualize the histograms for each method using a 2×3 subplot. This will allow us to compare the methods side-by-side and evaluate their effectiveness on our example dataset.
# Create a 2x3 subplot for remaining methods
fig, axes = plt.subplots(2, 3, figsize=(20, 12))
axes = axes.ravel()
# Methods to compare (excluding default matplotlib)
methods = ['sqrt', 'rice', 'fd', 'doane', 'scott', 'sturges']
for ax, method in zip(axes, methods):
sns.histplot(data=data, bins=method, ax=ax)
n_bins = len(np.histogram_bin_edges(data, bins=method)) - 1
ax.set_title(f"{method.capitalize()} Rule\n({n_bins} bins)", fontsize=20)
ax.set_xlabel("Values", fontsize=18)
ax.set_ylabel("Frequency", fontsize=18)
plt.tight_layout()
plt.show()
The 2×3 subplot provides a comprehensive overview of the additional binning methods applied to our example dataset. By visualizing the histograms side-by-side, we can easily compare the characteristics of each method and evaluate their suitability for our bimodal distribution.
The square root rule (`sqrt`) produces the most detailed histogram with 39 bins, clearly separating the two peaks and revealing fine structure. However, it may create too many bins for smaller datasets. On the other hand, Sturges’ rule, the most conservative method, results in only 12 bins, which is too coarse for our bimodal data and obscures the underlying structure.
The remaining methods strike a balance between these extremes. The Rice rule (23 bins) maintains the overall shape without excessive noise, while the Freedman-Diaconis rule (19 bins) and Scott’s rule (16 bins) adapt well to data variability and provide clear visualizations of the two peaks. Doane’s rule (16 bins) shows a modest improvement over Sturges’ rule by capturing the data’s asymmetry slightly better.
My Personal Favorite: The Rice Rule
After exploring the various binning methods and their impact on our example dataset, I find myself gravitating towards the Rice rule. With 23 bins, it strikes a balance between providing sufficient detail to capture the bimodal structure and maintaining a clean, interpretable visualization without excessive noise.
However, it’s important to recognize that there is no universally “perfect” binning method. The choice ultimately depends on your specific dataset, its characteristics, and the insights you aim to communicate through your histogram.
Practical Recommendations and Conclusion
When creating histograms, it’s often best to start with the default binning options provided by your chosen library, such as Matplotlib’s 10 bins or Seaborn’s auto method. These defaults serve as a solid foundation for initial data exploration and can help you quickly identify the general shape and characteristics of your distribution.
However, as we’ve seen throughout this tutorial, the default settings may not always be optimal for your specific dataset. This is where experimentation comes into play. By exploring additional binning methods and comparing their outputs, you can gain a deeper understanding of how different bin sizes and strategies affect the insights you can glean from your data.
As you continue your data exploration journey, keep these binning strategies in mind. They are powerful tools in your arsenal, enabling you to create informative and compelling histograms that shine a light on the hidden patterns and stories within your data.
