Image by Editor | Midjourney

The article “How to Analyze Your Data Before Choosing a Statistical Distribution in Phitter” explored how to analyze data before selecting a statistical distribution, using standardized blood pressure measurements from the scikit-learn diabetes dataset. It emphasized the importance of understanding data characteristics, such as sample size, value range, and continuity, before fitting a continuous distribution. The Phitter library was then used to fit multiple distributions, ranking them based on Sum of Squared Errors (SSE) to identify the best match.

However, SSE is just one piece of the puzzle. In this article, we will take a closer look at Phitter’s three goodness-of-fit tests to ensure our distribution choice is statistically sound.

Understanding Goodness-of-Fit Tests

Before we examine our results, let’s briefly understand what each test measures:

1. Chi-Square Test: Compares observed frequencies with expected frequencies across different intervals. It’s particularly sensitive to differences in the shape of the distribution.
2. Kolmogorov-Smirnov (K-S) Test: Measures the maximum distance between the empirical distribution function of your data and the cumulative distribution function of the reference distribution.
3. Anderson-Darling Test: Similar to the K-S test but gives more weight to the tails of the distribution. This makes it particularly useful for detecting discrepancies in the extremes of your data.

Setting Up Our Analysis Environment

Before diving into the goodness-of-fit tests, let’s quickly set up our analysis environment and confirm our initial findings using Phitter’s best_distribution method:

import phitter
from sklearn.datasets import load_diabetes
import numpy as np
import matplotlib.pyplot as plt

# Load the diabetes dataset
diabetes = load_diabetes()
bp_data = diabetes.data[:, 3]  # Blood pressure column

# Initialize Phitter with blood pressure data
phi = phitter.PHITTER(
    data=bp_data,
    fit_type="continuous",  # Explicit choice based on EDA
    confidence_level=0.95
)

# Fit distributions
phi.fit()

# Obtain the best distribution
phi.best_distribution

The best_distribution method produces the below results. It confirms our initial finding that the triangular distribution provides the best fit based on SSE, with parameters defining the minimum (a), maximum (b), and mode (c) of the distribution.

{'id': 'triangular',
 'parameters': {'a': -0.11339880254408448,
  'b': 0.13304361674121307,
  'c': -0.019644814197128727}}

However, to ensure this choice is statistically robust, we need to examine how well it—and other potential distributions—perform across multiple goodness-of-fit tests.

Applying the Tests to Our Blood Pressure Data

Let’s examine how our top distributions perform across all three tests. A comprehensive overview of the test results will help us identify patterns across different testing methodologies. The summarize() method provides this view by generating a dataframe of best-fit distributions ranked by SSE.

# Summarize the results
phi.summarize().head(5)

With one line of code, we can generate the below output: 

Image By Author

Diving Deeper into Each Test

Chi-Square Test Results

# Obtain Chi-square test results
phi.df_sorted_distributions_sse.head(5)[['distribution', 'chi_square']]

Image By Author

• The triangular distribution has a test statistic of 16.36, below its critical value of 16.92
• The p-value of 0.060 is just above our significance level of 0.05. This p-value indicates that we cannot reject the null hypothesis that our data follows the triangular distribution, providing statistical support for our choice.
• Both the Maxwell and Johnson SB distributions also pass, though with different margins

Kolmogorov-Smirnov Test Results

# Obtain Kolmogorov-Smirnov test results
phi.df_sorted_distributions_sse.head(5)[['distribution', 'kolmogorov_smirnov']]

Image By Author

• All distributions show test statistics well below their critical values
• The triangular distribution’s test statistic (0.040) is comfortably below the critical value (0.064)
• Higher p-values (0.47 for triangular) indicate strong alignment with our data

Anderson-Darling Test Results

# Obtain Anderson-Darling test results
phi.df_sorted_distributions_sse.head(5)[['distribution', 'anderson_darling']]

Image By Author

• All test statistics are well below the critical value of 2.49
• The triangular distribution’s statistic of 1.43 suggests good fit even in the tails
• The Johnson SB distribution performs particularly well here (0.996), indicating excellent tail behavior

What These Results Tell Us

Our comprehensive testing reveals several key insights:

1. Multiple Valid Options: While the triangular distribution showed the lowest SSE, several distributions provide statistically valid fits according to all three tests.
2. Different Strengths: Each distribution shows particular strengths:
   • Triangular: Best overall SSE and solid performance across all tests
   • Maxwell: Particularly strong K-S test results
   • Johnson SB: Excellent tail behavior as shown by Anderson-Darling
3. Practical Implications: The passing of all three tests suggests that any of these distributions would be valid choices for modeling our blood pressure data.

Making the Final Decision

When multiple distributions pass all goodness-of-fit tests, consider:

1. Simplicity: The triangular distribution, with its straightforward parameters, might be easier to work with.
2. Application Needs: If tail behavior is crucial for your application, the Johnson SB distribution’s strong Anderson-Darling performance might make it the better choice.
3. Domain Knowledge: Consider which distribution best aligns with the theoretical understanding of blood pressure variation.

This thorough validation process demonstrates how combining multiple statistical tests with domain knowledge leads to more robust distribution selection. While mathematical fit is essential, the final choice should align with both statistical evidence and the practical requirements of your analysis.