Image by Editor | Midjourney

Healthcare fraud detection presents unique data challenges. When working with claims data, we often need to bridge the gap between individual transactions and provider-level analysis. In this article, we’ll walk through the process of aggregating claims data to create meaningful provider features, visualize patterns using Yellowbrick’s Parallel Coordinates, and explore other visualization tools available for feature analysis. We’ll demonstrate how combining proper data integration with visual analytics can reveal patterns that distinguish fraudulent from legitimate healthcare providers.

For an introduction and access to the Healthcare Provider Fraud Detection dataset and our analysis on class imbalance visualization using Yellowbrick, please read our article on How to Visualize Class Imbalance with Yellowbrick. 

The Data Integration Challenge

The dataset presents an interesting structure:

• Train dataset (5,410 rows): Contains provider IDs and fraud labels
• Train Inpatient dataset (40,474 rows): Hospital admission claims
• Train Outpatient dataset (517,737 rows): Outpatient visit claims

The target variable (fraud/not fraud) exists at the provider level, but most of our valuable information is buried in individual claims spread across two separate tables. This mismatch between our analysis unit (providers) and data granularity (individual claims) requires careful aggregation.

Let’s walk through the process of creating our first provider-level features by aggregating claims amounts:

import pandas as pd

# Load our datasets
train = pd.read_csv('Train-1542865627584.csv')  # Provider fraud labels
inpat = pd.read_csv('Train_Inpatientdata-1542865627584.csv')  # Hospital stays
outpat = pd.read_csv('Train_Outpatientdata-1542865627584.csv')  # Outpatient visits

# Get provider-level IP claims totals
ip_provider_amounts = inpat.groupby('Provider')['InscClaimAmtReimbursed'].sum().reset_index()
ip_provider_amounts = ip_provider_amounts.rename(columns={'InscClaimAmtReimbursed': 'IP_Claims_Total'})

# Get provider-level OP claims totals 
op_provider_amounts = outpat.groupby('Provider')['InscClaimAmtReimbursed'].sum().reset_index()
op_provider_amounts = op_provider_amounts.rename(columns={'InscClaimAmtReimbursed': 'OP_Claims_Total'})

This initial step aggregates the total claim amounts for each provider, separately for inpatient and outpatient claims. But we need to handle an important detail – not all providers have both types of claims. Some may only see outpatients, while others may only handle inpatient cases.

Here’s how we handle this:

# Merge IP and OP provider totals
provider_claims = pd.merge(
    ip_provider_amounts,  # Left: Provider, IP_Claims_Total
    op_provider_amounts,  # Right: Provider, OP_Claims_Total
    on='Provider',
    how='outer'
).fillna(0)  # Fill NaN with 0 for providers with only one type of claims

# Then merge with fraud labels
final_df = pd.merge(
    provider_claims,
    train[['Provider', 'PotentialFraud']],
    on='Provider',
    how='outer'
)

print(final_df)

Let’s examine our result:

     Provider  IP_Claims_Total  OP_Claims_Total PotentialFraud
0    PRV51001          97000.0           7640.0            No
1    PRV51003         573000.0          32670.0           Yes
2    PRV51007          19000.0          14710.0            No
3    PRV51008          25000.0          10630.0            No
4    PRV51011           5000.0          11630.0            No

This integrated dataset tells us something interesting – we can now see the total financial scale of each provider’s claims, split between inpatient and outpatient services. Notice PRV51003, a provider flagged for potential fraud, has unusually high claim amounts in both categories.

The outer merge with zero-filling serves two purposes:

1. It preserves all providers, even those with only one type of claims.
2. It explicitly shows zero claim amounts rather than missing values, making the data more suitable for analysis.

Feature Patterns Through Parallel Coordinates

The Parallel Coordinates plot is a powerful tool for visualizing how features relate to our target variable in multivariate data. Each vertical line represents a feature, while the horizontal lines connecting them show how feature values flow from one attribute to another for each data point. In our case, we’ll use it to explore how inpatient and outpatient claims totals vary between fraudulent and non-fraudulent providers.

Here’s how we create the visualization using Yellowbrick:

# Parallel Coordinates: Visualize how individual feature values vary across classes
from yellowbrick.features import ParallelCoordinates
from sklearn.preprocessing import StandardScaler

# Prepare features and target
X = final_df[['IP_Claims_Total', 'OP_Claims_Total']]
y = (final_df['PotentialFraud'])

# Scale the features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
X_scaled = pd.DataFrame(X_scaled, columns=X.columns)

visualizer_parallel = ParallelCoordinates()
visualizer_parallel.fit_transform(X_scaled, y)
visualizer_parallel.show()

Image By Author

Looking at the resulting visualization, we can make several key observations:

1. Pattern Separation: The green lines (fraudulent providers) show more extreme patterns, particularly in the higher ranges of both inpatient and outpatient claims. This suggests that fraudulent providers tend to submit unusually high claims in both categories.

2. Line Slopes: Many of the green lines have steep slopes between IP_Claims_Total and OP_Claims_Total, indicating that fraudulent providers often have significantly different levels of activity between inpatient and outpatient services. This could suggest attempting to maximize reimbursements through various claim types.

3. Density Distribution: The blue lines (non-fraudulent providers) are densely concentrated in the lower ranges of both features, showing more consistent and moderate claiming patterns. This creates a baseline of “normal” behavior against which outliers become more apparent.

4. Feature Scale: Note that we standardized our features before visualization (using StandardScaler). This puts both inpatient and outpatient claims on the same scale, making patterns easier to identify.

This visualization reinforces what we noticed in our initial data exploration – fraudulent behavior often manifests in unusual claiming patterns across both service types. However, the parallel coordinates plot adds another dimension to our understanding by showing how these patterns flow between the two features for individual providers. 

Yellowbrick’s Feature Analysis Arsenal

While we focused on Parallel Coordinates for our healthcare fraud detection analysis, Yellowbrick offers a rich set of feature analysis tools. Each visualizer serves unique analytical purposes and can be integrated seamlessly with scikit-learn pipelines. Here’s an overview of key visualizers:

1. Rank Features (Rank1D and Rank2D): Evaluate feature importance through correlation and covariance metrics. Particularly useful when you need to identify the most influential variables in your dataset.
2. PCA Projection: Projects high-dimensional data onto a 2D space while preserving maximum variance. Helpful when dealing with datasets containing many features and you need to understand overall data structure.
3. Manifold Visualization: Offers sophisticated dimensionality reduction techniques that can preserve local relationships in your data. Adapts its visualization approach based on your target variable type.
4. RadViz Visualizer: Places features around a circular plot and uses spring forces to display instances – excellent for spotting class separation patterns.
5. Direct Data Visualization (Jointplots): Creates detailed views of feature-target relationships, combining scatter plots with distribution information. Particularly valuable when you need to understand how individual features relate to a continuous target variable.

Your choice of visualizer should align with your analytical goals. For instance:

• Use Rank Features when feature selection is a priority
• Apply PCA or Manifold when you need to reduce dimensionality while preserving data relationships
• Choose RadViz or Parallel Coordinates when investigating class separation patterns
• Opt for Jointplots when exploring continuous variable relationships

Each tool offers unique insights, and combining multiple visualizers often provides a more complete understanding of your data’s structure and relationships.

Conclusion

The combination of thoughtful data integration and visual analytics proves powerful in healthcare fraud detection. By aggregating claims data to the provider level and leveraging Yellowbrick’s visualization capabilities, we can identify patterns that might indicate fraudulent behavior. While we focused on Parallel Coordinates in this analysis, Yellowbrick’s suite of tools offers multiple perspectives for understanding feature relationships. The key lies in choosing the right visualization approach for your specific analytical needs – enabling you to transform complex data into meaningful insights.