Image by Author | Canva

Let’s face it: staring at a raw dataset for the first time can feel overwhelming. You’ve got rows of numbers, cryptic column names, and a lingering question: “Where do I even start?” That’s where Exploratory Data Analysis (EDA) comes in. Think of EDA as your detective toolkit for uncovering hidden patterns, spotting errors, and asking better questions about your data.

In this article, I’ll walk you through a practical, step-by-step EDA process using Python. You’ll learn how to clean, visualize, and interpret data efficiently—no PhD in statistics is required. I’ll even share a real-world example to keep things relatable. Let’s dive in.

What Is Exploratory Data Analysis (EDA)?

EDA is the process of investigating a dataset to summarize its key characteristics, such as mean, median, and data types. It helps identify errors like missing values, outliers, and duplicates. Additionally, EDA uncovers relationships between variables and guides your next steps, such as feature engineering or model selection.

Think of it like getting to know a new friend: you ask questions, notice quirks, and learn what makes them tick.

Why EDA Matters (and Where Beginners Stumble)

Imagine building a house on a faulty foundation—without EDA, your data analysis or machine learning model risks the same fate. Here’s why EDA is non-negotiable:

It helps you avoid the “Garbage In, Garbage Out” (GIGO) problem, where flawed data leads to flawed insights. Performing EDA saves time later by fixing data issues early, preventing headaches during modeling. It also allows you to spot hidden opportunities, as trends or outliers might inspire new hypotheses.

However, beginners often stumble into common pitfalls. These include skipping data cleaning (yes, it’s tedious but critical), jumping straight to complex models without understanding the basics, and ignoring the context behind the data (e.g., Why are 30% of ages missing?).

Tools You’ll Need

Here’s your Python EDA starter pack:

Install them with one line:

pip install pandas matplotlib seaborn numpy missingno 

Step-by-Step EDA in Python (With Titanic Dataset Example)

Let’s analyze the Titanic dataset—a classic dataset containing passenger details like age, ticket class, and survival status.

Step 1: Load and Inspect the Data

import pandas as pd

Load the dataset:

df = pd.read_csv('titanic.csv')

Get the first 5 rows:

print(df.head())

Image by Author

See the basic stats of your dataset:

print(df.describe())

Image by Author

See the data types and missing values:

print(df.info()) 

Image by Author

From the output above:

1. Dataset Structure

Columns: 9 columns including – Numerical(Age, Fare, PassengerId), Categorical(Survived (binary), Pclass, Sex, Embarked), Text(Name, Cabin) and Total rows (12 entries)

2. Data Quality Flags

Invalid Values: Survived has a max value of 3 (should only be 0/1) (Critical issue!) while Pclass shows a max value of 4 (should only be 1/2/3)

Missing Data: Age (1 missing value (11/12 rows filled)), Cabin (7 missing values (5/12 rows filled), Embarked has no missing values (contrary to original claim)

3. Data Type Issues

Misclassified Columns: Survived and Pclass are stored as integers but represent categories.

Step 2: Clean the Data

Fix Missing Values:

Check missing values:

import missingno as msno  
msno.matrix(df)

Drop or fill in missing values. We will fill age gaps with median:

df['Age'] = df['Age'].fillna(df['Age'].median()) 

Remove Cabin (too many missing):

df.drop(['Cabin'], axis=1, inplace=True)

df['Age'].fillna(df['Age'].median(), inplace=True)   
df.drop(['Cabin'], axis=1, inplace=True)

Check and handle duplicates:

df.duplicated().sum() 
df.drop_duplicates(inplace=True)

Check missing values again:

print(df)

Image by Author

Step 3: Visualize Distributions and Relationships

Age distribution:

import seaborn as sns  
import matplotlib.pyplot as plt

plt.figure(figsize=(5, 3)) 
sns.histplot(df['Age'], bins=20, kde=True)  
plt.title('Age Distribution of Passengers')  
plt.xlabel('Age') 
plt.ylabel('Frequency') 
plt.grid(True)
plt.show()

Image by Author

Insight

This visualization helps understand the distribution of passenger ages in the dataset.

• Histogram: Shows the frequency of passengers in different age groups.
• KDE Curve: This tool provides a smoothed representation of the age distribution, making it easier to identify trends or patterns.

Survival by gender:

sns.countplot(x='Survived', hue='Sex', data=df)  
plt.title('Survival Count by Gender')  
plt.xlabel('Survived')
plt.ylabel('Count')
plt.show()

Insight

Females had a higher survival rate.

Image by Author

Correlation heatmap:
Select only numeric columns for correlation

numeric_df = df.select_dtypes(include=['number'])

Now generate the heatmap:

sns.heatmap(numeric_df.corr(), annot=True, cmap='coolwarm')
plt.title('Correlation Heatmap')
plt.show()

Compute the correlation matrix:

correlation_matrix = numeric_df.corr()

Find weak correlations (absolute value < 0.2):

weak_correlations = correlation_matrix[(correlation_matrix > -0.2) & (correlation_matrix < 0.2)]

print(weak_correlations)

Image by Author

Insight

From the heatmap, we can identify the weakly correlated pairs as those with correlation values close to 0 (between -0.2 and 0.2 in absolute value). Looking at the heatmap:

• PassengerId and Age have a correlation of 0.047, which is weak
• Survived and PassengerId has a correlation of 0.057, which is also weak

Common EDA Mistakes (and How to Avoid Them)

• Overlooking Outliers: Use boxplots to spot them

sns.boxplot(x=df['Fare']) 

Output:
Overlooked Outliers: Axes(0.125,0.11;0.62x0.77)