How to Perform Basic Data Cleaning in R

Data cleaning is important for good analysis. It helps make sure your data is accurate and ready to use. In this article, we will cover basic steps for cleaning data in R.

Loading and Inspecting Data

First, load your data into R. Use the read.csv() function for CSV files.

# Load the data
data <- read.csv("Employee.csv")

# View the structure of the data
str(data)

Handling Missing Values

Missing values are a common issue in datasets. One option is to remove rows that have missing values. Another option is to replace missing values with the mean of the column. This keeps your data complete and usable for analysis.

# Check for missing values
missing_values <- sum(is.na(data))

# Print the total number of missing values
cat("Total number of missing values:", missing_values, "\n")

# Remove rows with missing values
data_clean <- na.omit(data)

Removing Duplicates

Duplicates can affect your analysis results. They can cause bias and distort outcomes. To fix this, remove duplicate rows from your dataset. Use the duplicated() function in R to find and eliminate them. This helps keep your data accurate.

# Count the total number of duplicate rows before removing them
total_duplicates <- sum(duplicated(data))

# Print the total number of duplicate rows
cat("Total number of duplicate rows:", total_duplicates, "\n")

# Remove duplicate rows
data_clean <- data[!duplicated(data), ]

Correcting Data Types

Ensure each column has the right data type. Convert dates to Date objects. Change numbers to numeric. Turn categories into factors. This makes your data easier to analyze.

# Convert the 'Gender' column to a factor (categorical variable)
data$Gender <- as.factor(data$Gender)

# Convert the 'Gender' column to numeric
data$Gender <- as.numeric(data$Gender)

# Print the first few rows of the data frame to check the changes
print(head(data))

The code changes the Gender column to a factor. A factor treats Gender as a category. Then, it changes the factor into numeric values. Each category is given a number.

Handling Outliers

Outliers are extreme values that can skew results. The IQR shows the spread of the middle 50% of your data. Values far outside this range may be outliers. Decide if you need to remove these outliers or handle them differently. Removing or adjusting outliers can make your analysis more accurate.

# Calculate the Interquartile Range (IQR)
Q1 <- quantile(data$Annual_Income, 0.25, na.rm = TRUE)
Q3 <- quantile(data$Annual_Income, 0.75, na.rm = TRUE)
IQR <- Q3 - Q1

# Define the lower and upper bounds for outliers
lower_bound <- Q1 - 1.5 * IQR
upper_bound <- Q3 + 1.5 * IQR

# Filter the data to capture outliers (values outside the IQR bounds)
outliers <- subset(data, Annual_Income < lower_bound | Annual_Income > upper_bound)

# View the outliers
print(outliers)

This code first calculates the Interquartile Range (IQR) for Annual_Income. It finds the lower and upper bounds for detecting outliers. Then, it filters the data to find rows where Annual_Income is outside these bounds (outliers).

Normalizing Data

Normalization changes numbers so they all fit between 0 and 1. This helps make different numbers comparable. If you don’t normalize, bigger numbers can affect your results more.

# Normalize the Spending_Score column
data$normalized_Spending_Score <- (data$Spending_Score - min(data$Spending_Score, na.rm = TRUE)) / 
                                   (max(data$Spending_Score, na.rm = TRUE) - min(data$Spending_Score, na.rm = TRUE))

# Print the head of the data
head(data)

This code normalizes the Spending_Score column. It subtracts the minimum value of Spending_Score from each value. Then, it divides by the range of Spending_Score. This scales the values to be between 0 and 1.

Conclusion

Data cleaning is essential for accurate analysis. It helps fix errors and inconsistencies in your data. The techniques mentioned in the article will help you get trustworthy results.