You can use the following basic syntax to import and merge multiple CSV files located in the same folder into R:
df <- list.files(path='C:/my/path/to/files') %>% lapply(read_csv) %>% bind_rows
The following step-by-step example shows how to use this syntax in practice.
Step 1: Create & Export Multiple Data Frames
First, we’ll use the following code to create and export three data frames to CSV files:
#create three data frames
df1 <- data.frame(points=c(4, 5, 5, 6, 8, 9),
assists=c(3, 2, 4, 4, 6, 3))
df2 <- data.frame(points=c(2, 10, 14, 15),
assists=c(3, 2, 9, 3))
df3 <- data.frame(points=c(6, 8, 9),
assists=c(10, 6, 4))
#export all three data frames to CSV files
write.csv(df1, 'C:/Users/bob/Documents/my_data_files/df1.csv', row.names=FALSE)
write.csv(df2, 'C:/Users/bob/Documents/my_data_files/df2.csv', row.names=FALSE)
write.csv(df3, 'C:/Users/bob/Documents/my_data_files/df3.csv', row.names=FALSE)
I can navigate to this folder and see that the three CSV files were successfully exported:
Step 2: Import & Merge Multiple CSV Files
Next, we’ll use the following code to import and merge all three CSV files into one data frame in R:
library(dplyr)
library(readr)
#import and merge all three CSV files into one data frame
df <- list.files(path='C:/Users/bob/Documents/my_data_files') %>%
lapply(read_csv) %>%
bind_rows
#view resulting data frame
df
# A tibble: 13 x 2
points assists
1 4 3
2 5 2
3 5 4
4 6 4
5 8 6
6 9 3
7 2 3
8 10 2
9 14 9
10 15 3
11 6 10
12 8 6
13 9 4
Notice that all three CSV files have been successfully merged into one data frame.
We can see that the resulting data frame has 13 rows and 2 columns.
Note: If the data frames do not have matching column names, R will still merge all of the data frames and simply fill in missing values with NA values.
Additional Resources
The following tutorials explain how to work with other file types in R:
How to Import Excel Files into R
How to Import TSV Files into R
How to Import Zip Files into R
How to Import SAS Files into R
How to Import .dta Files into R
Hello the group
How to fill in messing value of daily rainfall data using R?
Hello, thank you for this, it’s really helpful!
I have a query: I happen to have csvs with some fluff at the start, and need to import only from row 90 of every file; do you know of an easy way to incorporate that in this?
Thanks again!
Thank you very much
Hi Zach
I have to merge 56 CSV files, so I thought it would be good to try your code.
I started with using R Studio Desktop with tidyverse installed and added.
However, the df came back with 0 obs. of 0 variables
Just in case I made a mistake with my R Studio Desktop setup, I used R Studio Cloud with tidyverse installed and added. I ran your code line for line, and the df still came back with 0 obs. of 0 variables and A tibble: 0 × 0
What am I missing here? Any help would be excellent.
Cheers
Cledwyn
Hi Cledwyn…
It sounds like there might be an issue with the file paths or the content of your CSV files. Let’s go through the process step-by-step to troubleshoot:
### Step 1: Verify the CSV Files
1. **Check the Files**: Ensure that all 56 CSV files are in the correct directory.
2. **Preview the Files**: Open a few of the CSV files to ensure they contain data and are formatted properly (e.g., have headers, no corrupted rows, etc.).
### Step 2: Verify the Directory and File Paths
1. **Directory Path**: Make sure you specify the correct path where your CSV files are stored. For example:
“`r
files_path <- "path/to/your/csv/files" ``` 2. **List the Files**: Use the following command to check if R is detecting the files correctly: ```r csv_files <- list.files(path = files_path, pattern = "*.csv", full.names = TRUE) print(csv_files) ``` If this returns an empty vector, it means the path or pattern is incorrect. ### Step 3: Merge CSV Files Use the following code to merge the CSV files into a single data frame: ```r library(tidyverse) # Define the path to your CSV files files_path <- "path/to/your/csv/files" # List all CSV files in the directory csv_files <- list.files(path = files_path, pattern = "*.csv", full.names = TRUE) # Check the list of files print(csv_files) # Read and merge all CSV files df <- csv_files %>%
map_dfr(read_csv)
# View the resulting data frame
print(df)
“`
### Step 4: Debugging Common Issues
1. **No Files Detected**: If `csv_files` is empty, double-check:
– The directory path (`files_path`) is correct.
– The `pattern = “*.csv”` matches the file extensions exactly. For example, if the files are named with uppercase extensions (`.CSV`), use `pattern = “*.CSV”` or make it case-insensitive with `pattern = “(?i)\\.csv$”`.
2. **Empty Data Frame**: If `df` is empty:
– Check if the CSV files themselves are empty or contain only headers.
– Try reading one file manually to confirm it works:
“`r
test_file <- read_csv(csv_files[1]) print(test_file) ``` 3. **Different Column Structures**: If the CSV files have different column names or structures, this could cause problems during merging. To handle this: - Use `bind_rows()` instead of `map_dfr()` to merge the files and automatically handle mismatched columns: ```r df <- csv_files %>%
map(read_csv) %>%
bind_rows()
“`
4. **Reading Errors**: If there are issues reading the files, use `read.csv()` as a fallback:
“`r
df <- csv_files %>%
map(~ read.csv(.x, stringsAsFactors = FALSE)) %>%
bind_rows()
“`
### Step 5: Verify the Result
After the merge, check the structure of the resulting data frame:
“`r
str(df)
summary(df)
“`
### Additional Tips
– **Error Messages**: If you encounter specific error messages, share them for further assistance.
– **File Preview**: If a file is malformed, clean or preprocess it separately before merging.
– **Logging**: Print progress during the merge to identify which file might be causing an issue:
“`r
df <- csv_files %>%
map_dfr(~ {
print(.x)
read_csv(.x)
})
“`