You can use the following basic syntax to merge multiple CSV files located in the same folder into a pandas DataFrame:
import pandas as pd
import glob
import os
#define path to CSV files
path = r'C:\Users\bob\Documents\my_data_files'
#identify all CSV files
all_files = glob.glob(os.path.join("*.csv"))
#merge all CSV files into one DataFrame
df = pd.concat((pd.read_csv(f) for f in all_files), ignore_index=True)
This particular example will merge all of the CSV files located in the folder called my_data_files into one pandas DataFrame.
The following example shows how to use this syntax in practice.
Example: Merge Multiple CSV Files in Pandas
Suppose I have a folder on my computer called my_data_files that contains three CSV files:
Each CSV file contains two columns called points and assists, which represents the points and assists of various basketball players.
Here is what the first CSV called df1 looks like:
We can use the following syntax to merge all three CSV files from the folder into one pandas DataFrame:
import pandas as pd
import glob
import os
#define path to CSV files
path = r'C:\Users\bob\Documents\my_data_files'
#identify all CSV files
all_files = glob.glob(os.path.join("*.csv"))
#merge all CSV files into one DataFrame
df = pd.concat((pd.read_csv(f) for f in all_files), ignore_index=True)
#view resulting DataFrame
print(df)
points assists
0 4 3
1 5 2
2 5 4
3 6 4
4 8 6
5 9 3
6 2 3
7 10 2
8 14 9
9 15 3
10 6 10
11 8 6
12 9 4
Notice that all three CSV files have been successfully imported and merged into one DataFrame.
The final DataFrame contains 13 rows and 2 columns.
Note: You can find the complete documentation for the pandas read_csv() function here.
Additional Resources
The following tutorials explain how to perform other common tasks in Python:
Pandas: How to Skip Rows when Reading CSV File
Pandas: How to Append Data to Existing CSV File
Pandas: How to Specify dtypes when Importing CSV File
Pandas: How to Set Column Names when Importing CSV File