How to Fix in Python: ValueError: Trailing data


One error you may encounter when using Python is:

ValueError: Trailing data

This error usually occurs when you attempt to import a JSON file into a pandas DataFrame, yet the data is written in lines separated by endlines like ‘\n‘.

The easiest way to fix this error is to simply specify lines=True when importing the data:

df = pd.read_json('my_data.json', lines=True)

The following example shows how to fix this error in practice.

How to Reproduce the Error

Suppose we have the following JSON file:

Now suppose we attempt to import this JSON file into a pandas DataFrame:

#attempt to import JSON file into pandas DataFrame
df = pd.read_json('Documents/DataFiles/my_data.json')

ValueError: Trailing data

We receive an error because the “Review” item in our JSON file contains \n to represent endlines.

How to Fix the Error

The easiest way to fix this error is to simply specify lines=True when importing the data:

#import JSON file into pandas DataFrame
df = pd.read_json('Documents/DataFiles/my_data.json', lines=True)

#view DataFrame
df

	ID	Rating	Review
0	A	8	Great movie.\nI would recommend it.
1	B	5	Mediocre movie.\nWould not recommend it.
2	C	3	Bad movie.\nI would not recommend.
3	D	7	Decent movie.\nI might recommend it.

Notice that we’re able to successfully import the JSON file into a pandas DataFrame without any errors.

If we’d like to remove the \n endlines from the “Review” column, we can use the following syntax:

#replace \n with empty space in 'Review' column
df['Review'] = df['Review'].str.replace('\n', ' ')

#view updated DataFrame
df

	ID	Rating	Review
0	A	8	Great movie. I would recommend it.
1	B	5	Mediocre movie. Would not recommend it.
2	C	3	Bad movie. I would not recommend.
3	D	7	Decent movie. I might recommend it.

The \n values are now removed from the “Review” column.

Additional Resources

The following tutorials explain how to perform other common operations in pandas:

How to Convert a Pandas DataFrame to JSON File
How to Convert a JSON File to Pandas DataFrame

Leave a Reply

Your email address will not be published. Required fields are marked *