Let’s learn how to perform string data manipulation in Pandas.
Preparation
We would need the Pandas packages installed, so we can install them using the following code:
pip install pandas
With the packages installed, let’s jump into the article.
String Manipulation in Pandas
A string is text that we consider a data point. In Python, you can see that the string is using a quote symbol (‘like this‘ or “like this”). The string could contain any character, including numbers, alphabet, symbols, and others.
Let’s try manipulating some sample text data in a Pandas DataFrame
import pandas as pd
df = pd.DataFrame({'text': ['This is a Text', 'We can do it', '11 Books @ Table']})
First, we can try to convert the string text to lowercase.
print(df['text'].str.lower())
Output:
0 this is a text 1 we can do it 2 11 books @ table
Likewise, we can convert the text to uppercase.
print(df['text'].str.upper())
0 THIS IS A TEXT 1 WE CAN DO IT 2 11 BOOKS @ TABLE
It’s also possible to remove the whitespace, either the trailing, leading, or both.
# Remove leading and trailing whitespace df['text'].str.strip() # Remove leading (left) whitespace df['text'].str.lstrip() # Remove trailing (right) whitespace df['text'].str.rstrip()
Next, we can replace particular characters in the string using the following code.
print(df['text'].str.replace(' ', '_'))
Output:
0 This_is_a_Text 1 We_can_do_it 2 11_Books_@_Table
With the string function, we can split the string into a list.
print(df['text'].str.split())
Output:
0 [This, is, a, Text] 1 [We, can, do, it] 2 [11, Books, @, Table]
We can also transform each text chunk (or word) into its own column.
print(df['text'].str.split(' ', expand=True))
Output:
0 1 2 3
0 This is a Text
1 We can do it
2 11 Books @ Table
Next let’s see how to detect if certain string characters are in the text or not.
print(df['text'].str.contains('Table'))
Output:
0 False 1 False 2 True
Now we shall see how we can get the string characters based on the position as well.
print(df['text'].str[0:5])
Output:
0 This 1 We ca 2 11 Bo
We can get the string’s length with the following function.
print(df['text'].str.len())
Output:
0 14 1 12 2 16
Lastly, you can use regex to replace the string with certain patterns.
print(df['text'].str.replace(r'[^\w]', '', regex=True))
Output:
0 ThisisaText 1 Wecandoit 2 11BooksTable
Mastering the string manipulation function could help improve your data analysis skills.
