Efficiently selecting and manipulating columns is a fundamental task in data analysis. Polars provides you with the flexible select() method to help you work with specific columns in your DataFrame. In this post, we’ll explore different ways to use select() for choosing, transforming, and renaming columns in Polars.

Basic Column Selection

Let’s start with the basics. The select() method allows you to choose specific columns from your DataFrame by passing their names as a list. Here’s a simple example:

import polars as pl

# Create a sample DataFrame
df = pl.DataFrame({
    'A': [1, 2, 3, 4, 5],
    'B': ['a', 'b', 'c', 'd', 'e'],
    'C': [10, 20, 30, 40, 50],
    'D': [100, 200, 300, 400, 500]
})

# Select columns 'A' and 'C'
result = df.select(['A', 'C'])
print(result)

Output:

shape: (5, 2)
┌─────┬─────┐
│ A   ┆ C   │
│ --- ┆ --- │
│ i64 ┆ i64 │
╞═════╪═════╡
│ 1   ┆ 10  │
│ 2   ┆ 20  │
│ 3   ┆ 30  │
│ 4   ┆ 40  │
│ 5   ┆ 50  │
└─────┴─────┘

In this example, we’ve created a DataFrame with four columns and used select() to choose only columns ‘A’ and ‘C’. The resulting DataFrame contains only these two columns, maintaining the original order of rows.

Selecting and Transforming Columns

The select() method is not limited to just picking existing columns. You can also use it to create new columns based on existing ones. Let’s see an example:

# Select columns 'A' and 'B', and create a new column 'A_squared'
result = df.select([
    'A',
    'B',
    (pl.col('A') ** 2).alias('A_squared')
])
print(result)

Output:

shape: (5, 3)
┌─────┬─────┬───────────┐
│ A   ┆ B   ┆ A_squared │
│ --- ┆ --- ┆ ---       │
│ i64 ┆ str ┆ i64       │
╞═════╪═════╪═══════════╡
│ 1   ┆ a   ┆ 1         │
│ 2   ┆ b   ┆ 4         │
│ 3   ┆ c   ┆ 9         │
│ 4   ┆ d   ┆ 16        │
│ 5   ┆ e   ┆ 25        │
└─────┴─────┴───────────┘

In this example, we’ve selected columns ‘A’ and ‘B’, and created a new column ‘A_squared’ by squaring the values in column ‘A’. The pl.col(‘A’) ** 2 expression creates the new column, and alias() is used to name it.

Using Regular Expressions for Column Selection

Polars allows you to use regular expressions to select columns based on patterns in their names. This is particularly useful when dealing with large datasets with many similarly named columns. Here’s an example:

# Create a DataFrame with more columns
df = pl.DataFrame({
    'name': ['Alice', 'Bob', 'Charlie'],
    'age': [25, 30, 35],
    'score_math': [85, 75, 90],
    'score_science': [80, 85, 95],
    'score_history': [75, 80, 85]
})

# Select all columns that start with 'score_'
result = df.select([
    'name',
    pl.col('^score_.*$')
])
print(result)

Output:

shape: (3, 4)
┌─────────┬────────────┬───────────────┬───────────────┐
│ name    ┆ score_math ┆ score_science ┆ score_history │
│ ---     ┆ ---        ┆ ---           ┆ ---           │
│ str     ┆ i64        ┆ i64           ┆ i64           │
╞═════════╪════════════╪═══════════════╪═══════════════╡
│ Alice   ┆ 85         ┆ 80            ┆ 75            │
│ Bob     ┆ 75         ┆ 85            ┆ 80            │
│ Charlie ┆ 90         ┆ 95            ┆ 85            │
└─────────┴────────────┴───────────────┴───────────────┘

In this example, we’ve used the regular expression ^score_.*$ to select all columns that start with ‘score_’. This feature allows you to select multiple columns based on a pattern, which can save you time when working with many similarly named columns.

The .* pattern in regular expressions functions as a wildcard. The dot (.) matches any single character, while the asterisk (*) means “zero or more” of the preceding element. Together, .* matches any sequence of characters, allowing you to capture everything that follows “score_”.

Breaking down the regex pattern ^score_.*$:

• ^ asserts the start of the string.
• score_ matches the literal text “score_”.
• .* matches any characters after “score_”.
• $ asserts the end of the string.

This combination acts as a flexible wildcard, matching all column names that start with “score_” regardless of what comes after it.

Excluding Columns

Sometimes, it’s easier to specify which columns you don’t want rather than listing all the ones you do want. Polars provides an exclude() method that can be used within select() for this purpose:

# Select all columns except 'age' and 'score_history'
result = df.select(pl.col("*").exclude("age", "score_history"))
print(result)

Output:

shape: (3, 3)
┌─────────┬────────────┬───────────────┐
│ name    ┆ score_math ┆ score_science │
│ ---     ┆ ---        ┆ ---           │
│ str     ┆ i64        ┆ i64           │
╞═════════╪════════════╪═══════════════╡
│ Alice   ┆ 85         ┆ 80            │
│ Bob     ┆ 75         ┆ 85            │
│ Charlie ┆ 90         ┆ 95            │
└─────────┴────────────┴───────────────┘

Here, we’ve used pl.col(“*”) to select all columns, and then exclude() to remove the ‘age’ and ‘score_history’ columns. This approach can be particularly useful when working with DataFrames that have a large number of columns.

Conclusion

The select() method in Polars is more than just a tool for basic column selection—it’s a gateway to the powerful expression system at the heart of this library. Beyond selecting or excluding columns, select() enables you to build expressions that transform, filter, and reshape your data in flexible ways. Whether you’re using regular expressions to match column patterns or performing custom transformations with Polars’ expressions like alias(), filter(), or col(), this method provides a dynamic way to work with your data.

As you experiment with select() and Polars’ expression system, you’ll find it increasingly intuitive to chain transformations, create new columns, or apply advanced calculations—all while benefiting from Polars’ speed and performance. The ability to combine expressions gives you a concise, efficient way to handle even complex data manipulations. The examples we’ve covered here are just the beginning. As you explore further, you’ll discover how select() can help tackle more advanced tasks, unlocking new possibilities for your data analysis projects.