In Polars, the sort() method provides a flexible way to order your data by one or more columns, whether you’re working with numerical values, dates, or categorical data. It allows you to control the sort order and handle missing values with ease. In this tutorial, we’ll dive into different techniques for sorting a Polars DataFrame to help streamline your data manipulation tasks.

Setting Up Our Example DataFrame

Let’s start by creating a sample DataFrame that we’ll use throughout our examples:

import polars as pl

# Create a sample DataFrame
df = pl.DataFrame({
    'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eve'],
    'age': [25, 30, 35, 28, 32],
    'salary': [50000, 60000, 75000, 55000, 65000],
    'department': ['HR', 'IT', 'Finance', 'Marketing', 'IT']
})
print(df)

Output:

shape: (5, 4)
┌─────────┬─────┬────────┬────────────┐
│ name    ┆ age ┆ salary ┆ department │
│ ---     ┆ --- ┆ ---    ┆ ---        │
│ str     ┆ i64 ┆ i64    ┆ str        │
╞═════════╪═════╪════════╪════════════╡
│ Alice   ┆ 25  ┆ 50000  ┆ HR         │
│ Bob     ┆ 30  ┆ 60000  ┆ IT         │
│ Charlie ┆ 35  ┆ 75000  ┆ Finance    │
│ David   ┆ 28  ┆ 55000  ┆ Marketing  │
│ Eve     ┆ 32  ┆ 65000  ┆ IT         │
└─────────┴─────┴────────┴────────────┘

This DataFrame contains information about employees, including their names, ages, salaries, and departments.

Example 1: Sorting by a Single Column

Let’s start with the simplest case: sorting the DataFrame by a single column in ascending order.

# Sort the DataFrame by age in ascending order
df_sorted_age = df.sort('age')
print(df_sorted_age)

Output:

shape: (5, 4)
┌─────────┬─────┬────────┬────────────┐
│ name    ┆ age ┆ salary ┆ department │
│ ---     ┆ --- ┆ ---    ┆ ---        │
│ str     ┆ i64 ┆ i64    ┆ str        │
╞═════════╪═════╪════════╪════════════╡
│ Alice   ┆ 25  ┆ 50000  ┆ HR         │
│ David   ┆ 28  ┆ 55000  ┆ Marketing  │
│ Bob     ┆ 30  ┆ 60000  ┆ IT         │
│ Eve     ┆ 32  ┆ 65000  ┆ IT         │
│ Charlie ┆ 35  ┆ 75000  ┆ Finance    │
└─────────┴─────┴────────┴────────────┘

In this example, we use sort(‘age’) to arrange the DataFrame based on the ‘age’ column. By default, Polars sorts in ascending order.

To sort in descending order, you can use the descending parameter:

# Sort the DataFrame by salary in descending order
df_sorted_salary_desc = df.sort('salary', descending=True)
print(df_sorted_salary_desc)

Output:

shape: (5, 4)
┌─────────┬─────┬────────┬────────────┐
│ name    ┆ age ┆ salary ┆ department │
│ ---     ┆ --- ┆ ---    ┆ ---        │
│ str     ┆ i64 ┆ i64    ┆ str        │
╞═════════╪═════╪════════╪════════════╡
│ Charlie ┆ 35  ┆ 75000  ┆ Finance    │
│ Eve     ┆ 32  ┆ 65000  ┆ IT         │
│ Bob     ┆ 30  ┆ 60000  ┆ IT         │
│ David   ┆ 28  ┆ 55000  ┆ Marketing  │
│ Alice   ┆ 25  ┆ 50000  ┆ HR         │
└─────────┴─────┴────────┴────────────┘

Example 2: Sorting by Multiple Columns

Polars allows you to sort by multiple columns, which is useful when you want to establish a hierarchy in your sorting criteria.

# Sort the DataFrame by department (ascending) and then by salary (descending)
df_sorted_multi = df.sort(['department', 'salary'], descending=[False, True])
print(df_sorted_multi)

Output:

shape: (5, 4)
┌─────────┬─────┬────────┬────────────┐
│ name    ┆ age ┆ salary ┆ department │
│ ---     ┆ --- ┆ ---    ┆ ---        │
│ str     ┆ i64 ┆ i64    ┆ str        │
╞═════════╪═════╪════════╪════════════╡
│ Charlie ┆ 35  ┆ 75000  ┆ Finance    │
│ Alice   ┆ 25  ┆ 50000  ┆ HR         │
│ Eve     ┆ 32  ┆ 65000  ┆ IT         │
│ Bob     ┆ 30  ┆ 60000  ┆ IT         │
│ David   ┆ 28  ┆ 55000  ┆ Marketing  │
└─────────┴─────┴────────┴────────────┘

In this example, we first sort by the ‘department’ column in ascending order, and then by the ‘salary’ column in descending order within each department. The descending parameter takes a list of boolean values corresponding to each column in the sorting order.

Example 3: Using Expressions for Complex Sorting with Aliases

Polars accepts the use of its internal expression language that allows for more complex sorting scenarios. Let’s create an example that calculates a salary-to-age ratio (a simplified measure of earning potential), adds it as a new column, and then sorts the DataFrame based on this new column, all in a single step:

# Calculate the salary-to-age ratio, add it as a new column, and sort in a single step
df_with_ratio = df.with_columns(
    (pl.col('salary') / pl.col('age')).alias('earning_potential')
).sort('earning_potential')
print(df_with_ratio)

Output:

shape: (5, 5)
┌─────────┬─────┬────────┬────────────┬───────────────────┐
│ name    ┆ age ┆ salary ┆ department ┆ earning_potential │
│ ---     ┆ --- ┆ ---    ┆ ---        ┆ ---               │
│ str     ┆ i64 ┆ i64    ┆ str        ┆ f64               │
╞═════════╪═════╪════════╪════════════╪═══════════════════╡
│ David   ┆ 28  ┆ 55000  ┆ Marketing  ┆ 1964.285714       │
│ Alice   ┆ 25  ┆ 50000  ┆ HR         ┆ 2000.0            │
│ Bob     ┆ 30  ┆ 60000  ┆ IT         ┆ 2000.0            │
│ Eve     ┆ 32  ┆ 65000  ┆ IT         ┆ 2031.25           │
│ Charlie ┆ 35  ┆ 75000  ┆ Finance    ┆ 2142.857143       │
└─────────┴─────┴────────┴────────────┴───────────────────┘

In this example, we’re using Polars expressions to:

1. Calculate the salary-to-age ratio using pl.col(‘salary’) / pl.col(‘age’).
2. Give this calculated ratio an alias ‘earning_potential’ using .alias(‘earning_potential’).
3. Add this new column to our DataFrame using with_columns().
4. Immediately sort the DataFrame by ‘earning_potential’ in ascending order.

This approach showcases how you can use arithmetic operations, column references, and aliases to create complex sorting criteria and add calculated columns to your results in Polars, all while maintaining clean and efficient code.

Example 4: Null Handling in Sorting

Polars provides options for handling null values when sorting. By default, null values are placed last in ascending order and first in descending order. You can modify this behavior using the nulls_last parameter:

# Create a DataFrame with null values
df_with_nulls = pl.DataFrame({
    'name': ['Alice', 'Bob', None, 'David', 'Eve'],
    'score': [80, 90, 75, None, 85]
})
# Sort by score, placing nulls last regardless of sort order
df_sorted_nulls = df_with_nulls.sort('score', nulls_last=True)
print(df_sorted_nulls)

Output:

shape: (5, 2)
┌───────┬───────┐
│ name  ┆ score │
│ ---   ┆ ---   │
│ str   ┆ i64   │
╞═══════╪═══════╡
│ null  ┆ 75    │
│ Alice ┆ 80    │
│ Eve   ┆ 85    │
│ Bob   ┆ 90    │
│ David ┆ null  │
└───────┴───────┘

In this example, we sort the ‘score’ column and explicitly place null values at the end of the sorted DataFrame, regardless of whether we’re sorting in ascending or descending order.

Conclusion

The sort() method in Polars provides a flexible and efficient way to order your DataFrame based on one or more columns. Whether you need a simple sort on a single column or a complex multi-column sort with custom expressions, Polars has you covered.

Remember these key points when using sort():

• For single-column sorting, simply provide the column name.
• Use lists to sort by multiple columns.
• The descending parameter controls the sort order.
• Leverage Polars expressions for more complex sorting scenarios.
• Use nulls_last to control the placement of null values in the sorted result.

By practicing the sort() method, you’ll be able to efficiently organize and analyze your data in Polars DataFrames, enhancing the precision and efficiency of your data analysis and decision-making processes.