Temporal data, which includes dates and times, is a common feature of many datasets. Polars, a high-performance data manipulation library in Python, offers powerful tools for handling such data efficiently. In this tutorial, we’ll explore how to work with temporal data in Polars using its date and time functions.

Introduction to Temporal Data in Polars

Polars represents temporal data using specialized datatypes. These allow for efficient storage and manipulation of temporal information, providing a wide range of functions to perform operations on dates and times.

The specialized temporal datatypes include: 

• Date: Represents a calendar date without a time of day. 
• Time: Focuses only on the time of day. 
• Datetime: Represents a calendar date and time of day.
• Duration: Represents a time interval, similar to Python’s timedelta. 

Let’s work through some examples to illustrate how to work with temporal data in Polars.

Example 1: Creating a DataFrame with Temporal Data

We begin by creating a simple DataFrame containing temporal data:

import polars as pl
from datetime import datetime, date, time

df = pl.DataFrame({
    'date': [date(2023, 1, 1), date(2023, 2, 15), date(2023, 3, 30)],
    'time': [time(9, 0), time(14, 30), time(18, 45)],
    'datetime': [datetime(2023, 1, 1, 9, 0), datetime(2023, 2, 15, 14, 30), datetime(2023, 3, 30, 18, 45)]
})

print(df)

Output:

shape: (3, 3)
┌────────────┬──────────┬─────────────────────┐
│ date       ┆ time     ┆ datetime            │
│ ---        ┆ ---      ┆ ---                 │
│ date       ┆ time     ┆ datetime[μs]        │
╞════════════╪══════════╪═════════════════════╡
│ 2023-01-01 ┆ 09:00:00 ┆ 2023-01-01 09:00:00 │
│ 2023-02-15 ┆ 14:30:00 ┆ 2023-02-15 14:30:00 │
│ 2023-03-30 ┆ 18:45:00 ┆ 2023-03-30 18:45:00 │
└────────────┴──────────┴─────────────────────┘

This code creates a DataFrame with three columns: date, time, and datetime. Each column uses the appropriate Polars datatype for temporal data. Notice how Polars automatically recognizes and correctly formats each temporal type in the output.

Example 2: Extracting Components from Dates and Times

Polars provides a rich set of methods to extract various components from dates and times. Let’s extract the year, month, and day from the date column, and the hour and minute from the time column:

df_extracted = df.select([
    pl.col('date').dt.year().alias('year'),
    pl.col('date').dt.month().alias('month'),
    pl.col('date').dt.day().alias('day'),
    pl.col('time').dt.hour().alias('hour'),
    pl.col('time').dt.minute().alias('minute')
])

print(df_extracted)

Output:

shape: (3, 5)
┌──────┬───────┬─────┬──────┬────────┐
│ year ┆ month ┆ day ┆ hour ┆ minute │
│ ---  ┆ ---   ┆ --- ┆ ---  ┆ ---    │
│ i32  ┆ i8    ┆ i8  ┆ i8   ┆ i8     │
╞══════╪═══════╪═════╪══════╪════════╡
│ 2023 ┆ 1     ┆ 1   ┆ 9    ┆ 0      │
│ 2023 ┆ 2     ┆ 15  ┆ 14   ┆ 30     │
│ 2023 ┆ 3     ┆ 30  ┆ 18   ┆ 45     │
└──────┴───────┴─────┴──────┴────────┘

This example demonstrates how to use the dt accessor to apply date and time functions to columns. These methods are part of the descriptive functions in Polars’ temporal data namespace, allowing you to easily extract specific components from your temporal data. Note how each extracted component is stored as an integer type (i32 for year, i8 for others) for efficient memory usage and fast computations.

Example 3: Performing Date Arithmetic

Polars makes it easy to perform date arithmetic. Let’s add 7 days to each date and calculate the difference between the original and new dates:

df_date_math = df.select([
    pl.col('date').alias('original_date'),
    (pl.col('date') + pl.duration(days=7)).alias('date_plus_7_days'),
    (pl.col('date') + pl.duration(days=7) - pl.col('date')).alias('days_difference')
])

print(df_date_math)

Output:

shape: (3, 3)
┌───────────────┬──────────────────┬─────────────────┐
│ original_date ┆ date_plus_7_days ┆ days_difference │
│ ---           ┆ ---              ┆ ---             │
│ date          ┆ date             ┆ duration[ms]    │
╞═══════════════╪══════════════════╪═════════════════╡
│ 2023-01-01    ┆ 2023-01-08       ┆ 7d              │
│ 2023-02-15    ┆ 2023-02-22       ┆ 7d              │
│ 2023-03-30    ┆ 2023-04-06       ┆ 7d              │
└───────────────┴──────────────────┴─────────────────┘

This example showcases how to use pl.duration() for date arithmetic and how to perform operations between date columns. The Duration type in Polars is specifically designed for such time interval calculations, making it easy to add or subtract time periods from dates. Notice how the days_difference column is of type duration[ms], representing the time difference in milliseconds, but displayed as days for readability.

Example 4: Working with Timezones

Polars also supports working with timezones, which is useful for handling data from different regions or coordinating events across time zones. Let’s convert our datetime column to UTC and then to a different timezone:

df_timezone = df.select([
    pl.col('datetime').alias('original_datetime'),
    pl.col('datetime').dt.replace_time_zone('UTC').alias('utc_time'),
    pl.col('datetime').dt.replace_time_zone('UTC').dt.convert_time_zone('America/New_York').alias('new_york_time')
])

print(df_timezone)

Output:

shape: (3, 3)
┌─────────────────────┬─────────────────────────┬────────────────────────────────┐
│ original_datetime   ┆ utc_time                ┆ new_york_time                  │
│ ---                 ┆ ---                     ┆ ---                            │
│ datetime[μs]        ┆ datetime[μs, UTC]       ┆ datetime[μs, America/New_York] │
╞═════════════════════╪═════════════════════════╪════════════════════════════════╡
│ 2023-01-01 09:00:00 ┆ 2023-01-01 09:00:00 UTC ┆ 2023-01-01 04:00:00 EST        │
│ 2023-02-15 14:30:00 ┆ 2023-02-15 14:30:00 UTC ┆ 2023-02-15 09:30:00 EST        │
│ 2023-03-30 18:45:00 ┆ 2023-03-30 18:45:00 UTC ┆ 2023-03-30 14:45:00 EDT        │
└─────────────────────┴─────────────────────────┴────────────────────────────────┘

This example demonstrates how to use replace_time_zone() to set a timezone and convert_time_zone() to convert between timezones. These functions are part of Polars’ manipulation methods for temporal data, allowing you to handle the complexities of different time zones efficiently. Note how Polars automatically adjusts for Daylight Saving Time (EST vs EDT) in the New York timezone.

Additional Temporal Data Operations in Polars

Polars offers many more operations for working with temporal data:

1. Conversion: You can convert between different temporal types or to strings using methods like dt.cast_time_unit(), dt.strftime(), and dt.to_string().
2. Descriptive Methods: Besides the ones we’ve used, there are methods like dt.is_leap_year(), dt.quarter(), dt.weekday(), and many more for extracting specific information from dates.
3. Manipulation: You can perform operations like dt.round() to round dates to specific units, dt.truncate() to divide date ranges into buckets, or dt.offset_by() to shift dates by a specified interval.
4. Time Series Specific: For time series analysis, Polars provides methods to create date ranges (pl.date_range()), which is useful for creating time series data or filling in missing dates in your data.

Conclusion

Working with temporal data is a common task in data analysis. Polars provides a wide range of tools for handling dates and times efficiently. In this tutorial, we covered: 

1. Creating a DataFrame with temporal data
2. Extracting components from dates and times
3. Performing date arithmetic
4. Working with timezones

These examples showcase some of the key functionalities Polars offers for temporal data manipulation. As you become more comfortable with these basics, you can explore more advanced temporal operations in Polars, such as resampling time series data, rolling window calculations, and joining datasets based on temporal keys. Practice these examples and experiment with your own datasets to become proficient in handling temporal data with Polars.