Image by Editor | ChatGPT

Introduction

Feature engineering is one of the most time-consuming and complex aspects of building machine learning models. Traditional feature engineering techniques require manual intervention and expert knowledge to identify useful patterns in data. Deep feature synthesis (DFS) automates this process. It generates features for relational and temporal data.

DFS is an automated method that creates complex features by stacking simple operations. The deep refers to stacking multiple steps to generate new, higher-level features. This process automatically builds features that capture relationships between different parts of your data.

In this article, we’ll look closer at how DFS works. We’ll use Featuretools to perform feature engineering on a mock customer transaction dataset.

DFS works with an EntitySet, which is a collection of structured data in Featuretools. It transforms and combines data to create new features that describe complex behavior.

Preparing the Data

Before running DFS, your data needs to be structured into an EntitySet. Each DataFrame represents a different entity, such as customers, transactions, or sessions. Relationships between these entities are defined using foreign keys.

We will use a mock customer transaction dataset to demonstrate how to prepare the data.

import featuretools as ft

# Load a demo dataset that simulates customer transactions
es = ft.demo.load_mock_customer(return_entityset=True)
print(es)

Entityset: transactions
DataFrames:
    transactions [Rows: 500, Columns: 6]
    products [Rows: 5, Columns: 3]
    sessions [Rows: 35, Columns: 5]
    customers [Rows: 5, Columns: 5]
Relationships:
    transactions.product_id -> products.product_id
    transactions.session_id -> sessions.session_id
    sessions.customer_id -> customers.customer_id

Running DFS

Once your data is prepared as an EntitySet, you can use DFS to generate new features. To run DFS, we specify the target DataFrame and the aggregation and transformation primitives. For instance, we can create features like:

• The count of sessions for each customer
• The month of a customer’s birthday

# Running DFS with aggregation and transformation primitives
feature_matrix, feature_defs = ft.dfs(
    entityset=es,
    target_dataframe_name="customers",  # Target dataframe is customers
    agg_primitives=["count"],  # Aggregation primitive
    trans_primitives=["month"],  # Transformation primitive
    max_depth=1  # Maximum depth of feature synthesis
)

# Display the resulting feature matrix
print(feature_matrix)

            zip_code  COUNT(sessions) MONTH(birthday) MONTH(join_date)
customer_id                                                           
5              60091                6               7                7
4              60091                8               8                4
1              60091                8               7                4
3              13244                6              11                8
2              13244                7               8                4

Creating Deeper Features

The true power of DFS comes from stacking multiple aggregation and transformation operations to create more expressive features. By setting a higher `max_depth`, you can generate features that combine multiple primitives.

Let’s explore how setting max_depth=2 generates more complex features.

# Running DFS with max_depth=2
feature_matrix, feature_defs = ft.dfs(
    entityset=es,
    target_dataframe_name="customers",
    agg_primitives=["mean", "sum", "mode"],
    trans_primitives=["month", "hour"],
    max_depth=2
)

Let’s break down one of the features at depth 2:

feature_matrix[["MEAN(sessions.SUM(transactions.amount))"]]

This feature performs the following operations:

• SUM(transactions.amount): Calculates the total transaction amount per session
• MEAN(sessions.SUM(transactions.amount)): Calculates the average transaction amount across all sessions for each customer

Another depth 2 feature calculates the most common hour of the day that each customer starts their sessions.

feature_matrix[["MODE(sessions.HOUR(session_start))"]]

This feature performs the following operations:

• HOUR(sessions.session_start): Extracts the hour from the start time of each session
• MODE(sessions.HOUR(session_start)): Identifies the most frequent hour for each customer across all their sessions

Conclusion

DFS automates feature engineering, which cuts down on manual work and helps create useful features. DFS combines simple and complex methods to find hidden patterns in data. Tools like Featuretools allow you to generate, visualize, and explain features, helping you understand how they affect your model. Using DFS can improve your models and speed up data science tasks.