Data validation often requires organization-specific rules that go beyond standard checks. Great Expectations provides powerful built-in validation methods, but there are times when you need to create custom rules that perfectly match your business requirements. This guide will show you how to create custom expectation classes that encapsulate your unique data quality standards while maintaining consistent validation practices across your organization.

Understanding Custom Expectations

Great Expectations provides a rich library of validation classes through its Expectation Gallery, offering numerous built-in validations for common data quality checks. However, sometimes you need to extend these base classes to create more specialized validations. Custom expectation classes allow you to build upon these foundation classes, creating reusable, self-documenting components that reflect your specific business rules.

Consider validating order values in an e-commerce system: you might need specific rules about minimum and maximum order values based on your business policies around pricing errors and approval thresholds. By extending a base expectation class like ExpectColumnValuesToBeBetween, you can create a specialized validation that encodes these rules while maintaining the robust validation framework that Great Expectations provides.

Benefits of Custom Expectation Classes

Creating custom expectation classes offers several key advantages that make them superior to using basic expectations repeatedly:

1. A custom expectation class encodes business logic directly in the code, ensuring consistent validation rules across your organization. When business rules change, you can update them in one place rather than hunting down every instance where the validation is used.
2. The class can include detailed documentation that explains not just what the validation does, but why certain rules exist. This documentation automatically appears in Data Docs, making it easier for team members to understand the purpose of each validation.
3. By using domain-specific naming, custom expectations make their intent immediately clear to anyone reading the code. Instead of generic method names, you can use terms that reflect your business domain and make the code more maintainable.

These advantages transform Great Expectations from a validation tool into a framework for encoding, documenting, and maintaining your organization’s data quality standards in a centralized, maintainable way.

Creating Your First Custom Expectation

Let’s create our custom expectation by extending the ExpectColumnValuesToBeBetween class from Great Expectations’ core library. This base class is perfect for our order validation use case since we need to check if values fall within a specific range. By subclassing it, we inherit its validation logic while adding our business-specific defaults and documentation.

import great_expectations as gx
import pandas as pd

class ExpectOrderValueWithinRange(gx.expectations.ExpectColumnValuesToBeBetween):
    """An expectation that validates if order values fall within an acceptable range based on business rules."""
    
    # Default values that reflect business logic
    column: str = "order_value"
    min_value: float = 10.0  # Minimum order value
    max_value: float = 10000.0  # Maximum order value
    
    # Custom description for Data Docs
    description: str = """
    Order values must be between **$10** and **$10,000**.
    - Orders below $10 may indicate pricing errors
    - Orders above $10,000 require special approval
    """

This custom class demonstrates several key concepts we discussed earlier:

• We’ve given it a descriptive name (ExpectOrderValueWithinRange) that clearly communicates its business purpose.
• We’ve set default values that reflect our business rules ($10 minimum, $10,000 maximum).
• We’ve included detailed documentation that explains both the validation rules and their business context.
• The description uses Markdown formatting, which will render nicely in Data Docs.

Implementing the Custom Expectation

Now that we’ve defined our custom expectation class, let’s see how to implement it with real data. We’ll create a simple demonstration that validates a set of order values:

def demonstrate_custom_expectation():
    # Create sample data
    data = {
        'order_value': [5.0, 25.0, 150.0, 12000.0, 50.0],
        'order_date': ['2025-01-01'] * 5
    }
    df = pd.DataFrame(data)
    
    # Set up Great Expectations context
    context = gx.get_context()
    
    # Add data source and create batch
    data_source = context.data_sources.add_pandas(name="order_data")
    data_asset = data_source.add_dataframe_asset(name="orders")
    batch_definition = data_asset.add_batch_definition_whole_dataframe("order_batch")
    batch = batch_definition.get_batch(batch_parameters={"dataframe": df})
    
    # Create and validate expectation
    expectation = ExpectOrderValueWithinRange()
    results = batch.validate(expectation)
    
    return results

Let’s break down what this implementation does:

1. Creates a sample dataset with five orders, including some that violate our business rules.
2. Sets up a Great Expectations context and data source.
3. Creates a batch of data for validation.
4. Instantiates our custom expectation class.
5. Validates the data against our business rules.

Running the Validation

When we run our validation, we can see exactly which orders violate our business rules. 

if __name__ == "__main__":
    results = demonstrate_custom_expectation()
    print("\nValidation Results:")
    print(f"Success: {results.success}")
    if not results.success:
        print(f"Invalid values: {results.result['partial_unexpected_list']}")
        print(f"Total failures: {results.result['unexpected_count']}")

Below is the output we should get when we run our customized expectation class: 

Validation Results:
Success: False
Invalid values: [5.0, 12000.0]
Total failures: 2

The validation results show us that:

• One order ($5.00) falls below our minimum threshold.
• One order ($12,000.00) exceeds our maximum threshold.
• Three orders are within acceptable bounds.

Best Practices for Custom Expectations

When creating your own custom expectations, consider these best practices:

1. Name for Clarity: Choose class names that clearly communicate the business purpose (like ExpectOrderValueWithinRange).
2. Document Thoroughly: Use the description attribute to explain both the what and why of your validation rules.
3. Set Sensible Defaults: Choose default values that match your most common use cases.
4. Keep It Simple: Inherit from existing expectation classes when possible instead of creating complex custom logic.
5. Maintain Consistency: Once deployed, avoid overriding the default values when creating instances.

By following these practices, you can create custom expectations that are both powerful and maintainable, while providing clear documentation for your team.

Conclusion

Custom expectation classes represent a powerful way to elevate your data validation strategy beyond basic checks. By combining Great Expectations’ robust validation framework with your organization’s specific business rules, you can create sophisticated, self-documenting validation components that grow with your needs. These custom classes ensure that your data quality standards are not just rules in a document, but living code that actively enforces your business requirements.

Whether you’re handling financial transactions, monitoring IoT sensors, or validating customer data, custom expectations help bridge the gap between technical implementation and business requirements. The ability to encode business logic, document validation rules, and maintain consistency across your organization makes custom expectations an essential tool for any team serious about data quality.