This quick tutorial shows how to allow missing values in your Pandera schemas without triggering validation errors. For a complete introduction to data validation with Pandera, see our Data Validation in Python with Pandera: A Practical Introduction guide.
The Problem
Real-world datasets rarely arrive perfectly clean. Customer emails might be missing, optional survey responses stay blank, and sensor readings fail periodically. When you apply Pandera’s default validation to data with these gaps, validation fails immediately because Pandera treats all columns as required by default. You end up with SchemaError exceptions even when missing values are expected and acceptable for your use case.
The Solution
The key is the nullable parameter, which tells Pandera whether a column can contain null values. Here’s what happens when you try to validate data with missing values using a standard schema:
import pandas as pd
import pandera.pandas as pa
from pandera.pandas import Column, DataFrameSchema, Check
# Data with missing values
data = {
'customer_id': [101, 102, 103, 104],
'email': ['alice@email.com', None, 'charlie@email.com', None],
'purchase_amount': [49.99, 29.99, None, 99.99]
}
df = pd.DataFrame(data)
# Schema without nullable - this will fail
schema = DataFrameSchema({
'customer_id': Column(int),
'email': Column(str),
'purchase_amount': Column(float, Check.greater_than(0))
})
validated_df = schema.validate(df)
This raises an error:
SchemaError: non-nullable series 'email' contains null values: 1 None 3 None Name: email, dtype: object
To fix this, add nullable=True to columns that can contain missing data:
# Schema with nullable columns
schema_with_nulls = DataFrameSchema({
'customer_id': Column(int, nullable=False),
'email': Column(str, nullable=True),
'purchase_amount': Column(float, Check.greater_than(0), nullable=True)
})
validated_df = schema_with_nulls.validate(df)
print("Validation successful!")
print(validated_df)
Output:
Validation successful! customer_id email purchase_amount 0 101 alice@email.com 49.99 1 102 None 29.99 2 103 charlie@email.com NaN 3 104 None 99.99
The validation now passes. Notice that customer_id has nullable=False to enforce that every row must have an ID, while email and purchase_amount allow missing values. This lets you be explicit about which fields are required versus optional.
When to Allow Nulls
Set nullable=True for optional fields like secondary contact information, supplementary measurements, or user-provided data that may be incomplete. Keep nullable=False (or omit it, since False is the default) for critical identifiers, required timestamps, or any field your downstream analysis depends on. If you allow nulls but still want to validate present values, combine nullable=True with checks — Pandera only applies checks to non-null values. For example, Column(float, Check.greater_than(0), nullable=True) accepts both missing values and positive numbers, but rejects zero or negatives.
Conclusion
The nullable parameter gives you precise control over where missing data is acceptable. This prevents false validation failures while still catching actual data quality issues in required fields.
