Introduction

pytest is a mature full-featured testing tool for testing Python code. This article will focus on a particular feature of pytest: marking tests as expected to fail, commonly referred to as xfail.

Marking tests as xfail allows you to acknowledge a particular test is flawed without having to comment out the code or remove the test altogether. This is useful when you are developing new features or debugging existing ones, and you know some tests will fail during the process. By marking the test as xfail, you acknowledge the failure without breaking your entire testing process.

Using pytest xfail

Before you can use pytest, you need to install it. This can be accomplished with pip, the Python package manager:

pip install pytest

Let’s start with a simple function that will fail. Consider the following example:

def add(a, b):
    return a + b
def test_add():
    assert add(1, 1) == 3  # This test is expected to fail

In this case, the test_add function is designed to fail because 1 + 1 does not equal 3.

To mark this test as an expected failure, we use the xfail marker as follows:

import pytest

@pytest.mark.xfail
def test_add():
    assert add(1, 1) == 3  # This test is expected to fail

By adding the @pytest.mark.xfail decorator before the test function, we indicate that this function is expected to fail.

To run the tests, use the pytest command in your terminal:

pytest

When pytest runs, it reports any tests marked with xfail as “XFAIL” in the output, indicating that these tests were expected to fail and did fail. If a test marked with xfail actually passes, it is reported as “XPASS,” meaning it was expected to fail but passed.

The xfail marker can also take an optional reason, which can be used to document why the test is expected to fail:

@pytest.mark.xfail(reason="Known bug in the add function")
def test_add():
    assert add(1, 1) == 3

You can group multiple tests that are expected to fail under the xfail marker:

@pytest.mark.xfail
def test_subtract():
    assert subtract(2, 1) == 0  # Expected to fail

@pytest.mark.xfail(reason="Feature not implemented")
def test_multiply():
    assert multiply(2, 3) == 7  # Expected to fail

The xfail marker can also be used with parameterized tests:

import pytest
@pytest.mark.parametrize("a, b, expected", [
    (1, 1, 2),
    (2, 2, 5),  # This case is expected to fail
])
@pytest.mark.xfail
def test_add(a, b, expected):
    assert add(a, b) == expected

Wrapping Up

Using the xfail marker in pytest is an effective way to manage your tests during the development process. Make sure you check out the pytest documentation website for more information.