languages/python/testing.md

Python Testing Guide: pytest Ecosystem (2026)

A practical guide to the modern pytest ecosystem: fixtures and scoping, parametrization, coverage.py, Hypothesis property-based testing, pytest-xdist parallelism, and pytest-mock for maintainable, fast test suites.

Introduction / Overview

pytest remains the de facto standard test runner for Python in 2026. Its plugin ecosystem - fixtures, parametrization, coverage integration, property-based testing via Hypothesis, and parallel execution via pytest-xdist - covers nearly every testing need without resorting to unittest's more verbose class-based API.

This document focuses on the language-level pytest ecosystem. Framework-specific testing patterns (FastAPI's TestClient, Django's pytest-django) build directly on top of these fundamentals; see the respective framework documents for those layers.

When to use this guidance: Setting up a new test suite, improving fixture design in an existing one, introducing property-based tests for critical business logic, or speeding up a slow CI pipeline.

Core Concepts

  • Fixtures as dependency injection for tests: pytest fixtures provide setup/teardown and shared state without the boilerplate of setUp/tearDown methods, and compose naturally via function arguments.
  • Fixture scope controls cost vs. isolation: Scope (function, class, module, session) trades test isolation against setup cost. Wider scope is faster but risks state leaking between tests.
  • Parametrization separates test logic from test data: A single test function can validate dozens of input/output combinations without duplicating assertions.
  • Coverage is a signal, not a target: coverage.py (via pytest-cov) measures which lines and branches executed, but 100% coverage does not imply correctness - it only means no code was left completely untested.
  • Property-based testing complements example-based testing: Hypothesis generates a wide range of inputs (including adversarial edge cases) from a strategy, uncovering bugs that hand-written examples miss.
  • Parallelism trades determinism for speed: pytest-xdist distributes tests across workers/processes; tests must be independent (no shared mutable file/DB state) to be parallel-safe.

Best Practices

1. Design Fixtures with the Narrowest Correct Scope

Rationale: function-scoped fixtures (the default) guarantee isolation but re-run setup for every test. Widening scope to module or session is a deliberate performance trade-off that should only apply to genuinely expensive, side-effect-free resources.

import pytest

@pytest.fixture(scope="session")
def database_engine():
    # Expensive: created once per test session
    engine = create_engine("postgresql://test-db/...")
    yield engine
    engine.dispose()

@pytest.fixture
def db_session(database_engine):
    # Cheap, function-scoped: fresh transaction per test, rolled back after
    connection = database_engine.connect()
    transaction = connection.begin()
    session = Session(bind=connection)
    yield session
    session.close()
    transaction.rollback()
    connection.close()

Why this works well in production: The expensive engine/connection pool is created once; each test still gets full transactional isolation via rollback, so tests never leak state into one another.

2. Use Fixture Factories for Parameterized Object Creation

Rationale: A plain fixture returns one fixed object. A factory fixture (a fixture that returns a function) lets each test customize the object it needs while reusing shared setup logic.

@pytest.fixture
def make_user(db_session):
    def _make_user(*, email: str = "user@example.com", is_active: bool = True) -> User:
        user = User(email=email, is_active=is_active)
        db_session.add(user)
        db_session.flush()
        return user
    return _make_user

def test_inactive_user_cannot_login(make_user, auth_service):
    user = make_user(is_active=False)
    with pytest.raises(AuthenticationError):
        auth_service.login(user.email, "password")

3. Use parametrize to Cover Input Variations Without Duplication

Rationale: Table-driven tests keep the assertion logic in one place and make it trivial to add a new case by adding one line.

import pytest

@pytest.mark.parametrize(
    "raw_input, expected",
    [
        ("  hello  ", "hello"),
        ("HELLO", "hello"),
        ("", ""),
        ("already-clean", "already-clean"),
    ],
    ids=["strips-whitespace", "lowercases", "empty-string", "no-op"],
)
def test_normalize(raw_input: str, expected: str) -> None:
    assert normalize(raw_input) == expected

Production tip: Always supply ids for non-trivial parameter sets - failure output shows the id, not the raw tuple, which dramatically speeds up debugging.

4. Track Branch Coverage, Not Just Line Coverage

Rationale: Line coverage can be misleadingly high while conditional branches (the else of an if, exception paths) remain untested. coverage.py with branch = true closes that gap.

# pyproject.toml
[tool.coverage.run]
branch = true
source = ["src"]

[tool.coverage.report]
fail_under = 85
show_missing = true
exclude_lines = [
    "pragma: no cover",
    "if TYPE_CHECKING:",
    "raise NotImplementedError",
]
uv run pytest --cov --cov-report=term-missing --cov-report=xml

Production tip: Set fail_under as a CI gate, but treat coverage regressions (a PR that lowers the percentage) as a stronger signal than an absolute threshold - a codebase can plateau at 85% and still be well tested if the uncovered 15% is deliberately excluded boilerplate.

5. Use Hypothesis for Property-Based Testing of Pure Logic

Rationale: Hypothesis generates hundreds of inputs per test run, including boundary values (empty collections, negative numbers, Unicode edge cases) that hand-written examples routinely miss. It is most valuable on pure functions with well-defined invariants.

from hypothesis import given, strategies as st

@given(st.lists(st.integers()))
def test_sort_is_idempotent(values: list[int]) -> None:
    once = sorted(values)
    twice = sorted(once)
    assert once == twice

@given(st.integers(min_value=0), st.integers(min_value=1))
def test_divmod_reconstructs_original(a: int, b: int) -> None:
    quotient, remainder = divmod(a, b)
    assert quotient * b + remainder == a

Production tip: Cap maxexamples in PR-blocking CI (settings(maxexamples=50)) for fast feedback, and run a larger nightly job (max_examples=1000) to hunt for rare edge cases. Hypothesis is safe to combine with pytest-xdist; each worker gets an independent seed.

6. Parallelize with pytest-xdist Once Suite Runtime Crosses ~2 Minutes

Rationale: pytest-xdist distributes tests across CPU cores, often cutting wall-clock CI time by 3-8x on multi-core runners, but requires strict test independence.

uv run pytest -n auto --dist=loadgroup
@pytest.mark.xdist_group(name="serial_migration_tests")
def test_requires_exclusive_database_access():
    ...

Why this works well in production: --dist=loadgroup keeps explicitly grouped tests on the same worker (for tests that must run serially against a shared resource) while distributing everything else freely.

7. Use pytest-mock's mocker Fixture Instead of Raw unittest.mock

Rationale: mocker auto-undoes all patches at the end of each test, eliminating the common bug of a mock.patch context manager or decorator leaking into subsequent tests.

def test_sends_welcome_email(mocker, user_service):
    send_email = mocker.patch("myapp.services.user_service.send_email")
    user_service.register(email="new@example.com")
    send_email.assert_called_once_with(
        to="new@example.com", template="welcome"
    )

Common Pitfalls & Anti-Patterns

Pitfall: Session-Scoped Fixtures That Mutate Shared State

Problem: A session-scoped fixture returning a mutable object (e.g., a shared list or in-memory cache) causes test order-dependence and flaky failures when tests run in a different order or in parallel.

Recommended approach: Keep expensive connections session-scoped, but wrap actual test data in function-scoped fixtures using transactions or fresh instances (see Best Practice #1).

Pitfall: Over-Mocking Until the Test Verifies Nothing Real

Problem: Mocking every collaborator of the unit under test can produce a test that passes even when the real integration is broken, because it only asserts that mocks were called correctly.

Recommended approach: Reserve mocking for genuine boundaries (network calls, time, randomness, third-party APIs); prefer real objects or lightweight fakes for in-process collaborators. See Unit Testing Principles.

Pitfall: Ignoring Flaky Tests Instead of Fixing Them

Problem: Re-running a flaky test until it passes ("just rerun CI") erodes trust in the suite and hides real concurrency or ordering bugs.

Recommended approach: Quarantine flaky tests with an explicit @pytest.mark.flaky (via pytest-rerunfailures) and a tracked ticket, not a silent skip; treat flakiness as a bug in the test or the system under test.

Testing Strategies

  • Unit tests: Pure functions and isolated business logic, no I/O, run in milliseconds, the majority of the suite.
  • Integration tests: Real database (via test containers or transactional fixtures), real HTTP client against an in-process app - see FastAPI Best Practices and Django Best Practices for framework specifics.
  • Property-based tests: Applied selectively to algorithms with clear invariants (parsers, serializers, mathematical functions, idempotency checks).
  • Contract/characterization tests: For legacy code being refactored without a specification, capture current behavior as a baseline before making changes.

Recommended test stack for Python projects in 2026:

  • Runner: pytest
  • Coverage: pytest-cov (wraps coverage.py)
  • Property-based: hypothesis
  • Parallelism: pytest-xdist
  • Mocking: pytest-mock
  • Fixtures/factories for ORM objects: factory_boy or polyfactory (Pydantic-native)

Performance, Security & Scalability Considerations

  • Suite runtime as a first-class metric: A test suite that takes 20+ minutes discourages running it locally before push; invest in pytest-xdist and fixture scope tuning before adding more tests unconditionally.
  • Database test isolation at scale: Prefer transactional rollback per test over TRUNCATE/recreate between tests - it is dramatically faster at scale and still guarantees isolation.
  • Secrets in test fixtures: Never commit real credentials in fixture data; use clearly fake values (test@example.com, sktest...) and load anything sensitive from environment variables scoped to CI.
  • Hypothesis and untrusted input: Property-based tests are an effective way to fuzz-test input validation and parsing code before it reaches production, surfacing injection-adjacent edge cases early.

Edge Cases & Advanced Usage

  • Testing asyncio code: Use pytest-asyncio with @pytest.mark.asyncio (or asyncio_mode = "auto" in configuration) - see Python Concurrency for TaskGroup-specific testing patterns.
  • Snapshot testing: For complex output structures (large JSON responses, rendered templates), syrupy provides snapshot assertions that are easier to review as diffs than hand-written equality checks.
  • Testing time-dependent code: Use freezegun or time-machine to freeze or travel time deterministically rather than injecting sleep() calls into tests.
  • Mutation testing: Tools like mutmut intentionally introduce small bugs and check whether the test suite catches them - a stronger signal of test quality than coverage percentage alone.
  • Doctest integration: pytest --doctest-modules runs docstring examples as tests; useful for keeping documentation examples honest, but should not replace a real test suite.

When Not to Use / Alternatives

Dimensionpytestunittest (stdlib)Winner
BoilerplateMinimal (plain functions)Verbose (class + self.assert*)pytest
Fixture compositionExcellent (DI-style)Limited (setUp/tearDown)pytest
Zero-dependency requirementRequires external packageBuilt into stdlibunittest
Plugin ecosystemEnormousSmallpytest

Use plain unittest only in constrained environments where adding third-party dependencies is genuinely prohibited (rare in 2026); pytest is fully capable of running unittest.TestCase-based tests, so migration is incremental, not all-or-nothing.

Glossary

  • Fixture: A pytest function providing setup, teardown, and/or injected values to test functions via argument name matching.
  • Fixture scope: The lifetime of a fixture instance (function, class, module, package, session).
  • Parametrization: Running the same test function against multiple input/expected-output pairs.
  • Property-based testing: A testing style where inputs are generated from a strategy and a general property is asserted, rather than hand-picking examples.
  • Branch coverage: A coverage metric tracking whether both branches of each conditional were executed, stricter than line coverage.
  • Flaky test: A test whose outcome is nondeterministic across identical runs, typically due to shared state, timing, or ordering assumptions.