diff --git a/doc/extdev/testing.rst b/doc/extdev/testing.rst index db03855bc4..c0b00de4d7 100644 --- a/doc/extdev/testing.rst +++ b/doc/extdev/testing.rst @@ -6,27 +6,177 @@ Testing API .. versionadded:: 1.6 -Utility functions and pytest fixtures for testing -are provided in :py:mod:`!sphinx.testing`. -If you are a developer of Sphinx extensions, -you can write unit tests with pytest_. +Sphinx provides utility functions and pytest_ fixtures to make it easier +to write test suites that build multiple Sphinx documentation projects. .. _pytest: https://docs.pytest.org/en/latest/ -``pytest`` configuration -------------------------- +Using the ``pytest`` plugin +--------------------------- -To use pytest fixtures that are provided by ``sphinx.testing``, -add the ``'sphinx.testing.fixtures'`` plugin -to your test modules or :file:`conftest.py` files as follows: +To use pytest helpers that are provided by ``sphinx.testing``, add the +``'sphinx.testing.fixtures'`` plugin to your :file:`conftest.py` file +as follows: .. code-block:: python pytest_plugins = ('sphinx.testing.fixtures',) -Usage ------ +Markers +~~~~~~~ -If you want to know more detailed usage, -please refer to :file:`tests/conftest.py` and other :file:`test_*.py` files -under the :file:`tests/` directory. +.. py:decorator:: pytest.mark.sphinx(buildername='html', *, testroot='root', ...) + + Arguments to initialize the Sphinx test application. + + :param str buildername: Builder to use. + :kwparam str testroot: Test root directory to use. + :kwparam srcdir: Source directory to use. + :kwparam dict confoverrides: Configuration values to override. + :kwparam bool freshenv: Whether to refresh the environment. + :kwparam bool warningiserror: Treat warnings as errors. + :kwparam tags: List of tags to set. + :kwparam int verbosity: Verbosity level. + :kwparam int parallel: Number of parallel processes. + :kwparam builddir: Build directory. + :kwparam docutils_conf: Docutils configuration. + + .. code-block:: python + + @pytest.mark.sphinx('html', testroot='something') + def test_html_output(app: SphinxTestApp) -> None: + app.build() + # ... test something about the HTML output ... + +.. py:decorator:: pytest.mark.test_params(*, shared_result=...) + + Parameters associated with a test. + + :kwparam str shared_result: + A key that allows sharing the build result, status, and warnings + between tests that use the same key. + + When multiple tests with a module are marked with the same + ``shared_result`` value, they will share the same build result as + well as status and warning buffers. This allows related tests to + avoid redundant rebuilds and reuse the same build context. + + .. attention:: *shared_result* and *srcdir* are mutually incompatible. + + .. code-block:: python + + @pytest.mark.test_params(shared_result='html_build') + def test_html_title(app: SphinxTestApp) -> None: + app.build() + # ... test something about the HTML output ... + + @pytest.mark.test_params(shared_result='html_build') + def test_html_index(app: SphinxTestApp) -> None: + app.build() + # ... test something else about the HTML output ... + +Fixtures +~~~~~~~~ + +.. py:data:: sphinx.testing.fixtures.app + :type: SphinxTestApp + + :scope: function + + Provides a :class:`~sphinx.testing.util.SphinxTestApp` instance. + This is the most common way to get a Sphinx application for testing. + + The app can be configured by using the :deco:`pytest.mark.sphinx` marker. + + .. code-block:: python + + def test_something(app: SphinxTestApp) -> None: + app.build() + # ... test something about the built documentation ... + +.. py:function:: sphinx.testing.fixtures.make_app(*args: Any, **kwargs: Any) -> SphinxTestApp + + :scope: function + + Factory function that constructs a :class:`~sphinx.testing.util.SphinxTestApp` + instance for use in tests. This is the preferred way to create instances + of the :class:`~sphinx.application.Sphinx` object, as it handles clean-up. + The arguments are the same as those to ``SphinxTestApp``. + + .. code-block:: python + + def test_something(make_app: Callable[..., SphinxTestApp]) -> None: + app = make_app('html') + app.build() + # ... test something about the built documentation ... + +.. py:data:: sphinx.testing.fixtures.app_params + :type: tuple[Sequence[Any], Mapping[str, Any]] + + :scope: function + + The positional keyword arguments used to create the + :class:`~sphinx.testing.util.SphinxTestApp` for this test. + These are derived from the markers_ applied to the test function. + + Returns a namedtuple of ``(args, kwargs)``. + +.. py:data:: sphinx.testing.fixtures.rootdir + :type: pathlib.Path | None + + :scope: session + + Default is ``None``, meaning tests use empty temporary directories. + + Can be overridden in a project's :file:`conftest.py` to return a + :class:`~pathlib.Path` to a directory, containing multiple Sphinx + documentation sources under sub-directories prefixed with ``test-``. + + .. code-block:: python + + @pytest.fixture(scope='session') + def rootdir() -> Path: + return Path(__file__).resolve().parent / 'roots' + + .. code-block:: text + + tests/ + ├── conftest.py <-- defines rootdir fixture + ├── roots/ + │ ├── test-example1/ + │ │ ├── conf.py + │ │ └── index.rst + │ └── test-example2/ + │ ├── conf.py + │ └── index.rst + └── test_something.py + +.. py:data:: sphinx.testing.fixtures.sphinx_test_tempdir + :type: pathlib.Path + + :scope: session + + Base temporary directory :class:`~pathlib.Path` used for building + the test apps. + +Utilities +--------- + +.. autoclass:: sphinx.testing.util.SphinxTestApp + :members: + :show-inheritance: + + .. py:attribute:: extras + + A dictionary to store arbitrary data associated with this app. + +.. autoclass:: sphinx.testing.util.SphinxTestAppWrapperForSkipBuilding + :members: + :show-inheritance: + +Examples +-------- + +For practical examples, refer to :file:`tests/conftest.py` +and other :file:`test_*.py` files under the :file:`tests/` directory +of the Sphinx source code.