diff --git a/CHANGES b/CHANGES index 33cc8ccf7..d2a797046 100644 --- a/CHANGES +++ b/CHANGES @@ -32,11 +32,15 @@ Incompatible changes * Ignore filenames without file extension given to ``Builder.build_specific()`` API directly +* #6230: The anchor of term in glossary directive is changed if it is consisted + by non-ASCII characters +* #4550: html: Centering tables by default using CSS Deprecated ---------- * ``sphinx.builders.latex.LaTeXBuilder.apply_transforms()`` +* ``sphinx.builders._epub_base.EpubBuilder.esc()`` * ``sphinx.directives.Acks`` * ``sphinx.directives.Author`` * ``sphinx.directives.Centered`` @@ -54,11 +58,24 @@ Deprecated * ``sphinx.directives.TabularColumns`` * ``sphinx.directives.TocTree`` * ``sphinx.directives.VersionChange`` +* ``sphinx.domains.python.PyClassmember`` +* ``sphinx.domains.python.PyModulelevel`` +* ``sphinx.domains.std.StandardDomain._resolve_citation_xref()`` +* ``sphinx.domains.std.StandardDomain.note_citations()`` +* ``sphinx.domains.std.StandardDomain.note_citation_refs()`` +* ``sphinx.domains.std.StandardDomain.note_labels()`` * ``sphinx.environment.NoUri`` +* ``sphinx.ext.apidoc.format_directive()`` +* ``sphinx.ext.apidoc.format_heading()`` * ``sphinx.ext.autodoc.importer.MockFinder`` * ``sphinx.ext.autodoc.importer.MockLoader`` * ``sphinx.ext.autodoc.importer.mock()`` * ``sphinx.ext.autosummary.autolink_role()`` +* ``sphinx.ext.imgmath.DOC_BODY`` +* ``sphinx.ext.imgmath.DOC_BODY_PREVIEW`` +* ``sphinx.ext.imgmath.DOC_HEAD`` +* ``sphinx.transforms.CitationReferences`` +* ``sphinx.transforms.SmartQuotesSkipper`` * ``sphinx.util.docfields.DocFieldTransformer.preprocess_fieldtypes()`` * ``sphinx.util.node.find_source_node()`` * ``sphinx.util.i18n.find_catalog()`` @@ -71,39 +88,78 @@ Features added -------------- * Add a helper class ``sphinx.transforms.post_transforms.SphinxPostTransform`` -* Add a helper method ``SphinxDirective.set_source_info()`` +* Add helper methods + + - ``PythonDomain.note_module()`` + - ``PythonDomain.note_object()`` + - ``SphinxDirective.set_source_info()`` + * #6180: Support ``--keep-going`` with BuildDoc setup command * ``math`` directive now supports ``:class:`` option +* #6310: imgmath: let :confval:`imgmath_use_preview` work also with the SVG + format for images rendering inline math * todo: ``todo`` directive now supports ``:name:`` option +* Enable override via environment of ``SPHINXOPTS`` and ``SPHINXBUILD`` Makefile + variables (refs: #6232, #6303) +* #6287: autodoc: Unable to document bound instance methods exported as module + functions +* #6289: autodoc: :confval:`autodoc_default_options` now supports + ``imported-members`` option +* #4777: autodoc: Support coroutine +* #6212 autosummary: Add :confval:`autosummary_imported_members` to display + imported members on autosummary +* #6271: ``make clean`` is catastrophically broken if building into '.' +* #4777: py domain: Add ``:async:`` option to :rst:dir:`py:function` directive +* py domain: Add new options to :rst:dir:`py:method` directive + + - ``:async:`` + - ``:classmethod:`` + - ``:property:`` + - ``:staticmethod:`` + +* rst domain: Add :rst:dir:`directive:option` directive to describe the option + for directive +* #6306: html: Add a label to search form for accessability purposes Bugs fixed ---------- +* #6230: Inappropriate node_id has been generated by glossary directive if term + is consisted by non-ASCII characters +* #6213: ifconfig: contents after headings are not shown +* commented term in glossary directive is wrongly recognized +* #6299: rst domain: rst:directive directive generates waste space +* #6331: man: invalid output when doctest follows rubric +* #6351: "Hyperlink target is not referenced" message is shown even if + referenced +* #6165: autodoc: ``tab_width`` setting of docutils has been ignored +* Generated Makefiles lack a final EOL (refs: #6232) + Testing -------- -Release 2.0.1 (in development) -============================== - -Dependencies ------------- - -Incompatible changes --------------------- - -Deprecated ----------- - -Features added --------------- +Release 2.0.1 (released Apr 08, 2019) +===================================== Bugs fixed ---------- * LaTeX: some system labels are not translated +* RemovedInSphinx30Warning is marked as pending +* deprecation warnings are not emitted -Testing --------- + - sphinx.application.CONFIG_FILENAME + - sphinx.builders.htmlhelp + - :confval:`viewcode_import` + +* #6208: C++, properly parse full xrefs that happen to have a short xref as prefix. +* #6220, #6225: napoleon: AttributeError is raised for raised section having + references +* #6245: circular import error on importing SerializingHTMLBuilder +* #6243: LaTeX: 'releasename' setting for latex_elements is ignored +* #6244: html: Search function is broken with 3rd party themes +* #6263: html: HTML5Translator crashed with invalid field node +* #6262: html theme: The style of field lists has changed in bizstyle theme Release 2.0.0 (released Mar 29, 2019) ===================================== diff --git a/doc/_templates/index.html b/doc/_templates/index.html index be174317d..c22eebbf9 100644 --- a/doc/_templates/index.html +++ b/doc/_templates/index.html @@ -97,6 +97,8 @@

{%trans%}A Japanese book about Sphinx has been published by O'Reilly: Sphinxをはじめよう / Learning Sphinx.{%endtrans%}

+

{%trans%}In 2019 the second edition of a German book about Sphinx was published: + Software-Dokumentation mit Sphinx.{%endtrans%}

diff --git a/doc/develop.rst b/doc/develop.rst index d061aae61..3828b709d 100644 --- a/doc/develop.rst +++ b/doc/develop.rst @@ -31,7 +31,8 @@ This is the current list of contributed extensions in that repository: - actdiag: embed activity diagrams by using actdiag_ - adadomain: an extension for Ada support (Sphinx 1.0 needed) - ansi: parse ANSI color sequences inside documents -- argdoc: automatically generate documentation for command-line arguments, descriptions, and help text +- argdoc: automatically generate documentation for command-line arguments, + descriptions and help text - astah: embed diagram by using astah - autoanysrc: Gather reST documentation from any source files - autorun: Execute code in a ``runblock`` directive @@ -64,7 +65,8 @@ This is the current list of contributed extensions in that repository: - imgur: embed Imgur images, albums, and metadata in documents - inlinesyntaxhighlight_: inline syntax highlighting - lassodomain: a domain for documenting Lasso_ source code -- libreoffice: an extension to include any drawing supported by LibreOffice (e.g. odg, vsd, ...) +- libreoffice: an extension to include any drawing supported by LibreOffice + (e.g. odg, vsd, ...) - lilypond: an extension inserting music scripts from Lilypond_ in PNG format - makedomain_: a domain for `GNU Make`_ - matlabdomain: document MATLAB_ code @@ -100,8 +102,8 @@ This is the current list of contributed extensions in that repository: - zopeext: provide an ``autointerface`` directive for using `Zope interfaces`_ -See the :doc:`extension tutorials <../development/tutorials/index>` on getting started with writing your -own extensions. +See the :doc:`extension tutorials <../development/tutorials/index>` on getting +started with writing your own extensions. .. _aafigure: https://launchpad.net/aafigure diff --git a/doc/extdev/appapi.rst b/doc/extdev/appapi.rst index 4cb8501be..18eea34e7 100644 --- a/doc/extdev/appapi.rst +++ b/doc/extdev/appapi.rst @@ -145,7 +145,7 @@ Sphinx core events ------------------ These events are known to the core. The arguments shown are given to the -registered event handlers. Use :meth:`.connect` in an extension's ``setup`` +registered event handlers. Use :meth:`.Sphinx.connect` in an extension's ``setup`` function (note that ``conf.py`` can also have a ``setup`` function) to connect handlers to the events. Example: diff --git a/doc/extdev/builderapi.rst b/doc/extdev/builderapi.rst index 2c2cf12e3..0ab7a30f4 100644 --- a/doc/extdev/builderapi.rst +++ b/doc/extdev/builderapi.rst @@ -38,3 +38,8 @@ Builder API .. automethod:: write_doc .. automethod:: finish + **Attributes** + + .. attribute:: events + + An :class:`.EventManager` object. diff --git a/doc/extdev/deprecated.rst b/doc/extdev/deprecated.rst index 99abc56eb..2ecb1e2e7 100644 --- a/doc/extdev/deprecated.rst +++ b/doc/extdev/deprecated.rst @@ -31,6 +31,11 @@ The following is a list of deprecated interfaces. - 4.0 - N/A + * - ``sphinx.builders._epub_base.EpubBuilder.esc()`` + - 2.1 + - 4.0 + - ``html.escape()`` + * - ``sphinx.directives.Acks`` - 2.1 - 4.0 @@ -116,10 +121,55 @@ The following is a list of deprecated interfaces. - 4.0 - ``sphinx.directives.other.VersionChange`` + * - ``sphinx.domains.python.PyClassmember`` + - 2.1 + - 4.0 + - ``sphinx.domains.python.PyAttribute``, + ``sphinx.domains.python.PyMethod``, + ``sphinx.domains.python.PyClassMethod``, + ``sphinx.domains.python.PyObject`` and + ``sphinx.domains.python.PyStaticMethod`` + + * - ``sphinx.domains.python.PyModulelevel`` + - 2.1 + - 4.0 + - ``sphinx.domains.python.PyFunction``, + ``sphinx.domains.python.PyObject`` and + ``sphinx.domains.python.PyVariable`` + + * - ``sphinx.domains.std.StandardDomain._resolve_citation_xref()`` + - 2.1 + - 4.0 + - ``sphinx.domains.citation.CitationDomain.resolve_xref()`` + + * - ``sphinx.domains.std.StandardDomain.note_citations()`` + - 2.1 + - 4.0 + - ``sphinx.domains.citation.CitationDomain.note_citation()`` + + * - ``sphinx.domains.std.StandardDomain.note_citation_refs()`` + - 2.1 + - 4.0 + - ``sphinx.domains.citation.CitationDomain.note_citation_reference()`` + + * - ``sphinx.domains.std.StandardDomain.note_labels()`` + - 2.1 + - 4.0 + - ``sphinx.domains.std.StandardDomain.process_doc()`` + * - ``sphinx.environment.NoUri`` - 2.1 - 4.0 - ``sphinx.errors.NoUri`` + * - ``sphinx.ext.apidoc.format_directive()`` + - 2.1 + - 4.0 + - N/A + + * - ``sphinx.ext.apidoc.format_heading()`` + - 2.1 + - 4.0 + - N/A * - ``sphinx.ext.autodoc.importer.MockFinder`` - 2.1 @@ -141,6 +191,31 @@ The following is a list of deprecated interfaces. - 4.0 - ``sphinx.ext.autosummary.AutoLink`` + * - ``sphinx.ext.imgmath.DOC_BODY`` + - 2.1 + - 4.0 + - N/A + + * - ``sphinx.ext.imgmath.DOC_BODY_PREVIEW`` + - 2.1 + - 4.0 + - N/A + + * - ``sphinx.ext.imgmath.DOC_HEAD`` + - 2.1 + - 4.0 + - N/A + + * - ``sphinx.transforms.CitationReferences`` + - 2.1 + - 4.0 + - ``sphinx.domains.citation.CitationReferenceTransform`` + + * - ``sphinx.transforms.SmartQuotesSkipper`` + - 2.1 + - 4.0 + - ``sphinx.domains.citation.CitationDefinitionTransform`` + * - ``sphinx.util.docfields.DocFieldTransformer.preprocess_fieldtypes()`` - 2.1 - 4.0 diff --git a/doc/extdev/envapi.rst b/doc/extdev/envapi.rst index 1dee6a576..d7ec23925 100644 --- a/doc/extdev/envapi.rst +++ b/doc/extdev/envapi.rst @@ -27,6 +27,10 @@ Build environment API Directory for storing pickled doctrees. + .. attribute:: events + + An :class:`.EventManager` object. + .. attribute:: found_docs A set of all existing docnames. diff --git a/doc/extdev/index.rst b/doc/extdev/index.rst index eac4ded40..c70ca37be 100644 --- a/doc/extdev/index.rst +++ b/doc/extdev/index.rst @@ -97,7 +97,8 @@ extension. These are: The config is available as ``app.config`` or ``env.config``. -To see an example of use of these objects, refer to :doc:`../development/tutorials/index`. +To see an example of use of these objects, refer to +:doc:`../development/tutorials/index`. .. _build-phases: diff --git a/doc/extdev/markupapi.rst b/doc/extdev/markupapi.rst index ffa08cae7..fc25c2327 100644 --- a/doc/extdev/markupapi.rst +++ b/doc/extdev/markupapi.rst @@ -147,5 +147,6 @@ return ``node.children`` from the Directive. .. seealso:: - `Creating directives `_ - HOWTO of the Docutils documentation + `Creating directives`_ HOWTO of the Docutils documentation + +.. _Creating directives: http://docutils.sourceforge.net/docs/howto/rst-directives.html diff --git a/doc/extdev/utils.rst b/doc/extdev/utils.rst index 2a94a34bb..e842f3032 100644 --- a/doc/extdev/utils.rst +++ b/doc/extdev/utils.rst @@ -29,3 +29,9 @@ components (e.g. :class:`.Config`, :class:`.BuildEnvironment` and so on) easily. .. autoclass:: sphinx.transforms.post_transforms.images.ImageConverter :members: + +Utility components +------------------ + +.. autoclass:: sphinx.events.EventManager + :members: diff --git a/doc/faq.rst b/doc/faq.rst index e7c23c131..cd5eb26c8 100644 --- a/doc/faq.rst +++ b/doc/faq.rst @@ -51,11 +51,11 @@ Using Sphinx with... -------------------- Read the Docs - https://readthedocs.org is a documentation hosting service based around + `Read the Docs `_ is a documentation hosting service based around Sphinx. They will host sphinx documentation, along with supporting a number of other features including version support, PDF generation, and more. The `Getting Started - `_ + `_ guide is a good place to start. Epydoc diff --git a/doc/templating.rst b/doc/templating.rst index b3a26c4b1..3790275f5 100644 --- a/doc/templating.rst +++ b/doc/templating.rst @@ -354,8 +354,8 @@ are in HTML form), these variables are also available: .. data:: body - A string containing the content of the page in HTML form as produced by the HTML builder, - before the theme is applied. + A string containing the content of the page in HTML form as produced by the + HTML builder, before the theme is applied. .. data:: display_toc @@ -382,8 +382,9 @@ are in HTML form), these variables are also available: .. data:: page_source_suffix - The suffix of the file that was rendered. Since we support a list of :confval:`source_suffix`, - this will allow you to properly link to the original source file. + The suffix of the file that was rendered. Since we support a list of + :confval:`source_suffix`, this will allow you to properly link to the + original source file. .. data:: parents diff --git a/doc/usage/extensions/autodoc.rst b/doc/usage/extensions/autodoc.rst index 6d7ba8272..0b6061e78 100644 --- a/doc/usage/extensions/autodoc.rst +++ b/doc/usage/extensions/autodoc.rst @@ -387,14 +387,17 @@ There are also config values that you can set: The supported options are ``'members'``, ``'member-order'``, ``'undoc-members'``, ``'private-members'``, ``'special-members'``, - ``'inherited-members'``, ``'show-inheritance'``, ``'ignore-module-all'`` and - ``'exclude-members'``. + ``'inherited-members'``, ``'show-inheritance'``, ``'ignore-module-all'``, + ``'imported-members'`` and ``'exclude-members'``. .. versionadded:: 1.8 .. versionchanged:: 2.0 Accepts ``True`` as a value. + .. versionchanged:: 2.1 + Added ``'imported-members'``. + .. confval:: autodoc_docstring_signature Functions imported from C modules cannot be introspected, and therefore the diff --git a/doc/usage/extensions/autosummary.rst b/doc/usage/extensions/autosummary.rst index 3d373908a..8e43f8a7b 100644 --- a/doc/usage/extensions/autosummary.rst +++ b/doc/usage/extensions/autosummary.rst @@ -165,6 +165,16 @@ also use these config values: :confval:`autodoc_mock_imports` for more details. It defaults to :confval:`autodoc_mock_imports`. + .. versionadded:: 2.0 + +.. confval:: autosummary_imported_members + + A boolean flag indicating whether to document classes and functions imported + in modules. Default is ``False`` + + .. versionadded:: 2.1 + + Customizing templates --------------------- diff --git a/doc/usage/extensions/ifconfig.rst b/doc/usage/extensions/ifconfig.rst index f64ca6c58..2bd9d0e3b 100644 --- a/doc/usage/extensions/ifconfig.rst +++ b/doc/usage/extensions/ifconfig.rst @@ -8,6 +8,11 @@ This extension is quite simple, and features only one directive: +.. warning:: + + This directive is designed to control only content of document. It could + not control sections, labels and so on. + .. rst:directive:: ifconfig Include content of the directive only if the Python expression given as an diff --git a/doc/usage/extensions/math.rst b/doc/usage/extensions/math.rst index 9e62c1425..75cafff6b 100644 --- a/doc/usage/extensions/math.rst +++ b/doc/usage/extensions/math.rst @@ -15,7 +15,8 @@ Math support for HTML outputs in Sphinx So mathbase extension is no longer needed. Since mathematical notation isn't natively supported by HTML in any way, Sphinx -gives a math support to HTML document with several extensions. +gives a math support to HTML document with several extensions. These use the +reStructuredText math :rst:dir:`directive ` and :rst:role:`role `. :mod:`sphinx.ext.imgmath` -- Render math as images -------------------------------------------------- @@ -29,13 +30,39 @@ This extension renders math via LaTeX and dvipng_ or dvisvgm_ into PNG or SVG images. This of course means that the computer where the docs are built must have both programs available. -There are various config values you can set to influence how the images are -built: +There are various configuration values you can set to influence how the images +are built: .. confval:: imgmath_image_format - The output image format. The default is ``'png'``. It should be either - ``'png'`` or ``'svg'``. + The output image format. The default is ``'png'``. It should be either + ``'png'`` or ``'svg'``. The image is produced by first executing ``latex`` + on the TeX mathematical mark-up then (depending on the requested format) + either `dvipng`_ or `dvisvgm`_. + +.. confval:: imgmath_use_preview + + ``dvipng`` and ``dvisvgm`` both have the ability to collect from LaTeX the + "depth" of the rendered math: an inline image should use this "depth" in a + ``vertical-align`` style to get correctly aligned with surrounding text. + + This mechanism requires the `LaTeX preview package`_ (available as + ``preview-latex-style`` on Ubuntu xenial). Therefore, the default for this + option is ``False`` but it is strongly recommended to set it to ``True``. + + .. versionchanged:: 2.1 + + This option can be used with the ``'svg'`` :confval:`imgmath_image_format`. + +.. confval:: imgmath_add_tooltips + + Default: ``True``. If false, do not add the LaTeX code as an "alt" attribute + for math images. + +.. confval:: imgmath_font_size + + The font size (in ``pt``) of the displayed math. The default value is + ``12``. It must be a positive integer. .. confval:: imgmath_latex @@ -53,20 +80,6 @@ built: This value should only contain the path to the latex executable, not further arguments; use :confval:`imgmath_latex_args` for that purpose. -.. confval:: imgmath_dvipng - - The command name with which to invoke ``dvipng``. The default is - ``'dvipng'``; you may need to set this to a full path if ``dvipng`` is not in - the executable search path. This option is only used when - ``imgmath_image_format`` is set to ``'png'``. - -.. confval:: imgmath_dvisvgm - - The command name with which to invoke ``dvisvgm``. The default is - ``'dvisvgm'``; you may need to set this to a full path if ``dvisvgm`` is not - in the executable search path. This option is only used when - ``imgmath_image_format`` is ``'svg'``. - .. confval:: imgmath_latex_args Additional arguments to give to latex, as a list. The default is an empty @@ -74,48 +87,43 @@ built: .. confval:: imgmath_latex_preamble - Additional LaTeX code to put into the preamble of the short LaTeX files that - are used to translate the math snippets. This is empty by default. Use it - e.g. to add more packages whose commands you want to use in the math. + Additional LaTeX code to put into the preamble of the LaTeX files used to + translate the math snippets. This is left empty by default. Use it + e.g. to add packages which modify the fonts used for math, such as + ``'\\usepackage{newtxsf}'`` for sans-serif fonts, or + ``'\\usepackage{fouriernc}'`` for serif fonts. Indeed, the default LaTeX + math fonts have rather thin glyphs which (in HTML output) often do not + match well with the font for text. + +.. confval:: imgmath_dvipng + + The command name to invoke ``dvipng``. The default is + ``'dvipng'``; you may need to set this to a full path if ``dvipng`` is not in + the executable search path. This option is only used when + ``imgmath_image_format`` is set to ``'png'``. .. confval:: imgmath_dvipng_args Additional arguments to give to dvipng, as a list. The default value is ``['-gamma', '1.5', '-D', '110', '-bg', 'Transparent']`` which makes the - image a bit darker and larger then it is by default, and produces PNGs with a + image a bit darker and larger then it is by default (this compensates + somewhat for the thinness of default LaTeX math fonts), and produces PNGs with a transparent background. This option is used only when ``imgmath_image_format`` is ``'png'``. +.. confval:: imgmath_dvisvgm + + The command name to invoke ``dvisvgm``. The default is + ``'dvisvgm'``; you may need to set this to a full path if ``dvisvgm`` is not + in the executable search path. This option is only used when + ``imgmath_image_format`` is ``'svg'``. + .. confval:: imgmath_dvisvgm_args - Additional arguments to give to dvisvgm, as a list. The default value is - ``['--no-fonts']``. This option is used only when ``imgmath_image_format`` - is ``'svg'``. - -.. confval:: imgmath_use_preview - - ``dvipng`` has the ability to determine the "depth" of the rendered text: for - example, when typesetting a fraction inline, the baseline of surrounding text - should not be flush with the bottom of the image, rather the image should - extend a bit below the baseline. This is what TeX calls "depth". When this - is enabled, the images put into the HTML document will get a - ``vertical-align`` style that correctly aligns the baselines. - - Unfortunately, this only works when the `preview-latex package`_ is - installed. Therefore, the default for this option is ``False``. - - Currently this option is only used when ``imgmath_image_format`` is - ``'png'``. - -.. confval:: imgmath_add_tooltips - - Default: ``True``. If false, do not add the LaTeX code as an "alt" attribute - for math images. - -.. confval:: imgmath_font_size - - The font size (in ``pt``) of the displayed math. The default value is - ``12``. It must be a positive integer. + Additional arguments to give to dvisvgm, as a list. The default value is + ``['--no-fonts']``, which means that ``dvisvgm`` will render glyphs as path + elements (cf the `dvisvgm FAQ`_). This option is used only when + ``imgmath_image_format`` is ``'svg'``. :mod:`sphinx.ext.mathjax` -- Render math via JavaScript @@ -131,7 +139,13 @@ MathJax_ is then loaded and transforms the LaTeX markup to readable math live in the browser. Because MathJax (and the necessary fonts) is very large, it is not included in -Sphinx. +Sphinx but is set to automatically include it from a third-party site. + +.. attention:: + + You should use the math :rst:dir:`directive ` and + :rst:role:`role `, not the native MathJax ``$$``, ``\(``, etc. + .. confval:: mathjax_path @@ -140,8 +154,9 @@ Sphinx. The default is the ``https://`` URL that loads the JS files from the `cdnjs`__ Content Delivery Network. See the `MathJax Getting Started - page`__ for details. If you want MathJax to be available offline, you have - to download it and set this value to a different path. + page`__ for details. If you want MathJax to be available offline or + without including resources from a third-party site, you have to + download it and set this value to a different path. __ https://cdnjs.com @@ -209,7 +224,8 @@ package jsMath_. It provides this config value: .. _dvipng: https://savannah.nongnu.org/projects/dvipng/ -.. _dvisvgm: http://dvisvgm.bplaced.net/ +.. _dvisvgm: https://dvisvgm.de/ +.. _dvisvgm FAQ: https://dvisvgm.de/FAQ .. _MathJax: https://www.mathjax.org/ .. _jsMath: http://www.math.union.edu/~dpvc/jsmath/ -.. _preview-latex package: https://www.gnu.org/software/auctex/preview-latex.html +.. _LaTeX preview package: https://www.gnu.org/software/auctex/preview-latex.html diff --git a/doc/usage/restructuredtext/domains.rst b/doc/usage/restructuredtext/domains.rst index d0da75d4d..8b06bf0e0 100644 --- a/doc/usage/restructuredtext/domains.rst +++ b/doc/usage/restructuredtext/domains.rst @@ -169,6 +169,13 @@ The following directives are provided for module and class contents: This information can (in any ``py`` directive) optionally be given in a structured form, see :ref:`info-field-lists`. + The ``async`` option can be given (with no value) to indicate the function is + an async method. + + .. versionchanged:: 2.1 + + ``:async:`` option added. + .. rst:directive:: .. py:data:: name Describes global data in a module, including both variables and values used @@ -216,6 +223,20 @@ The following directives are provided for module and class contents: described for ``function``. See also :ref:`signatures` and :ref:`info-field-lists`. + The ``async`` option can be given (with no value) to indicate the method is + an async method. + + The ``classmethod`` option and ``staticmethod`` option can be given (with + no value) to indicate the method is a class method (or a static method). + + The ``property`` option can be given (with no value) to indicate the method + is a property. + + .. versionchanged:: 2.1 + + ``:async:``, ``:classmethod:``, ``:property:`` and ``:staticmethod:`` + options added. + .. rst:directive:: .. py:staticmethod:: name(parameters) Like :rst:dir:`py:method`, but indicates that the method is a static method. @@ -1062,15 +1083,16 @@ These roles link to the given declaration types: .. admonition:: Note on References with Templates Parameters/Arguments - These roles follow the Sphinx :ref:`xref-syntax` rules. This means care must be - taken when referencing a (partial) template specialization, e.g. if the link looks like - this: ``:cpp:class:`MyClass```. + These roles follow the Sphinx :ref:`xref-syntax` rules. This means care must + be taken when referencing a (partial) template specialization, e.g. if the + link looks like this: ``:cpp:class:`MyClass```. This is interpreted as a link to ``int`` with a title of ``MyClass``. In this case, escape the opening angle bracket with a backslash, like this: ``:cpp:class:`MyClass\```. - When a custom title is not needed it may be useful to use the roles for inline expressions, - :rst:role:`cpp:expr` and :rst:role:`cpp:texpr`, where angle brackets do not need escaping. + When a custom title is not needed it may be useful to use the roles for + inline expressions, :rst:role:`cpp:expr` and :rst:role:`cpp:texpr`, where + angle brackets do not need escaping. Declarations without template parameters and template arguments ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1402,6 +1424,43 @@ The reStructuredText domain (name **rst**) provides the following directives: Bar description. +.. rst:directive:: .. rst:directive:option:: name + + Describes an option for reST directive. The *name* can be a single option + name or option name with arguments which separated with colon (``:``). + For example:: + + .. rst:directive:: toctree + + .. rst:directive:option:: caption: caption of ToC + + .. rst:directive:option:: glob + + will be rendered as: + + .. rst:directive:: toctree + :noindex: + + .. rst:directive:option:: caption: caption of ToC + + .. rst:directive:option:: glob + + .. rubric:: options + + .. rst:directive:option:: type + :type: description for the option of directive + + Describe the type of option value. + + For example:: + + .. rst:directive:: toctree + + .. rst:directive:option:: maxdepth + :type: integer or no value + + .. versionadded:: 2.1 + .. rst:directive:: .. rst:role:: name Describes a reST role. For example:: diff --git a/setup.cfg b/setup.cfg index 2db007339..c91a31879 100644 --- a/setup.cfg +++ b/setup.cfg @@ -55,6 +55,11 @@ strict_optional = False filterwarnings = all ignore::DeprecationWarning:docutils.io +markers = + sphinx + apidoc + setup_command + test_params [coverage:run] branch = True diff --git a/setup.py b/setup.py index 30f8625c8..91b3e12cc 100644 --- a/setup.py +++ b/setup.py @@ -38,6 +38,9 @@ extras_require = { ':sys_platform=="win32"': [ 'colorama>=0.3.5', ], + 'docs': [ + 'sphinxcontrib-websupport', + ], 'test': [ 'pytest', 'pytest-cov', diff --git a/sphinx/application.py b/sphinx/application.py index 516b7be58..e9b950c83 100644 --- a/sphinx/application.py +++ b/sphinx/application.py @@ -23,7 +23,6 @@ from docutils.parsers.rst import Directive, roles import sphinx from sphinx import package_dir, locale from sphinx.config import Config -from sphinx.config import CONFIG_FILENAME # NOQA # for compatibility (RemovedInSphinx30) from sphinx.deprecation import RemovedInSphinx40Warning from sphinx.environment import BuildEnvironment from sphinx.errors import ApplicationError, ConfigError, VersionRequirementError @@ -73,6 +72,7 @@ builtin_extensions = ( 'sphinx.config', 'sphinx.domains.c', 'sphinx.domains.changeset', + 'sphinx.domains.citation', 'sphinx.domains.cpp', 'sphinx.domains.javascript', 'sphinx.domains.math', @@ -182,7 +182,7 @@ class Sphinx: self.warningiserror = warningiserror logging.setup(self, self._status, self._warning) - self.events = EventManager() + self.events = EventManager(self) # keep last few messages for traceback # This will be filled by sphinx.util.logging.LastMessagesWriter @@ -249,7 +249,7 @@ class Sphinx: # now that we know all config values, collect them from conf.py self.config.init_values() - self.emit('config-inited', self.config) + self.events.emit('config-inited', self.config) # create the project self.project = Project(self.srcdir, self.config.source_suffix) @@ -319,7 +319,7 @@ class Sphinx: # type: () -> None self.builder.set_environment(self.env) self.builder.init() - self.emit('builder-inited') + self.events.emit('builder-inited') # ---- main "build" method ------------------------------------------------- @@ -360,10 +360,10 @@ class Sphinx: envfile = path.join(self.doctreedir, ENV_PICKLE_FILENAME) if path.isfile(envfile): os.unlink(envfile) - self.emit('build-finished', err) + self.events.emit('build-finished', err) raise else: - self.emit('build-finished', None) + self.events.emit('build-finished', None) self.builder.cleanup() # ---- general extensibility interface ------------------------------------- @@ -420,13 +420,7 @@ class Sphinx: Return the return values of all callbacks as a list. Do not emit core Sphinx events in extensions! """ - try: - logger.debug('[app] emitting event: %r%s', event, repr(args)[:100]) - except Exception: - # not every object likes to be repr()'d (think - # random stuff coming via autodoc) - pass - return self.events.emit(event, self, *args) + return self.events.emit(event, *args) def emit_firstresult(self, event, *args): # type: (str, Any) -> Any @@ -436,7 +430,7 @@ class Sphinx: .. versionadded:: 0.5 """ - return self.events.emit_firstresult(event, self, *args) + return self.events.emit_firstresult(event, *args) # registering addon parts diff --git a/sphinx/builders/__init__.py b/sphinx/builders/__init__.py index 1b29fa983..8eaa0e215 100644 --- a/sphinx/builders/__init__.py +++ b/sphinx/builders/__init__.py @@ -43,6 +43,7 @@ if False: from sphinx.application import Sphinx # NOQA from sphinx.config import Config # NOQA from sphinx.environment import BuildEnvironment # NOQA + from sphinx.events import EventManager # NOQA from sphinx.util.i18n import CatalogInfo # NOQA from sphinx.util.tags import Tags # NOQA @@ -93,6 +94,7 @@ class Builder: self.app = app # type: Sphinx self.env = None # type: BuildEnvironment + self.events = app.events # type: EventManager self.config = app.config # type: Config self.tags = app.tags # type: Tags self.tags.add(self.format) @@ -399,7 +401,7 @@ class Builder: added, changed, removed = self.env.get_outdated_files(updated) # allow user intervention as well - for docs in self.app.emit('env-get-outdated', self, added, changed, removed): + for docs in self.events.emit('env-get-outdated', self, added, changed, removed): changed.update(set(docs) & self.env.found_docs) # if files were added or removed, all documents with globbed toctrees @@ -416,13 +418,13 @@ class Builder: # clear all files no longer present for docname in removed: - self.app.emit('env-purge-doc', self.env, docname) + self.events.emit('env-purge-doc', self.env, docname) self.env.clear_doc(docname) # read all new and changed files docnames = sorted(added | changed) # allow changing and reordering the list of docs to read - self.app.emit('env-before-read-docs', self.env, docnames) + self.events.emit('env-before-read-docs', self.env, docnames) # check if we should do parallel or serial read if parallel_available and len(docnames) > 5 and self.app.parallel > 1: @@ -439,7 +441,7 @@ class Builder: raise SphinxError('master file %s not found' % self.env.doc2path(self.config.master_doc)) - for retval in self.app.emit('env-updated', self.env): + for retval in self.events.emit('env-updated', self.env): if retval is not None: docnames.extend(retval) @@ -453,7 +455,7 @@ class Builder: for docname in status_iterator(docnames, __('reading sources... '), "purple", len(docnames), self.app.verbosity): # remove all inventory entries for that file - self.app.emit('env-purge-doc', self.env, docname) + self.events.emit('env-purge-doc', self.env, docname) self.env.clear_doc(docname) self.read_doc(docname) @@ -461,7 +463,7 @@ class Builder: # type: (List[str], int) -> None # clear all outdated docs at once for docname in docnames: - self.app.emit('env-purge-doc', self.env, docname) + self.events.emit('env-purge-doc', self.env, docname) self.env.clear_doc(docname) def read_process(docs): diff --git a/sphinx/builders/_epub_base.py b/sphinx/builders/_epub_base.py index 90ab6c12d..140f2748d 100644 --- a/sphinx/builders/_epub_base.py +++ b/sphinx/builders/_epub_base.py @@ -8,6 +8,7 @@ :license: BSD, see LICENSE for details. """ +import html import os import re import warnings @@ -178,7 +179,9 @@ class EpubBuilder(StandaloneHTMLBuilder): def esc(self, name): # type: (str) -> str """Replace all characters not allowed in text an attribute values.""" - # Like cgi.escape, but also replace apostrophe + warnings.warn( + '%s.esc() is deprecated. Use html.escape() instead.' % self.__class__.__name__, + RemovedInSphinx40Warning) name = name.replace('&', '&') name = name.replace('<', '<') name = name.replace('>', '>') @@ -201,8 +204,8 @@ class EpubBuilder(StandaloneHTMLBuilder): if (self.toctree_template % level) in classes: result.append({ 'level': level, - 'refuri': self.esc(refuri), - 'text': ssp(self.esc(doctree.astext())) + 'refuri': html.escape(refuri), + 'text': ssp(html.escape(doctree.astext())) }) break elif isinstance(doctree, nodes.Element): @@ -241,21 +244,21 @@ class EpubBuilder(StandaloneHTMLBuilder): """ refnodes.insert(0, { 'level': 1, - 'refuri': self.esc(self.config.master_doc + self.out_suffix), - 'text': ssp(self.esc( + 'refuri': html.escape(self.config.master_doc + self.out_suffix), + 'text': ssp(html.escape( self.env.titles[self.config.master_doc].astext())) }) for file, text in reversed(self.config.epub_pre_files): refnodes.insert(0, { 'level': 1, - 'refuri': self.esc(file), - 'text': ssp(self.esc(text)) + 'refuri': html.escape(file), + 'text': ssp(html.escape(text)) }) for file, text in self.config.epub_post_files: refnodes.append({ 'level': 1, - 'refuri': self.esc(file), - 'text': ssp(self.esc(text)) + 'refuri': html.escape(file), + 'text': ssp(html.escape(text)) }) def fix_fragment(self, prefix, fragment): @@ -511,15 +514,15 @@ class EpubBuilder(StandaloneHTMLBuilder): file properly escaped. """ metadata = {} # type: Dict[str, Any] - metadata['title'] = self.esc(self.config.epub_title) - metadata['author'] = self.esc(self.config.epub_author) - metadata['uid'] = self.esc(self.config.epub_uid) - metadata['lang'] = self.esc(self.config.epub_language) - metadata['publisher'] = self.esc(self.config.epub_publisher) - metadata['copyright'] = self.esc(self.config.epub_copyright) - metadata['scheme'] = self.esc(self.config.epub_scheme) - metadata['id'] = self.esc(self.config.epub_identifier) - metadata['date'] = self.esc(format_date("%Y-%m-%d")) + metadata['title'] = html.escape(self.config.epub_title) + metadata['author'] = html.escape(self.config.epub_author) + metadata['uid'] = html.escape(self.config.epub_uid) + metadata['lang'] = html.escape(self.config.epub_language) + metadata['publisher'] = html.escape(self.config.epub_publisher) + metadata['copyright'] = html.escape(self.config.epub_copyright) + metadata['scheme'] = html.escape(self.config.epub_scheme) + metadata['id'] = html.escape(self.config.epub_identifier) + metadata['date'] = html.escape(format_date("%Y-%m-%d")) metadata['manifest_items'] = [] metadata['spines'] = [] metadata['guides'] = [] @@ -566,9 +569,9 @@ class EpubBuilder(StandaloneHTMLBuilder): type='epub', subtype='unknown_project_files') continue filename = filename.replace(os.sep, '/') - item = ManifestItem(self.esc(filename), - self.esc(self.make_id(filename)), - self.esc(self.media_types[ext])) + item = ManifestItem(html.escape(filename), + html.escape(self.make_id(filename)), + html.escape(self.media_types[ext])) metadata['manifest_items'].append(item) self.files.append(filename) @@ -579,21 +582,21 @@ class EpubBuilder(StandaloneHTMLBuilder): continue if refnode['refuri'] in self.ignored_files: continue - spine = Spine(self.esc(self.make_id(refnode['refuri'])), True) + spine = Spine(html.escape(self.make_id(refnode['refuri'])), True) metadata['spines'].append(spine) spinefiles.add(refnode['refuri']) for info in self.domain_indices: - spine = Spine(self.esc(self.make_id(info[0] + self.out_suffix)), True) + spine = Spine(html.escape(self.make_id(info[0] + self.out_suffix)), True) metadata['spines'].append(spine) spinefiles.add(info[0] + self.out_suffix) if self.use_index: - spine = Spine(self.esc(self.make_id('genindex' + self.out_suffix)), True) + spine = Spine(html.escape(self.make_id('genindex' + self.out_suffix)), True) metadata['spines'].append(spine) spinefiles.add('genindex' + self.out_suffix) # add auto generated files for name in self.files: if name not in spinefiles and name.endswith(self.out_suffix): - spine = Spine(self.esc(self.make_id(name)), False) + spine = Spine(html.escape(self.make_id(name)), False) metadata['spines'].append(spine) # add the optional cover @@ -601,18 +604,18 @@ class EpubBuilder(StandaloneHTMLBuilder): if self.config.epub_cover: image, html_tmpl = self.config.epub_cover image = image.replace(os.sep, '/') - metadata['cover'] = self.esc(self.make_id(image)) + metadata['cover'] = html.escape(self.make_id(image)) if html_tmpl: - spine = Spine(self.esc(self.make_id(self.coverpage_name)), True) + spine = Spine(html.escape(self.make_id(self.coverpage_name)), True) metadata['spines'].insert(0, spine) if self.coverpage_name not in self.files: ext = path.splitext(self.coverpage_name)[-1] self.files.append(self.coverpage_name) - item = ManifestItem(self.esc(self.coverpage_name), - self.esc(self.make_id(self.coverpage_name)), - self.esc(self.media_types[ext])) + item = ManifestItem(html.escape(self.coverpage_name), + html.escape(self.make_id(self.coverpage_name)), + html.escape(self.media_types[ext])) metadata['manifest_items'].append(item) - ctx = {'image': self.esc(image), 'title': self.config.project} + ctx = {'image': html.escape(image), 'title': self.config.project} self.handle_page( path.splitext(self.coverpage_name)[0], ctx, html_tmpl) spinefiles.add(self.coverpage_name) @@ -628,17 +631,17 @@ class EpubBuilder(StandaloneHTMLBuilder): auto_add_cover = False if type == 'toc': auto_add_toc = False - metadata['guides'].append(Guide(self.esc(type), - self.esc(title), - self.esc(uri))) + metadata['guides'].append(Guide(html.escape(type), + html.escape(title), + html.escape(uri))) if auto_add_cover and html_tmpl: metadata['guides'].append(Guide('cover', self.guide_titles['cover'], - self.esc(self.coverpage_name))) + html.escape(self.coverpage_name))) if auto_add_toc and self.refnodes: metadata['guides'].append(Guide('toc', self.guide_titles['toc'], - self.esc(self.refnodes[0]['refuri']))) + html.escape(self.refnodes[0]['refuri']))) # write the project file copy_asset_file(path.join(self.template_dir, 'content.opf_t'), @@ -707,7 +710,7 @@ class EpubBuilder(StandaloneHTMLBuilder): """ metadata = {} # type: Dict[str, Any] metadata['uid'] = self.config.epub_uid - metadata['title'] = self.esc(self.config.epub_title) + metadata['title'] = html.escape(self.config.epub_title) metadata['level'] = level metadata['navpoints'] = navpoints return metadata diff --git a/sphinx/builders/dirhtml.py b/sphinx/builders/dirhtml.py index 7ab6fad98..d5d61c273 100644 --- a/sphinx/builders/dirhtml.py +++ b/sphinx/builders/dirhtml.py @@ -11,6 +11,7 @@ from os import path from sphinx.builders.html import StandaloneHTMLBuilder +from sphinx.deprecation import RemovedInSphinx40Warning, deprecated_alias from sphinx.util import logging from sphinx.util.osutil import SEP, os_path @@ -55,6 +56,14 @@ class DirectoryHTMLBuilder(StandaloneHTMLBuilder): self.globalcontext['no_search_suffix'] = True +# for compatibility +deprecated_alias('sphinx.builders.html', + { + 'DirectoryHTMLBuilder': DirectoryHTMLBuilder, + }, + RemovedInSphinx40Warning) + + def setup(app): # type: (Sphinx) -> Dict[str, Any] app.setup_extension('sphinx.builders.html') diff --git a/sphinx/builders/epub3.py b/sphinx/builders/epub3.py index 3116cd493..9b3f58d7a 100644 --- a/sphinx/builders/epub3.py +++ b/sphinx/builders/epub3.py @@ -9,6 +9,7 @@ :license: BSD, see LICENSE for details. """ +import html import warnings from collections import namedtuple from os import path @@ -98,12 +99,12 @@ class Epub3Builder(_epub_base.EpubBuilder): writing_mode = self.config.epub_writing_mode metadata = super().content_metadata() - metadata['description'] = self.esc(self.config.epub_description) - metadata['contributor'] = self.esc(self.config.epub_contributor) + metadata['description'] = html.escape(self.config.epub_description) + metadata['contributor'] = html.escape(self.config.epub_contributor) metadata['page_progression_direction'] = PAGE_PROGRESSION_DIRECTIONS.get(writing_mode) metadata['ibook_scroll_axis'] = IBOOK_SCROLL_AXIS.get(writing_mode) - metadata['date'] = self.esc(format_date("%Y-%m-%dT%H:%M:%SZ")) - metadata['version'] = self.esc(self.config.version) + metadata['date'] = html.escape(format_date("%Y-%m-%dT%H:%M:%SZ")) + metadata['version'] = html.escape(self.config.version) metadata['epub_version'] = self.config.epub_version return metadata @@ -166,8 +167,8 @@ class Epub3Builder(_epub_base.EpubBuilder): properly escaped. """ metadata = {} # type: Dict - metadata['lang'] = self.esc(self.config.epub_language) - metadata['toc_locale'] = self.esc(self.guide_titles['toc']) + metadata['lang'] = html.escape(self.config.epub_language) + metadata['toc_locale'] = html.escape(self.guide_titles['toc']) metadata['navlist'] = navlist return metadata diff --git a/sphinx/builders/html.py b/sphinx/builders/html.py index 5621f9a75..81c64d445 100644 --- a/sphinx/builders/html.py +++ b/sphinx/builders/html.py @@ -24,7 +24,7 @@ from docutils.utils import relative_path from sphinx import package_dir, __display_version__ from sphinx.builders import Builder -from sphinx.deprecation import RemovedInSphinx40Warning, deprecated_alias +from sphinx.deprecation import RemovedInSphinx40Warning from sphinx.environment.adapters.asset import ImageAdapter from sphinx.environment.adapters.indexentries import IndexEntries from sphinx.environment.adapters.toctree import TocTree @@ -653,7 +653,7 @@ class StandaloneHTMLBuilder(Builder): def gen_additional_pages(self): # type: () -> None # pages from extensions - for pagelist in self.app.emit('html-collect-pages'): + for pagelist in self.events.emit('html-collect-pages'): for pagename, context, template in pagelist: self.handle_page(pagename, context, template) @@ -1187,23 +1187,9 @@ def validate_math_renderer(app): # for compatibility -from sphinx.builders.dirhtml import DirectoryHTMLBuilder # NOQA -from sphinx.builders.singlehtml import SingleFileHTMLBuilder # NOQA -from sphinxcontrib.serializinghtml import ( # NOQA - LAST_BUILD_FILENAME, JSONHTMLBuilder, PickleHTMLBuilder, SerializingHTMLBuilder -) - -deprecated_alias('sphinx.builders.html', - { - 'LAST_BUILD_FILENAME': LAST_BUILD_FILENAME, - 'DirectoryHTMLBuilder': DirectoryHTMLBuilder, - 'JSONHTMLBuilder': JSONHTMLBuilder, - 'PickleHTMLBuilder': PickleHTMLBuilder, - 'SerializingHTMLBuilder': SerializingHTMLBuilder, - 'SingleFileHTMLBuilder': SingleFileHTMLBuilder, - 'WebHTMLBuilder': PickleHTMLBuilder, - }, - RemovedInSphinx40Warning) +import sphinx.builders.dirhtml # NOQA +import sphinx.builders.singlehtml # NOQA +import sphinxcontrib.serializinghtml # NOQA def setup(app): diff --git a/sphinx/builders/htmlhelp.py b/sphinx/builders/htmlhelp.py index 2e7e8f083..be365ef7e 100644 --- a/sphinx/builders/htmlhelp.py +++ b/sphinx/builders/htmlhelp.py @@ -24,7 +24,7 @@ if False: from sphinx.application import Sphinx # NOQA -deprecated_alias('sphinx.builders.devhelp', +deprecated_alias('sphinx.builders.htmlhelp', { 'chm_locales': chm_locales, 'chm_htmlescape': chm_htmlescape, diff --git a/sphinx/builders/latex/__init__.py b/sphinx/builders/latex/__init__.py index e6467601d..973e7c67d 100644 --- a/sphinx/builders/latex/__init__.py +++ b/sphinx/builders/latex/__init__.py @@ -208,7 +208,7 @@ class LaTeXBuilder(Builder): self.context['indexname'] = _('Index') if self.config.release: # Show the release label only if release value exists - self.context['releasename'] = _('Release') + self.context.setdefault('releasename', _('Release')) def init_babel(self): # type: () -> None diff --git a/sphinx/builders/latex/transforms.py b/sphinx/builders/latex/transforms.py index 746446fbc..6381780ae 100644 --- a/sphinx/builders/latex/transforms.py +++ b/sphinx/builders/latex/transforms.py @@ -16,6 +16,7 @@ from sphinx import addnodes from sphinx.builders.latex.nodes import ( captioned_literal_block, footnotemark, footnotetext, math_reference, thebibliography ) +from sphinx.domains.citation import CitationDomain from sphinx.transforms import SphinxTransform from sphinx.transforms.post_transforms import SphinxPostTransform from sphinx.util.nodes import NodeMatcher @@ -545,10 +546,10 @@ class CitationReferenceTransform(SphinxPostTransform): def run(self, **kwargs): # type: (Any) -> None - matcher = NodeMatcher(addnodes.pending_xref, refdomain='std', reftype='citation') - citations = self.env.get_domain('std').data['citations'] + domain = cast(CitationDomain, self.env.get_domain('citation')) + matcher = NodeMatcher(addnodes.pending_xref, refdomain='citation', reftype='ref') for node in self.document.traverse(matcher): # type: addnodes.pending_xref - docname, labelid, _ = citations.get(node['reftarget'], ('', '', 0)) + docname, labelid, _ = domain.citations.get(node['reftarget'], ('', '', 0)) if docname: citation_ref = nodes.citation_reference('', '', *node.children, docname=docname, refname=labelid) diff --git a/sphinx/builders/singlehtml.py b/sphinx/builders/singlehtml.py index 1ee5a37b1..068d1c1c2 100644 --- a/sphinx/builders/singlehtml.py +++ b/sphinx/builders/singlehtml.py @@ -13,6 +13,7 @@ from os import path from docutils import nodes from sphinx.builders.html import StandaloneHTMLBuilder +from sphinx.deprecation import RemovedInSphinx40Warning, deprecated_alias from sphinx.environment.adapters.toctree import TocTree from sphinx.locale import __ from sphinx.util import logging @@ -201,6 +202,14 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder): self.handle_page('opensearch', {}, 'opensearch.xml', outfilename=fn) +# for compatibility +deprecated_alias('sphinx.builders.html', + { + 'SingleFileHTMLBuilder': SingleFileHTMLBuilder, + }, + RemovedInSphinx40Warning) + + def setup(app): # type: (Sphinx) -> Dict[str, Any] app.setup_extension('sphinx.builders.html') diff --git a/sphinx/cmd/make_mode.py b/sphinx/cmd/make_mode.py index 82a88933d..e87aa02fc 100644 --- a/sphinx/cmd/make_mode.py +++ b/sphinx/cmd/make_mode.py @@ -72,11 +72,19 @@ class Make: def build_clean(self): # type: () -> int + srcdir = path.abspath(self.srcdir) + builddir = path.abspath(self.builddir) if not path.exists(self.builddir): return 0 elif not path.isdir(self.builddir): print("Error: %r is not a directory!" % self.builddir) return 1 + elif srcdir == builddir: + print("Error: %r is same as source directory!" % self.builddir) + return 1 + elif path.commonpath([srcdir, builddir]) == builddir: + print("Error: %r directory contains source directory!" % self.builddir) + return 1 print("Removing everything under %r..." % self.builddir) for item in os.listdir(self.builddir): rmtree(self.builddir_join(item)) diff --git a/sphinx/directives/__init__.py b/sphinx/directives/__init__.py index fd7bec586..40f838c48 100644 --- a/sphinx/directives/__init__.py +++ b/sphinx/directives/__init__.py @@ -166,7 +166,7 @@ class ObjectDescription(SphinxDirective): node['objtype'] = node['desctype'] = self.objtype node['noindex'] = noindex = ('noindex' in self.options) - self.names = [] # type: List[str] + self.names = [] # type: List[Any] signatures = self.get_signatures() for i, sig in enumerate(signatures): # add a signature node for each signature in the current unit diff --git a/sphinx/domains/citation.py b/sphinx/domains/citation.py new file mode 100644 index 000000000..2bb49def9 --- /dev/null +++ b/sphinx/domains/citation.py @@ -0,0 +1,167 @@ +""" + sphinx.domains.citation + ~~~~~~~~~~~~~~~~~~~~~~~ + + The citation domain. + + :copyright: Copyright 2007-2019 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from typing import cast + +from docutils import nodes + +from sphinx import addnodes +from sphinx.domains import Domain +from sphinx.locale import __ +from sphinx.transforms import SphinxTransform +from sphinx.util import logging +from sphinx.util.nodes import copy_source_info, make_refnode + +if False: + # For type annotation + from typing import Any, Dict, List, Set, Tuple, Union # NOQA + from sphinx.application import Sphinx # NOQA + from sphinx.builders import Builder # NOQA + from sphinx.environment import BuildEnvironment # NOQA + +logger = logging.getLogger(__name__) + + +class CitationDomain(Domain): + """Domain for citations.""" + + name = 'citation' + label = 'citation' + + dangling_warnings = { + 'ref': 'citation not found: %(target)s', + } + + @property + def citations(self): + # type: () -> Dict[str, Tuple[str, str, int]] + return self.data.setdefault('citations', {}) + + @property + def citation_refs(self): + # type: () -> Dict[str, Set[str]] + return self.data.setdefault('citation_refs', {}) + + def clear_doc(self, docname): + # type: (str) -> None + for key, (fn, _l, lineno) in list(self.citations.items()): + if fn == docname: + del self.citations[key] + for key, docnames in list(self.citation_refs.items()): + if docnames == {docname}: + del self.citation_refs[key] + elif docname in docnames: + docnames.remove(docname) + + def merge_domaindata(self, docnames, otherdata): + # type: (List[str], Dict) -> None + # XXX duplicates? + for key, data in otherdata['citations'].items(): + if data[0] in docnames: + self.citations[key] = data + for key, data in otherdata['citation_refs'].items(): + citation_refs = self.citation_refs.setdefault(key, set()) + for docname in data: + if docname in docnames: + citation_refs.add(docname) + + def note_citation(self, node): + # type: (nodes.citation) -> None + label = node[0].astext() + if label in self.citations: + path = self.env.doc2path(self.citations[label][0]) + logger.warning(__('duplicate citation %s, other instance in %s'), label, path, + location=node, type='ref', subtype='citation') + self.citations[label] = (node['docname'], node['ids'][0], node.line) + + def note_citation_reference(self, node): + # type: (addnodes.pending_xref) -> None + docnames = self.citation_refs.setdefault(node['reftarget'], set()) + docnames.add(self.env.docname) + + def check_consistency(self): + # type: () -> None + for name, (docname, labelid, lineno) in self.citations.items(): + if name not in self.citation_refs: + logger.warning(__('Citation [%s] is not referenced.'), name, + type='ref', subtype='citation', location=(docname, lineno)) + + def resolve_xref(self, env, fromdocname, builder, typ, target, node, contnode): + # type: (BuildEnvironment, str, Builder, str, str, addnodes.pending_xref, nodes.Element) -> nodes.Element # NOQA + docname, labelid, lineno = self.citations.get(target, ('', '', 0)) + if not docname: + return None + + return make_refnode(builder, fromdocname, docname, + labelid, contnode) + + def resolve_any_xref(self, env, fromdocname, builder, target, node, contnode): + # type: (BuildEnvironment, str, Builder, str, addnodes.pending_xref, nodes.Element) -> List[Tuple[str, nodes.Element]] # NOQA + refnode = self.resolve_xref(env, fromdocname, builder, 'ref', target, node, contnode) + if refnode is None: + return [] + else: + return [('ref', refnode)] + + +class CitationDefinitionTransform(SphinxTransform): + """Mark citation definition labels as not smartquoted.""" + default_priority = 619 + + def apply(self, **kwargs): + # type: (Any) -> None + domain = cast(CitationDomain, self.env.get_domain('citation')) + for node in self.document.traverse(nodes.citation): + # register citation node to domain + node['docname'] = self.env.docname + domain.note_citation(node) + + # mark citation labels as not smartquoted + label = cast(nodes.label, node[0]) + label['support_smartquotes'] = False + + +class CitationReferenceTransform(SphinxTransform): + """ + Replace citation references by pending_xref nodes before the default + docutils transform tries to resolve them. + """ + default_priority = 619 + + def apply(self, **kwargs): + # type: (Any) -> None + domain = cast(CitationDomain, self.env.get_domain('citation')) + for node in self.document.traverse(nodes.citation_reference): + target = node.astext() + ref = addnodes.pending_xref(target, refdomain='citation', reftype='ref', + reftarget=target, refwarn=True, + support_smartquotes=False, + ids=node["ids"], + classes=node.get('classes', [])) + ref += nodes.inline(target, '[%s]' % target) + copy_source_info(node, ref) + node.replace_self(ref) + + # register reference node to domain + domain.note_citation_reference(ref) + + +def setup(app): + # type: (Sphinx) -> Dict[str, Any] + app.add_domain(CitationDomain) + app.add_transform(CitationDefinitionTransform) + app.add_transform(CitationReferenceTransform) + + return { + 'version': 'builtin', + 'env_version': 1, + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/sphinx/domains/cpp.py b/sphinx/domains/cpp.py index 6760328d4..a9f4b1c24 100644 --- a/sphinx/domains/cpp.py +++ b/sphinx/domains/cpp.py @@ -6391,6 +6391,7 @@ class DefinitionParser: # if there are '()' left, just skip them self.skip_ws() self.skip_string('()') + self.assert_end() templatePrefix = self._check_template_consistency(name, templatePrefix, fullSpecShorthand=True) res1 = ASTNamespace(name, templatePrefix) @@ -6403,6 +6404,7 @@ class DefinitionParser: # if there are '()' left, just skip them self.skip_ws() self.skip_string('()') + self.assert_end() return res2, False except DefinitionError as e2: errs = [] @@ -7145,7 +7147,6 @@ class CPPDomain(Domain): parser = DefinitionParser(target, warner, env.config) try: ast, isShorthand = parser.parse_xref_object() - parser.assert_end() except DefinitionError as e: def findWarning(e): # as arg to stop flake8 from complaining if typ != 'any' and typ != 'func': @@ -7154,7 +7155,6 @@ class CPPDomain(Domain): parser2 = DefinitionParser(target[:-2], warner, env.config) try: parser2.parse_xref_object() - parser2.assert_end() except DefinitionError as e2: return target[:-2], e2 # strange, that we don't get the error now, use the original diff --git a/sphinx/domains/python.py b/sphinx/domains/python.py index 2203ee6e3..4bfaaf848 100644 --- a/sphinx/domains/python.py +++ b/sphinx/domains/python.py @@ -9,11 +9,14 @@ """ import re +import warnings +from typing import cast from docutils import nodes from docutils.parsers.rst import directives from sphinx import addnodes +from sphinx.deprecation import RemovedInSphinx40Warning from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType, Index, IndexEntry from sphinx.locale import _, __ @@ -309,14 +312,13 @@ class PyObject(ObjectDescription): return fullname, prefix def get_index_text(self, modname, name): - # type: (str, str) -> str + # type: (str, Tuple[str, str]) -> str """Return the text for the index entry of the object.""" raise NotImplementedError('must be implemented in subclasses') def add_target_and_index(self, name_cls, sig, signode): - # type: (str, str, addnodes.desc_signature) -> None - modname = self.options.get( - 'module', self.env.ref_context.get('py:module')) + # type: (Tuple[str, str], str, addnodes.desc_signature) -> None + modname = self.options.get('module', self.env.ref_context.get('py:module')) fullname = (modname and modname + '.' or '') + name_cls[0] # note target if fullname not in self.state.document.ids: @@ -324,15 +326,9 @@ class PyObject(ObjectDescription): signode['ids'].append(fullname) signode['first'] = (not self.names) self.state.document.note_explicit_target(signode) - objects = self.env.domaindata['py']['objects'] - if fullname in objects: - self.state_machine.reporter.warning( - 'duplicate object description of %s, ' % fullname + - 'other instance in ' + - self.env.doc2path(objects[fullname][0]) + - ', use :noindex: for one of them', - line=self.lineno) - objects[fullname] = (self.env.docname, self.objtype) + + domain = cast(PythonDomain, self.env.get_domain('py')) + domain.note_object(fullname, self.objtype) indextext = self.get_index_text(modname, name_cls) if indextext: @@ -405,12 +401,19 @@ class PyModulelevel(PyObject): Description of an object on module level (functions, data). """ + def run(self): + # type: () -> List[nodes.Node] + warnings.warn('PyClassmember is deprecated.', + RemovedInSphinx40Warning) + + return super().run() + def needs_arglist(self): # type: () -> bool return self.objtype == 'function' def get_index_text(self, modname, name_cls): - # type: (str, str) -> str + # type: (str, Tuple[str, str]) -> str if self.objtype == 'function': if not modname: return _('%s() (built-in function)') % name_cls[0] @@ -423,6 +426,46 @@ class PyModulelevel(PyObject): return '' +class PyFunction(PyObject): + """Description of a function.""" + + option_spec = PyObject.option_spec.copy() + option_spec.update({ + 'async': directives.flag, + }) + + def get_signature_prefix(self, sig): + # type: (str) -> str + if 'async' in self.options: + return 'async ' + else: + return '' + + def needs_arglist(self): + # type: () -> bool + return True + + def get_index_text(self, modname, name_cls): + # type: (str, Tuple[str, str]) -> str + name, cls = name_cls + if modname: + return _('%s() (in module %s)') % (name, modname) + else: + return _('%s() (built-in function)') % name + + +class PyVariable(PyObject): + """Description of a variable.""" + + def get_index_text(self, modname, name_cls): + # type: (str, Tuple[str, str]) -> str + name, cls = name_cls + if modname: + return _('%s (in module %s)') % (name, modname) + else: + return _('%s (built-in variable)') % name + + class PyClasslike(PyObject): """ Description of a class-like object (classes, interfaces, exceptions). @@ -435,7 +478,7 @@ class PyClasslike(PyObject): return self.objtype + ' ' def get_index_text(self, modname, name_cls): - # type: (str, str) -> str + # type: (str, Tuple[str, str]) -> str if self.objtype == 'class': if not modname: return _('%s (built-in class)') % name_cls[0] @@ -451,6 +494,13 @@ class PyClassmember(PyObject): Description of a class member (methods, attributes). """ + def run(self): + # type: () -> List[nodes.Node] + warnings.warn('PyClassmember is deprecated.', + RemovedInSphinx40Warning) + + return super().run() + def needs_arglist(self): # type: () -> bool return self.objtype.endswith('method') @@ -464,7 +514,7 @@ class PyClassmember(PyObject): return '' def get_index_text(self, modname, name_cls): - # type: (str, str) -> str + # type: (str, Tuple[str, str]) -> str name, cls = name_cls add_modules = self.env.config.add_module_names if self.objtype == 'method': @@ -521,6 +571,109 @@ class PyClassmember(PyObject): return '' +class PyMethod(PyObject): + """Description of a method.""" + + option_spec = PyObject.option_spec.copy() + option_spec.update({ + 'async': directives.flag, + 'classmethod': directives.flag, + 'property': directives.flag, + 'staticmethod': directives.flag, + }) + + def needs_arglist(self): + # type: () -> bool + if 'property' in self.options: + return False + else: + return True + + def get_signature_prefix(self, sig): + # type: (str) -> str + prefix = [] + if 'async' in self.options: + prefix.append('async') + if 'classmethod' in self.options: + prefix.append('classmethod') + if 'property' in self.options: + prefix.append('property') + if 'staticmethod' in self.options: + prefix.append('static') + + if prefix: + return ' '.join(prefix) + ' ' + else: + return '' + + def get_index_text(self, modname, name_cls): + # type: (str, Tuple[str, str]) -> str + name, cls = name_cls + try: + clsname, methname = name.rsplit('.', 1) + if modname and self.env.config.add_module_names: + clsname = '.'.join([modname, clsname]) + except ValueError: + if modname: + return _('%s() (in module %s)') % (name, modname) + else: + return '%s()' % name + + if 'classmethod' in self.options: + return _('%s() (%s class method)') % (methname, clsname) + elif 'property' in self.options: + return _('%s() (%s property)') % (methname, clsname) + elif 'staticmethod' in self.options: + return _('%s() (%s static method)') % (methname, clsname) + else: + return _('%s() (%s method)') % (methname, clsname) + + +class PyClassMethod(PyMethod): + """Description of a classmethod.""" + + option_spec = PyObject.option_spec.copy() + + def run(self): + # type: () -> List[nodes.Node] + self.name = 'py:method' + self.options['classmethod'] = True + + return super().run() + + +class PyStaticMethod(PyMethod): + """Description of a staticmethod.""" + + option_spec = PyObject.option_spec.copy() + + def run(self): + # type: () -> List[nodes.Node] + self.name = 'py:method' + self.options['staticmethod'] = True + + return super().run() + + +class PyAttribute(PyObject): + """Description of an attribute.""" + + def get_index_text(self, modname, name_cls): + # type: (str, Tuple[str, str]) -> str + name, cls = name_cls + try: + clsname, attrname = name.rsplit('.', 1) + if modname and self.env.config.add_module_names: + clsname = '.'.join([modname, clsname]) + except ValueError: + if modname: + return _('%s (in module %s)') % (name, modname) + else: + return name + + return _('%s (%s attribute)') % (attrname, clsname) + + class PyDecoratorMixin: """ Mixin for decorator directives. @@ -575,18 +728,20 @@ class PyModule(SphinxDirective): def run(self): # type: () -> List[nodes.Node] + domain = cast(PythonDomain, self.env.get_domain('py')) + modname = self.arguments[0].strip() noindex = 'noindex' in self.options self.env.ref_context['py:module'] = modname ret = [] # type: List[nodes.Node] if not noindex: - self.env.domaindata['py']['modules'][modname] = (self.env.docname, - self.options.get('synopsis', ''), - self.options.get('platform', ''), - 'deprecated' in self.options) - # make a duplicate entry in 'objects' to facilitate searching for - # the module in PythonDomain.find_obj() - self.env.domaindata['py']['objects'][modname] = (self.env.docname, 'module') + # note module to the domain + domain.note_module(modname, + self.options.get('synopsis', ''), + self.options.get('platform', ''), + 'deprecated' in self.options) + domain.note_object(modname, 'module') + targetnode = nodes.target('', '', ids=['module-' + modname], ismod=True) self.state.document.note_explicit_target(targetnode) @@ -737,14 +892,14 @@ class PythonDomain(Domain): } # type: Dict[str, ObjType] directives = { - 'function': PyModulelevel, - 'data': PyModulelevel, + 'function': PyFunction, + 'data': PyVariable, 'class': PyClasslike, 'exception': PyClasslike, - 'method': PyClassmember, - 'classmethod': PyClassmember, - 'staticmethod': PyClassmember, - 'attribute': PyClassmember, + 'method': PyMethod, + 'classmethod': PyClassMethod, + 'staticmethod': PyStaticMethod, + 'attribute': PyAttribute, 'module': PyModule, 'currentmodule': PyCurrentModule, 'decorator': PyDecoratorFunction, @@ -769,24 +924,55 @@ class PythonDomain(Domain): PythonModuleIndex, ] + @property + def objects(self): + # type: () -> Dict[str, Tuple[str, str]] + return self.data.setdefault('objects', {}) # fullname -> docname, objtype + + def note_object(self, name, objtype, location=None): + # type: (str, str, Any) -> None + """Note a python object for cross reference. + + .. versionadded:: 2.1 + """ + if name in self.objects: + docname = self.objects[name][0] + logger.warning(__('duplicate object description of %s, ' + 'other instance in %s, use :noindex: for one of them'), + name, docname, location=location) + self.objects[name] = (self.env.docname, objtype) + + @property + def modules(self): + # type: () -> Dict[str, Tuple[str, str, str, bool]] + return self.data.setdefault('modules', {}) # modname -> docname, synopsis, platform, deprecated # NOQA + + def note_module(self, name, synopsis, platform, deprecated): + # type: (str, str, str, bool) -> None + """Note a python module for cross reference. + + .. versionadded:: 2.1 + """ + self.modules[name] = (self.env.docname, synopsis, platform, deprecated) + def clear_doc(self, docname): # type: (str) -> None - for fullname, (fn, _l) in list(self.data['objects'].items()): + for fullname, (fn, _l) in list(self.objects.items()): if fn == docname: - del self.data['objects'][fullname] - for modname, (fn, _x, _x, _x) in list(self.data['modules'].items()): + del self.objects[fullname] + for modname, (fn, _x, _x, _y) in list(self.modules.items()): if fn == docname: - del self.data['modules'][modname] + del self.modules[modname] def merge_domaindata(self, docnames, otherdata): # type: (List[str], Dict) -> None # XXX check duplicates? for fullname, (fn, objtype) in otherdata['objects'].items(): if fn in docnames: - self.data['objects'][fullname] = (fn, objtype) + self.objects[fullname] = (fn, objtype) for modname, data in otherdata['modules'].items(): if data[0] in docnames: - self.data['modules'][modname] = data + self.modules[modname] = data def find_obj(self, env, modname, classname, name, type, searchmode=0): # type: (BuildEnvironment, str, str, str, str, int) -> List[Tuple[str, Any]] @@ -800,7 +986,6 @@ class PythonDomain(Domain): if not name: return [] - objects = self.data['objects'] matches = [] # type: List[Tuple[str, Any]] newname = None @@ -812,44 +997,44 @@ class PythonDomain(Domain): if objtypes is not None: if modname and classname: fullname = modname + '.' + classname + '.' + name - if fullname in objects and objects[fullname][1] in objtypes: + if fullname in self.objects and self.objects[fullname][1] in objtypes: newname = fullname if not newname: - if modname and modname + '.' + name in objects and \ - objects[modname + '.' + name][1] in objtypes: + if modname and modname + '.' + name in self.objects and \ + self.objects[modname + '.' + name][1] in objtypes: newname = modname + '.' + name - elif name in objects and objects[name][1] in objtypes: + elif name in self.objects and self.objects[name][1] in objtypes: newname = name else: # "fuzzy" searching mode searchname = '.' + name - matches = [(oname, objects[oname]) for oname in objects + matches = [(oname, self.objects[oname]) for oname in self.objects if oname.endswith(searchname) and - objects[oname][1] in objtypes] + self.objects[oname][1] in objtypes] else: # NOTE: searching for exact match, object type is not considered - if name in objects: + if name in self.objects: newname = name elif type == 'mod': # only exact matches allowed for modules return [] - elif classname and classname + '.' + name in objects: + elif classname and classname + '.' + name in self.objects: newname = classname + '.' + name - elif modname and modname + '.' + name in objects: + elif modname and modname + '.' + name in self.objects: newname = modname + '.' + name elif modname and classname and \ - modname + '.' + classname + '.' + name in objects: + modname + '.' + classname + '.' + name in self.objects: newname = modname + '.' + classname + '.' + name # special case: builtin exceptions have module "exceptions" set elif type == 'exc' and '.' not in name and \ - 'exceptions.' + name in objects: + 'exceptions.' + name in self.objects: newname = 'exceptions.' + name # special case: object methods elif type in ('func', 'meth') and '.' not in name and \ - 'object.' + name in objects: + 'object.' + name in self.objects: newname = 'object.' + name if newname is not None: - matches.append((newname, objects[newname])) + matches.append((newname, self.objects[newname])) return matches def resolve_xref(self, env, fromdocname, builder, @@ -896,7 +1081,7 @@ class PythonDomain(Domain): def _make_module_refnode(self, builder, fromdocname, name, contnode): # type: (Builder, str, str, nodes.Node) -> nodes.Element # get additional info for modules - docname, synopsis, platform, deprecated = self.data['modules'][name] + docname, synopsis, platform, deprecated = self.modules[name] title = name if synopsis: title += ': ' + synopsis @@ -909,9 +1094,9 @@ class PythonDomain(Domain): def get_objects(self): # type: () -> Iterator[Tuple[str, str, str, str, str, int]] - for modname, info in self.data['modules'].items(): + for modname, info in self.modules.items(): yield (modname, modname, 'module', info[0], 'module-' + modname, 0) - for refname, (docname, type) in self.data['objects'].items(): + for refname, (docname, type) in self.objects.items(): if type != 'module': # modules are already handled yield (refname, refname, type, docname, refname, 1) diff --git a/sphinx/domains/rst.py b/sphinx/domains/rst.py index 716b50105..f054abf28 100644 --- a/sphinx/domains/rst.py +++ b/sphinx/domains/rst.py @@ -9,12 +9,16 @@ """ import re +from typing import cast + +from docutils.parsers.rst import directives from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.roles import XRefRole +from sphinx.util import logging from sphinx.util.nodes import make_refnode if False: @@ -26,6 +30,8 @@ if False: from sphinx.environment import BuildEnvironment # NOQA +logger = logging.getLogger(__name__) + dir_sig_re = re.compile(r'\.\. (.+?)::(.*)$') @@ -43,14 +49,9 @@ class ReSTMarkup(ObjectDescription): signode['first'] = (not self.names) self.state.document.note_explicit_target(signode) - objects = self.env.domaindata['rst']['objects'] - key = (self.objtype, name) - if key in objects: - self.state_machine.reporter.warning( - 'duplicate description of %s %s, ' % (self.objtype, name) + - 'other instance in ' + self.env.doc2path(objects[key]), - line=self.lineno) - objects[key] = self.env.docname + domain = cast(ReSTDomain, self.env.get_domain('rst')) + domain.note_object(self.objtype, name, location=(self.env.docname, self.lineno)) + indextext = self.get_index_text(self.objtype, name) if indextext: self.indexnode['entries'].append(('single', indextext, @@ -58,10 +59,6 @@ class ReSTMarkup(ObjectDescription): def get_index_text(self, objectname, name): # type: (str, str) -> str - if self.objtype == 'directive': - return _('%s (directive)') % name - elif self.objtype == 'role': - return _('%s (role)') % name return '' @@ -80,7 +77,10 @@ def parse_directive(d): if not m: return (dir, '') parsed_dir, parsed_args = m.groups() - return (parsed_dir.strip(), ' ' + parsed_args.strip()) + if parsed_args.strip(): + return (parsed_dir.strip(), ' ' + parsed_args.strip()) + else: + return (parsed_dir.strip(), '') class ReSTDirective(ReSTMarkup): @@ -96,6 +96,78 @@ class ReSTDirective(ReSTMarkup): signode += addnodes.desc_addname(args, args) return name + def get_index_text(self, objectname, name): + # type: (str, str) -> str + return _('%s (directive)') % name + + def before_content(self): + # type: () -> None + if self.names: + directives = self.env.ref_context.setdefault('rst:directives', []) + directives.append(self.names[0]) + + def after_content(self): + # type: () -> None + directives = self.env.ref_context.setdefault('rst:directives', []) + if directives: + directives.pop() + + +class ReSTDirectiveOption(ReSTMarkup): + """ + Description of an option for reST directive. + """ + option_spec = ReSTMarkup.option_spec.copy() + option_spec.update({ + 'type': directives.unchanged, + }) + + def handle_signature(self, sig, signode): + # type: (str, addnodes.desc_signature) -> str + try: + name, argument = re.split(r'\s*:\s+', sig.strip(), 1) + except ValueError: + name, argument = sig, None + + signode += addnodes.desc_name(':%s:' % name, ':%s:' % name) + if argument: + signode += addnodes.desc_annotation(' ' + argument, ' ' + argument) + if self.options.get('type'): + text = ' (%s)' % self.options['type'] + signode += addnodes.desc_annotation(text, text) + return name + + def add_target_and_index(self, name, sig, signode): + # type: (str, str, addnodes.desc_signature) -> None + targetname = '-'.join([self.objtype, self.current_directive, name]) + if targetname not in self.state.document.ids: + signode['names'].append(targetname) + signode['ids'].append(targetname) + signode['first'] = (not self.names) + self.state.document.note_explicit_target(signode) + + domain = cast(ReSTDomain, self.env.get_domain('rst')) + domain.note_object(self.objtype, name, location=(self.env.docname, self.lineno)) + + if self.current_directive: + key = name[0].upper() + pair = [_('%s (directive)') % self.current_directive, + _(':%s: (directive option)') % name] + self.indexnode['entries'].append(('pair', '; '.join(pair), targetname, '', key)) + else: + key = name[0].upper() + text = _(':%s: (directive option)') % name + self.indexnode['entries'].append(('single', text, targetname, '', key)) + + @property + def current_directive(self): + # type: () -> str + directives = self.env.ref_context.get('rst:directives') + if directives: + return directives[-1] + else: + return '' + class ReSTRole(ReSTMarkup): """ @@ -106,6 +178,10 @@ class ReSTRole(ReSTMarkup): signode += addnodes.desc_name(':%s:' % sig, ':%s:' % sig) return sig + def get_index_text(self, objectname, name): + # type: (str, str) -> str + return _('%s (role)') % name + class ReSTDomain(Domain): """ReStructuredText domain.""" @@ -113,11 +189,13 @@ class ReSTDomain(Domain): label = 'reStructuredText' object_types = { - 'directive': ObjType(_('directive'), 'dir'), - 'role': ObjType(_('role'), 'role'), + 'directive': ObjType(_('directive'), 'dir'), + 'directive:option': ObjType(_('directive-option'), 'dir'), + 'role': ObjType(_('role'), 'role'), } directives = { 'directive': ReSTDirective, + 'directive:option': ReSTDirectiveOption, 'role': ReSTRole, } roles = { @@ -126,42 +204,54 @@ class ReSTDomain(Domain): } initial_data = { 'objects': {}, # fullname -> docname, objtype - } # type: Dict[str, Dict[str, Tuple[str, ObjType]]] + } # type: Dict[str, Dict[Tuple[str, str], str]] + + @property + def objects(self): + # type: () -> Dict[Tuple[str, str], str] + return self.data.setdefault('objects', {}) # (objtype, fullname) -> docname + + def note_object(self, objtype, name, location=None): + # type: (str, str, Any) -> None + if (objtype, name) in self.objects: + docname = self.objects[objtype, name] + logger.warning(__('duplicate description of %s %s, other instance in %s') % + (objtype, name, docname), location=location) + + self.objects[objtype, name] = self.env.docname def clear_doc(self, docname): # type: (str) -> None - for (typ, name), doc in list(self.data['objects'].items()): + for (typ, name), doc in list(self.objects.items()): if doc == docname: - del self.data['objects'][typ, name] + del self.objects[typ, name] def merge_domaindata(self, docnames, otherdata): # type: (List[str], Dict) -> None # XXX check duplicates for (typ, name), doc in otherdata['objects'].items(): if doc in docnames: - self.data['objects'][typ, name] = doc + self.objects[typ, name] = doc def resolve_xref(self, env, fromdocname, builder, typ, target, node, contnode): # type: (BuildEnvironment, str, Builder, str, str, addnodes.pending_xref, nodes.Element) -> nodes.Element # NOQA - objects = self.data['objects'] objtypes = self.objtypes_for_role(typ) for objtype in objtypes: - if (objtype, target) in objects: - return make_refnode(builder, fromdocname, - objects[objtype, target], + todocname = self.objects.get((objtype, target)) + if todocname: + return make_refnode(builder, fromdocname, todocname, objtype + '-' + target, contnode, target + ' ' + objtype) return None def resolve_any_xref(self, env, fromdocname, builder, target, node, contnode): # type: (BuildEnvironment, str, Builder, str, addnodes.pending_xref, nodes.Element) -> List[Tuple[str, nodes.Element]] # NOQA - objects = self.data['objects'] results = [] # type: List[Tuple[str, nodes.Element]] for objtype in self.object_types: - if (objtype, target) in self.data['objects']: + todocname = self.objects.get((objtype, target)) + if todocname: results.append(('rst:' + self.role_for_objtype(objtype), - make_refnode(builder, fromdocname, - objects[objtype, target], + make_refnode(builder, fromdocname, todocname, objtype + '-' + target, contnode, target + ' ' + objtype))) return results diff --git a/sphinx/domains/std.py b/sphinx/domains/std.py index e73c660e6..b42925f89 100644 --- a/sphinx/domains/std.py +++ b/sphinx/domains/std.py @@ -10,6 +10,7 @@ import re import unicodedata +import warnings from copy import copy from typing import cast @@ -18,9 +19,9 @@ from docutils.parsers.rst import directives from docutils.statemachine import StringList from sphinx import addnodes +from sphinx.deprecation import RemovedInSphinx40Warning from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType -from sphinx.errors import NoUri from sphinx.locale import _, __ from sphinx.roles import XRefRole from sphinx.util import ws_re, logging, docname_join @@ -255,6 +256,9 @@ def make_glossary_term(env, textnodes, index_key, source, lineno, new_id=None): termtext = term.astext() if new_id is None: new_id = nodes.make_id('term-' + termtext) + if new_id == 'term': + # the term is not good for node_id. Generate it by sequence number instead. + new_id = 'term-' + str(len(gloss_entries)) if new_id in gloss_entries: new_id = 'term-' + str(len(gloss_entries)) gloss_entries.add(new_id) @@ -300,6 +304,7 @@ class Glossary(SphinxDirective): # first, collect single entries entries = [] # type: List[Tuple[List[Tuple[str, str, int]], StringList]] in_definition = True + in_comment = False was_empty = True messages = [] # type: List[nodes.Node] for line, (source, lineno) in zip(self.content, self.content.items): @@ -313,27 +318,33 @@ class Glossary(SphinxDirective): if line and not line[0].isspace(): # enable comments if line.startswith('.. '): + in_comment = True continue + else: + in_comment = False + # first term of definition if in_definition: if not was_empty: - messages.append(self.state.reporter.system_message( - 2, 'glossary term must be preceded by empty line', + messages.append(self.state.reporter.warning( + _('glossary term must be preceded by empty line'), source=source, line=lineno)) entries.append(([(line, source, lineno)], StringList())) in_definition = False # second term and following else: if was_empty: - messages.append(self.state.reporter.system_message( - 2, 'glossary terms must not be separated by empty ' - 'lines', source=source, line=lineno)) + messages.append(self.state.reporter.warning( + _('glossary terms must not be separated by empty lines'), + source=source, line=lineno)) if entries: entries[-1][0].append((line, source, lineno)) else: - messages.append(self.state.reporter.system_message( - 2, 'glossary seems to be misformatted, check ' - 'indentation', source=source, line=lineno)) + messages.append(self.state.reporter.warning( + _('glossary seems to be misformatted, check indentation'), + source=source, line=lineno)) + elif in_comment: + pass else: if not in_definition: # first line of definition, determines indentation @@ -342,9 +353,9 @@ class Glossary(SphinxDirective): if entries: entries[-1][1].append(line[indent_len:], source, lineno) else: - messages.append(self.state.reporter.system_message( - 2, 'glossary seems to be misformatted, check ' - 'indentation', source=source, line=lineno)) + messages.append(self.state.reporter.warning( + _('glossary seems to be misformatted, check indentation'), + source=source, line=lineno)) was_empty = False # now, parse all the entries into a big definition list @@ -494,8 +505,6 @@ class StandardDomain(Domain): initial_data = { 'progoptions': {}, # (program, name) -> docname, labelid 'objects': {}, # (type, name) -> docname, labelid - 'citations': {}, # citation_name -> docname, labelid, lineno - 'citation_refs': {}, # citation_name -> list of docnames 'labels': { # labelname -> docname, labelid, sectionname 'genindex': ('genindex', '', _('Index')), 'modindex': ('py-modindex', '', _('Module Index')), @@ -516,7 +525,6 @@ class StandardDomain(Domain): 'keyword': 'unknown keyword: %(target)s', 'doc': 'unknown document: %(target)s', 'option': 'unknown option: %(target)s', - 'citation': 'citation not found: %(target)s', } enumerable_nodes = { # node_class -> (figtype, title_getter) @@ -534,81 +542,60 @@ class StandardDomain(Domain): for node, settings in env.app.registry.enumerable_nodes.items(): self.enumerable_nodes[node] = settings + @property + def objects(self): + # type: () -> Dict[Tuple[str, str], Tuple[str, str]] + return self.data.setdefault('objects', {}) # (objtype, name) -> docname, labelid + + @property + def progoptions(self): + # type: () -> Dict[Tuple[str, str], Tuple[str, str]] + return self.data.setdefault('progoptions', {}) # (program, name) -> docname, labelid + + @property + def labels(self): + # type: () -> Dict[str, Tuple[str, str, str]] + return self.data.setdefault('labels', {}) # labelname -> docname, labelid, sectionname + + @property + def anonlabels(self): + # type: () -> Dict[str, Tuple[str, str]] + return self.data.setdefault('anonlabels', {}) # labelname -> docname, labelid + def clear_doc(self, docname): # type: (str) -> None - for key, (fn, _l) in list(self.data['progoptions'].items()): + key = None # type: Any + for key, (fn, _l) in list(self.progoptions.items()): if fn == docname: - del self.data['progoptions'][key] - for key, (fn, _l) in list(self.data['objects'].items()): + del self.progoptions[key] + for key, (fn, _l) in list(self.objects.items()): if fn == docname: - del self.data['objects'][key] - for key, (fn, _l, lineno) in list(self.data['citations'].items()): + del self.objects[key] + for key, (fn, _l, _l) in list(self.labels.items()): if fn == docname: - del self.data['citations'][key] - for key, docnames in list(self.data['citation_refs'].items()): - if docnames == [docname]: - del self.data['citation_refs'][key] - elif docname in docnames: - docnames.remove(docname) - for key, (fn, _l, _l) in list(self.data['labels'].items()): + del self.labels[key] + for key, (fn, _l) in list(self.anonlabels.items()): if fn == docname: - del self.data['labels'][key] - for key, (fn, _l) in list(self.data['anonlabels'].items()): - if fn == docname: - del self.data['anonlabels'][key] + del self.anonlabels[key] def merge_domaindata(self, docnames, otherdata): # type: (List[str], Dict) -> None # XXX duplicates? for key, data in otherdata['progoptions'].items(): if data[0] in docnames: - self.data['progoptions'][key] = data + self.progoptions[key] = data for key, data in otherdata['objects'].items(): if data[0] in docnames: - self.data['objects'][key] = data - for key, data in otherdata['citations'].items(): - if data[0] in docnames: - self.data['citations'][key] = data - for key, data in otherdata['citation_refs'].items(): - citation_refs = self.data['citation_refs'].setdefault(key, []) - for docname in data: - if docname in docnames: - citation_refs.append(docname) + self.objects[key] = data for key, data in otherdata['labels'].items(): if data[0] in docnames: - self.data['labels'][key] = data + self.labels[key] = data for key, data in otherdata['anonlabels'].items(): if data[0] in docnames: - self.data['anonlabels'][key] = data + self.anonlabels[key] = data def process_doc(self, env, docname, document): # type: (BuildEnvironment, str, nodes.document) -> None - self.note_citations(env, docname, document) - self.note_citation_refs(env, docname, document) - self.note_labels(env, docname, document) - - def note_citations(self, env, docname, document): - # type: (BuildEnvironment, str, nodes.document) -> None - for node in document.traverse(nodes.citation): - node['docname'] = docname - label = cast(nodes.label, node[0]).astext() - if label in self.data['citations']: - path = env.doc2path(self.data['citations'][label][0]) - logger.warning(__('duplicate citation %s, other instance in %s'), label, path, - location=node, type='ref', subtype='citation') - self.data['citations'][label] = (docname, node['ids'][0], node.line) - - def note_citation_refs(self, env, docname, document): - # type: (BuildEnvironment, str, nodes.document) -> None - for node in document.traverse(addnodes.pending_xref): - if node['refdomain'] == 'std' and node['reftype'] == 'citation': - label = node['reftarget'] - citation_refs = self.data['citation_refs'].setdefault(label, []) - citation_refs.append(docname) - - def note_labels(self, env, docname, document): - # type: (BuildEnvironment, str, nodes.document) -> None - labels, anonlabels = self.data['labels'], self.data['anonlabels'] for name, explicit in document.nametypes.items(): if not explicit: continue @@ -626,11 +613,11 @@ class StandardDomain(Domain): # ignore footnote labels, labels automatically generated from a # link and object descriptions continue - if name in labels: + if name in self.labels: logger.warning(__('duplicate label %s, other instance in %s'), - name, env.doc2path(labels[name][0]), + name, env.doc2path(self.labels[name][0]), location=node) - anonlabels[name] = docname, labelid + self.anonlabels[name] = docname, labelid if node.tagname in ('section', 'rubric'): title = cast(nodes.title, node[0]) sectname = clean_astext(title) @@ -647,23 +634,15 @@ class StandardDomain(Domain): else: # anonymous-only labels continue - labels[name] = docname, labelid, sectname + self.labels[name] = docname, labelid, sectname def add_object(self, objtype, name, docname, labelid): # type: (str, str, str, str) -> None - self.data['objects'][objtype, name] = (docname, labelid) + self.objects[objtype, name] = (docname, labelid) def add_program_option(self, program, name, docname, labelid): # type: (str, str, str, str) -> None - self.data['progoptions'][program, name] = (docname, labelid) - - def check_consistency(self): - # type: () -> None - for name, (docname, labelid, lineno) in self.data['citations'].items(): - if name not in self.data['citation_refs']: - logger.warning(__('Citation [%s] is not referenced.'), name, - type='ref', subtype='citation', - location=(docname, lineno)) + self.progoptions[program, name] = (docname, labelid) def build_reference_node(self, fromdocname, builder, docname, labelid, sectname, rolename, **options): @@ -703,7 +682,10 @@ class StandardDomain(Domain): elif typ == 'option': resolver = self._resolve_option_xref elif typ == 'citation': - resolver = self._resolve_citation_xref + warnings.warn('pending_xref(domain=std, type=citation) is deprecated: %r' % node, + RemovedInSphinx40Warning) + domain = env.get_domain('citation') + return domain.resolve_xref(env, fromdocname, builder, typ, target, node, contnode) else: resolver = self._resolve_obj_xref @@ -714,13 +696,12 @@ class StandardDomain(Domain): if node['refexplicit']: # reference to anonymous label; the reference uses # the supplied link caption - docname, labelid = self.data['anonlabels'].get(target, ('', '')) + docname, labelid = self.anonlabels.get(target, ('', '')) sectname = node.astext() else: # reference to named label; the final node will # contain the section name after the label - docname, labelid, sectname = self.data['labels'].get(target, - ('', '', '')) + docname, labelid, sectname = self.labels.get(target, ('', '', '')) if not docname: return None @@ -729,10 +710,10 @@ class StandardDomain(Domain): def _resolve_numref_xref(self, env, fromdocname, builder, typ, target, node, contnode): # type: (BuildEnvironment, str, Builder, str, str, addnodes.pending_xref, nodes.Element) -> nodes.Element # NOQA - if target in self.data['labels']: - docname, labelid, figname = self.data['labels'].get(target, ('', '', '')) + if target in self.labels: + docname, labelid, figname = self.labels.get(target, ('', '', '')) else: - docname, labelid = self.data['anonlabels'].get(target, ('', '')) + docname, labelid = self.anonlabels.get(target, ('', '')) figname = None if not docname: @@ -791,7 +772,7 @@ class StandardDomain(Domain): def _resolve_keyword_xref(self, env, fromdocname, builder, typ, target, node, contnode): # type: (BuildEnvironment, str, Builder, str, str, addnodes.pending_xref, nodes.Element) -> nodes.Element # NOQA # keywords are oddballs: they are referenced by named labels - docname, labelid, _ = self.data['labels'].get(target, ('', '', '')) + docname, labelid, _ = self.labels.get(target, ('', '', '')) if not docname: return None return make_refnode(builder, fromdocname, docname, @@ -817,7 +798,7 @@ class StandardDomain(Domain): # type: (BuildEnvironment, str, Builder, str, str, addnodes.pending_xref, nodes.Element) -> nodes.Element # NOQA progname = node.get('std:program') target = target.strip() - docname, labelid = self.data['progoptions'].get((progname, target), ('', '')) + docname, labelid = self.progoptions.get((progname, target), ('', '')) if not docname: commands = [] while ws_re.search(target): @@ -825,8 +806,7 @@ class StandardDomain(Domain): commands.append(subcommand) progname = "-".join(commands) - docname, labelid = self.data['progoptions'].get((progname, target), - ('', '')) + docname, labelid = self.progoptions.get((progname, target), ('', '')) if docname: break else: @@ -835,33 +815,12 @@ class StandardDomain(Domain): return make_refnode(builder, fromdocname, docname, labelid, contnode) - def _resolve_citation_xref(self, env, fromdocname, builder, typ, target, node, contnode): - # type: (BuildEnvironment, str, Builder, str, str, addnodes.pending_xref, nodes.Element) -> nodes.Element # NOQA - docname, labelid, lineno = self.data['citations'].get(target, ('', '', 0)) - if not docname: - if 'ids' in node: - # remove ids attribute that annotated at - # transforms.CitationReference.apply. - del node['ids'][:] - return None - - try: - return make_refnode(builder, fromdocname, docname, - labelid, contnode) - except NoUri: - # remove the ids we added in the CitationReferences - # transform since they can't be transfered to - # the contnode (if it's a Text node) - if not isinstance(contnode, nodes.Element): - del node['ids'][:] - raise - def _resolve_obj_xref(self, env, fromdocname, builder, typ, target, node, contnode): # type: (BuildEnvironment, str, Builder, str, str, addnodes.pending_xref, nodes.Element) -> nodes.Element # NOQA objtypes = self.objtypes_for_role(typ) or [] for objtype in objtypes: - if (objtype, target) in self.data['objects']: - docname, labelid = self.data['objects'][objtype, target] + if (objtype, target) in self.objects: + docname, labelid = self.objects[objtype, target] break else: docname, labelid = '', '' @@ -885,8 +844,8 @@ class StandardDomain(Domain): key = (objtype, target) if objtype == 'term': key = (objtype, ltarget) - if key in self.data['objects']: - docname, labelid = self.data['objects'][key] + if key in self.objects: + docname, labelid = self.objects[key] results.append(('std:' + self.role_for_objtype(objtype), make_refnode(builder, fromdocname, docname, labelid, contnode))) @@ -897,22 +856,22 @@ class StandardDomain(Domain): # handle the special 'doc' reference here for doc in self.env.all_docs: yield (doc, clean_astext(self.env.titles[doc]), 'doc', doc, '', -1) - for (prog, option), info in self.data['progoptions'].items(): + for (prog, option), info in self.progoptions.items(): if prog: fullname = ".".join([prog, option]) yield (fullname, fullname, 'cmdoption', info[0], info[1], 1) else: yield (option, option, 'cmdoption', info[0], info[1], 1) - for (type, name), info in self.data['objects'].items(): + for (type, name), info in self.objects.items(): yield (name, name, type, info[0], info[1], self.object_types[type].attrs['searchprio']) - for name, info in self.data['labels'].items(): - yield (name, info[2], 'label', info[0], info[1], -1) + for name, (docname, labelid, sectionname) in self.labels.items(): + yield (name, sectionname, 'label', docname, labelid, -1) # add anonymous-only labels as well - non_anon_labels = set(self.data['labels']) - for name, info in self.data['anonlabels'].items(): + non_anon_labels = set(self.labels) + for name, (docname, labelid) in self.anonlabels.items(): if name not in non_anon_labels: - yield (name, name, 'label', info[0], info[1], -1) + yield (name, name, 'label', docname, labelid, -1) def get_type_name(self, type, primary=False): # type: (ObjType, bool) -> str @@ -993,6 +952,21 @@ class StandardDomain(Domain): else: return None + def note_citations(self, env, docname, document): + # type: (BuildEnvironment, str, nodes.document) -> None + warnings.warn('StandardDomain.note_citations() is deprecated.', + RemovedInSphinx40Warning) + + def note_citation_refs(self, env, docname, document): + # type: (BuildEnvironment, str, nodes.document) -> None + warnings.warn('StandardDomain.note_citation_refs() is deprecated.', + RemovedInSphinx40Warning) + + def note_labels(self, env, docname, document): + # type: (BuildEnvironment, str, nodes.document) -> None + warnings.warn('StandardDomain.note_labels() is deprecated.', + RemovedInSphinx40Warning) + def setup(app): # type: (Sphinx) -> Dict[str, Any] diff --git a/sphinx/environment/__init__.py b/sphinx/environment/__init__.py index f931b3b13..a5adcbb74 100644 --- a/sphinx/environment/__init__.py +++ b/sphinx/environment/__init__.py @@ -34,6 +34,7 @@ if False: from sphinx.application import Sphinx # NOQA from sphinx.builders import Builder # NOQA from sphinx.config import Config # NOQA + from sphinx.event import EventManager # NOQA from sphinx.domains import Domain # NOQA from sphinx.project import Project # NOQA @@ -95,6 +96,7 @@ class BuildEnvironment: self.srcdir = None # type: str self.config = None # type: Config self.config_status = None # type: int + self.events = None # type: EventManager self.project = None # type: Project self.version = None # type: Dict[str, str] @@ -190,7 +192,7 @@ class BuildEnvironment: # type: () -> Dict """Obtains serializable data for pickling.""" __dict__ = self.__dict__.copy() - __dict__.update(app=None, domains={}) # clear unpickable attributes + __dict__.update(app=None, domains={}, events=None) # clear unpickable attributes return __dict__ def __setstate__(self, state): @@ -210,6 +212,7 @@ class BuildEnvironment: self.app = app self.doctreedir = app.doctreedir + self.events = app.events self.srcdir = app.srcdir self.project = app.project self.version = app.registry.get_envversion(app) @@ -307,7 +310,7 @@ class BuildEnvironment: for domainname, domain in self.domains.items(): domain.merge_domaindata(docnames, other.domaindata[domainname]) - app.emit('env-merge-info', self, docnames, other) + self.events.emit('env-merge-info', self, docnames, other) def path2doc(self, filename): # type: (str) -> Optional[str] @@ -449,7 +452,7 @@ class BuildEnvironment: def check_dependents(self, app, already): # type: (Sphinx, Set[str]) -> Iterator[str] to_rewrite = [] # type: List[str] - for docnames in app.emit('env-get-updated', self): + for docnames in self.events.emit('env-get-updated', self): to_rewrite.extend(docnames) for docname in set(to_rewrite): if docname not in already: @@ -597,7 +600,7 @@ class BuildEnvironment: self.temp_data = backup # allow custom references to be resolved - self.app.emit('doctree-resolved', doctree, docname) + self.events.emit('doctree-resolved', doctree, docname) def collect_relations(self): # type: () -> Dict[str, List[str]] @@ -653,4 +656,4 @@ class BuildEnvironment: # call check-consistency for all extensions for domain in self.domains.values(): domain.check_consistency() - self.app.emit('env-check-consistency', self) + self.events.emit('env-check-consistency', self) diff --git a/sphinx/events.py b/sphinx/events.py index 25a378d7c..df72f8f21 100644 --- a/sphinx/events.py +++ b/sphinx/events.py @@ -10,14 +10,20 @@ :license: BSD, see LICENSE for details. """ +import warnings from collections import OrderedDict, defaultdict +from sphinx.deprecation import RemovedInSphinx40Warning from sphinx.errors import ExtensionError from sphinx.locale import __ +from sphinx.util import logging if False: # For type annotation from typing import Any, Callable, Dict, List # NOQA + from sphinx.application import Sphinx # NOQA + +logger = logging.getLogger(__name__) # List of all known core events. Maps name to arguments description. @@ -42,20 +48,28 @@ core_events = { class EventManager: - def __init__(self): - # type: () -> None + """Event manager for Sphinx.""" + + def __init__(self, app=None): + # type: (Sphinx) -> None + if app is None: + warnings.warn('app argument is required for EventManager.', + RemovedInSphinx40Warning) + self.app = app self.events = core_events.copy() self.listeners = defaultdict(OrderedDict) # type: Dict[str, Dict[int, Callable]] self.next_listener_id = 0 def add(self, name): # type: (str) -> None + """Register a custom Sphinx event.""" if name in self.events: raise ExtensionError(__('Event %r already present') % name) self.events[name] = '' def connect(self, name, callback): # type: (str, Callable) -> int + """Connect a handler to specific event.""" if name not in self.events: raise ExtensionError(__('Unknown event name: %s') % name) @@ -66,18 +80,35 @@ class EventManager: def disconnect(self, listener_id): # type: (int) -> None + """Disconnect a handler.""" for event in self.listeners.values(): event.pop(listener_id, None) def emit(self, name, *args): # type: (str, Any) -> List + """Emit a Sphinx event.""" + try: + logger.debug('[app] emitting event: %r%s', name, repr(args)[:100]) + except Exception: + # not every object likes to be repr()'d (think + # random stuff coming via autodoc) + pass + results = [] for callback in self.listeners[name].values(): - results.append(callback(*args)) + if self.app is None: + # for compatibility; RemovedInSphinx40Warning + results.append(callback(*args)) + else: + results.append(callback(self.app, *args)) return results def emit_firstresult(self, name, *args): # type: (str, Any) -> Any + """Emit a Sphinx event and returns first result. + + This returns the result of the first handler that doesn't return ``None``. + """ for result in self.emit(name, *args): if result is not None: return result diff --git a/sphinx/ext/apidoc.py b/sphinx/ext/apidoc.py index 2d9a771d1..2243e0644 100644 --- a/sphinx/ext/apidoc.py +++ b/sphinx/ext/apidoc.py @@ -19,15 +19,18 @@ import glob import locale import os import sys +import warnings from fnmatch import fnmatch from os import path import sphinx.locale from sphinx import __display_version__, package_dir from sphinx.cmd.quickstart import EXTENSIONS +from sphinx.deprecation import RemovedInSphinx40Warning from sphinx.locale import __ from sphinx.util import rst from sphinx.util.osutil import FileAvoidWrite, ensuredir +from sphinx.util.template import ReSTRenderer if False: # For type annotation @@ -47,6 +50,8 @@ else: INITPY = '__init__.py' PY_SUFFIXES = {'.py', '.pyx'} +template_dir = path.join(package_dir, 'templates', 'apidoc') + def makename(package, module): # type: (str, str) -> str @@ -79,6 +84,8 @@ def write_file(name, text, opts): def format_heading(level, text, escape=True): # type: (int, str, bool) -> str """Create a heading of [1, 2 or 3 supported].""" + warnings.warn('format_warning() is deprecated.', + RemovedInSphinx40Warning) if escape: text = rst.escape(text) underlining = ['=', '-', '~', ][level - 1] * len(text) @@ -88,100 +95,79 @@ def format_heading(level, text, escape=True): def format_directive(module, package=None): # type: (str, str) -> str """Create the automodule directive and add the options.""" + warnings.warn('format_directive() is deprecated.', + RemovedInSphinx40Warning) directive = '.. automodule:: %s\n' % makename(package, module) for option in OPTIONS: directive += ' :%s:\n' % option return directive -def create_module_file(package, module, opts): +def create_module_file(package, basename, opts): # type: (str, str, Any) -> None """Build the text of the file and write the file.""" - if not opts.noheadings: - text = format_heading(1, '%s module' % module) - else: - text = '' - # text += format_heading(2, ':mod:`%s` Module' % module) - text += format_directive(module, package) - write_file(makename(package, module), text, opts) + qualname = makename(package, basename) + context = { + 'show_headings': not opts.noheadings, + 'basename': basename, + 'qualname': qualname, + 'automodule_options': OPTIONS, + } + text = ReSTRenderer(template_dir).render('module.rst', context) + write_file(qualname, text, opts) def create_package_file(root, master_package, subroot, py_files, opts, subs, is_namespace, excludes=[]): # NOQA # type: (str, str, str, List[str], Any, List[str], bool, List[str]) -> None """Build the text of the file and write the file.""" - text = format_heading(1, ('%s package' if not is_namespace else "%s namespace") - % makename(master_package, subroot)) - - if opts.modulefirst and not is_namespace: - text += format_directive(subroot, master_package) - text += '\n' - - # build a list of directories that are szvpackages (contain an INITPY file) - # and also checks the INITPY file is not empty, or there are other python - # source files in that folder. - # (depending on settings - but shall_skip() takes care of that) - subs = [sub for sub in subs if not - shall_skip(path.join(root, sub, INITPY), opts, excludes)] - # if there are some package directories, add a TOC for theses subpackages - if subs: - text += format_heading(2, 'Subpackages') - text += '.. toctree::\n\n' - for sub in subs: - text += ' %s.%s\n' % (makename(master_package, subroot), sub) - text += '\n' - - submods = [path.splitext(sub)[0] for sub in py_files - if not shall_skip(path.join(root, sub), opts, excludes) and - sub != INITPY] - if submods: - text += format_heading(2, 'Submodules') - if opts.separatemodules: - text += '.. toctree::\n\n' - for submod in submods: - modfile = makename(master_package, makename(subroot, submod)) - text += ' %s\n' % modfile - - # generate separate file for this module - if not opts.noheadings: - filetext = format_heading(1, '%s module' % modfile) - else: - filetext = '' - filetext += format_directive(makename(subroot, submod), - master_package) - write_file(modfile, filetext, opts) - else: - for submod in submods: - modfile = makename(master_package, makename(subroot, submod)) - if not opts.noheadings: - text += format_heading(2, '%s module' % modfile) - text += format_directive(makename(subroot, submod), - master_package) - text += '\n' - text += '\n' - - if not opts.modulefirst and not is_namespace: - text += format_heading(2, 'Module contents') - text += format_directive(subroot, master_package) + # build a list of sub packages (directories containing an INITPY file) + subpackages = [sub for sub in subs if not + shall_skip(path.join(root, sub, INITPY), opts, excludes)] + subpackages = [makename(makename(master_package, subroot), pkgname) + for pkgname in subpackages] + # build a list of sub modules + submodules = [path.splitext(sub)[0] for sub in py_files + if not shall_skip(path.join(root, sub), opts, excludes) and + sub != INITPY] + submodules = [makename(master_package, makename(subroot, modname)) + for modname in submodules] + context = { + 'pkgname': makename(master_package, subroot), + 'subpackages': subpackages, + 'submodules': submodules, + 'is_namespace': is_namespace, + 'modulefirst': opts.modulefirst, + 'separatemodules': opts.separatemodules, + 'automodule_options': OPTIONS, + 'show_headings': not opts.noheadings, + } + text = ReSTRenderer(template_dir).render('package.rst', context) write_file(makename(master_package, subroot), text, opts) + if submodules and opts.separatemodules: + for submodule in submodules: + create_module_file(None, submodule, opts) + def create_modules_toc_file(modules, opts, name='modules'): # type: (List[str], Any, str) -> None """Create the module's index.""" - text = format_heading(1, '%s' % opts.header, escape=False) - text += '.. toctree::\n' - text += ' :maxdepth: %s\n\n' % opts.maxdepth - modules.sort() prev_module = '' - for module in modules: + for module in modules[:]: # look if the module is a subpackage and, if yes, ignore it if module.startswith(prev_module + '.'): - continue - prev_module = module - text += ' %s\n' % module + modules.remove(module) + else: + prev_module = module + context = { + 'header': opts.header, + 'maxdepth': opts.maxdepth, + 'docnames': modules, + } + text = ReSTRenderer(template_dir).render('toc.rst', context) write_file(name, text, opts) diff --git a/sphinx/ext/autodoc/__init__.py b/sphinx/ext/autodoc/__init__.py index 8446ab66b..b3c04e464 100644 --- a/sphinx/ext/autodoc/__init__.py +++ b/sphinx/ext/autodoc/__init__.py @@ -10,7 +10,6 @@ :license: BSD, see LICENSE for details. """ -import inspect import re import warnings from typing import Any @@ -23,12 +22,13 @@ from sphinx.ext.autodoc.importer import import_object, get_object_members from sphinx.ext.autodoc.mock import mock from sphinx.locale import _, __ from sphinx.pycode import ModuleAnalyzer, PycodeError +from sphinx.util import inspect from sphinx.util import logging from sphinx.util import rpartition from sphinx.util.docstrings import prepare_docstring -from sphinx.util.inspect import Signature, isdescriptor, safe_getmembers, \ - safe_getattr, object_description, is_builtin_class_method, \ - isenumattribute, isclassmethod, isstaticmethod, isfunction, isbuiltin, ispartial, getdoc +from sphinx.util.inspect import ( + Signature, getdoc, object_description, safe_getattr, safe_getmembers +) if False: # For type annotation @@ -357,7 +357,7 @@ class Documenter: return True modname = self.get_attr(self.object, '__module__', None) - if ispartial(self.object) and modname == '_functools': # for pypy + if inspect.ispartial(self.object) and modname == '_functools': # for pypy return True elif modname and modname != self.modname: return False @@ -403,9 +403,9 @@ class Documenter: retann = self.retann - result = self.env.app.emit_firstresult( - 'autodoc-process-signature', self.objtype, self.fullname, - self.object, self.options, args, retann) + result = self.env.events.emit_firstresult('autodoc-process-signature', + self.objtype, self.fullname, + self.object, self.options, args, retann) if result: args, retann = result @@ -440,7 +440,8 @@ class Documenter: docstring = getdoc(self.object, self.get_attr, self.env.config.autodoc_inherit_docstrings) if docstring: - return [prepare_docstring(docstring, ignore)] + tab_width = self.directive.state.document.settings.tab_width + return [prepare_docstring(docstring, ignore, tab_width)] return [] def process_doc(self, docstrings): @@ -934,7 +935,9 @@ class DocstringSignatureMixin: if base not in valid_names: continue # re-prepare docstring to ignore more leading indentation - self._new_docstrings[i] = prepare_docstring('\n'.join(doclines[1:])) + tab_width = self.directive.state.document.settings.tab_width # type: ignore + self._new_docstrings[i] = prepare_docstring('\n'.join(doclines[1:]), + tabsize=tab_width) result = args, retann # don't look any further break @@ -991,25 +994,27 @@ class FunctionDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter): # typ @classmethod def can_document_member(cls, member, membername, isattr, parent): # type: (Any, str, bool, Any) -> bool - return isfunction(member) or isbuiltin(member) + # supports functions, builtins and bound methods exported at the module level + return (inspect.isfunction(member) or inspect.isbuiltin(member) or + (inspect.isroutine(member) and isinstance(parent, ModuleDocumenter))) def format_args(self): # type: () -> str - if isbuiltin(self.object) or inspect.ismethoddescriptor(self.object): + if inspect.isbuiltin(self.object) or inspect.ismethoddescriptor(self.object): # cannot introspect arguments of a C function or method return None try: - if (not isfunction(self.object) and + if (not inspect.isfunction(self.object) and not inspect.ismethod(self.object) and - not isbuiltin(self.object) and + not inspect.isbuiltin(self.object) and not inspect.isclass(self.object) and hasattr(self.object, '__call__')): args = Signature(self.object.__call__).format_args() else: args = Signature(self.object).format_args() except TypeError: - if (is_builtin_class_method(self.object, '__new__') and - is_builtin_class_method(self.object, '__init__')): + if (inspect.is_builtin_class_method(self.object, '__new__') and + inspect.is_builtin_class_method(self.object, '__init__')): raise TypeError('%r is a builtin class' % self.object) # if a class should be documented as function (yay duck @@ -1030,6 +1035,14 @@ class FunctionDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter): # typ # type: (bool) -> None pass + def add_directive_header(self, sig): + # type: (str) -> None + sourcename = self.get_sourcename() + super().add_directive_header(sig) + + if inspect.iscoroutinefunction(self.object): + self.add_line(' :async:', sourcename) + class DecoratorDocumenter(FunctionDocumenter): """ @@ -1091,8 +1104,8 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter): # type: # classes without __init__ method, default __init__ or # __init__ written in C? if initmeth is None or \ - is_builtin_class_method(self.object, '__init__') or \ - not(inspect.ismethod(initmeth) or isfunction(initmeth)): + inspect.is_builtin_class_method(self.object, '__init__') or \ + not(inspect.ismethod(initmeth) or inspect.isfunction(initmeth)): return None try: return Signature(initmeth, bound_method=True, has_retval=False).format_args() @@ -1167,7 +1180,9 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter): # type: docstrings = [initdocstring] else: docstrings.append(initdocstring) - return [prepare_docstring(docstring, ignore) for docstring in docstrings] + + tab_width = self.directive.state.document.settings.tab_width + return [prepare_docstring(docstring, ignore, tab_width) for docstring in docstrings] def add_content(self, more_content, no_docstring=False): # type: (Any, bool) -> None @@ -1267,6 +1282,7 @@ class MethodDocumenter(DocstringSignatureMixin, ClassLevelDocumenter): # type: Specialized Documenter subclass for methods (normal, static and class). """ objtype = 'method' + directivetype = 'method' member_order = 50 priority = 1 # must be more than FunctionDocumenter @@ -1287,24 +1303,19 @@ class MethodDocumenter(DocstringSignatureMixin, ClassLevelDocumenter): # type: if obj is None: obj = self.object - if isclassmethod(obj): - self.directivetype = 'classmethod' + if (inspect.isclassmethod(obj) or + inspect.isstaticmethod(obj, cls=self.parent, name=self.object_name)): # document class and static members before ordinary ones self.member_order = self.member_order - 1 - elif isstaticmethod(obj, cls=self.parent, name=self.object_name): - self.directivetype = 'staticmethod' - # document class and static members before ordinary ones - self.member_order = self.member_order - 1 - else: - self.directivetype = 'method' + return ret def format_args(self): # type: () -> str - if isbuiltin(self.object) or inspect.ismethoddescriptor(self.object): + if inspect.isbuiltin(self.object) or inspect.ismethoddescriptor(self.object): # can never get arguments of a C function or method return None - if isstaticmethod(self.object, cls=self.parent, name=self.object_name): + if inspect.isstaticmethod(self.object, cls=self.parent, name=self.object_name): args = Signature(self.object, bound_method=False).format_args() else: args = Signature(self.object, bound_method=True).format_args() @@ -1312,6 +1323,19 @@ class MethodDocumenter(DocstringSignatureMixin, ClassLevelDocumenter): # type: args = args.replace('\\', '\\\\') return args + def add_directive_header(self, sig): + # type: (str) -> None + super().add_directive_header(sig) + + sourcename = self.get_sourcename() + obj = self.parent.__dict__.get(self.object_name, self.object) + if inspect.iscoroutinefunction(obj): + self.add_line(' :async:', sourcename) + if inspect.isclassmethod(obj): + self.add_line(' :classmethod:', sourcename) + if inspect.isstaticmethod(obj, cls=self.parent, name=self.object_name): + self.add_line(' :staticmethod:', sourcename) + def document_members(self, all_members=False): # type: (bool) -> None pass @@ -1333,22 +1357,19 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter): @staticmethod def is_function_or_method(obj): # type: (Any) -> bool - return isfunction(obj) or isbuiltin(obj) or inspect.ismethod(obj) + return inspect.isfunction(obj) or inspect.isbuiltin(obj) or inspect.ismethod(obj) @classmethod def can_document_member(cls, member, membername, isattr, parent): # type: (Any, str, bool, Any) -> bool - non_attr_types = (type, MethodDescriptorType) - isdatadesc = isdescriptor(member) and not \ - cls.is_function_or_method(member) and not \ - isinstance(member, non_attr_types) and not \ - type(member).__name__ == "instancemethod" - # That last condition addresses an obscure case of C-defined - # methods using a deprecated type in Python 3, that is not otherwise - # exported anywhere by Python - return isdatadesc or (not isinstance(parent, ModuleDocumenter) and - not inspect.isroutine(member) and - not isinstance(member, type)) + if inspect.isattributedescriptor(member): + return True + elif (not isinstance(parent, ModuleDocumenter) and + not inspect.isroutine(member) and + not isinstance(member, type)): + return True + else: + return False def document_members(self, all_members=False): # type: (bool) -> None @@ -1357,10 +1378,9 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter): def import_object(self): # type: () -> Any ret = super().import_object() - if isenumattribute(self.object): + if inspect.isenumattribute(self.object): self.object = self.object.value - if isdescriptor(self.object) and \ - not self.is_function_or_method(self.object): + if inspect.isattributedescriptor(self.object): self._datadescriptor = True else: # if it's not a data descriptor @@ -1398,6 +1418,37 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter): super().add_content(more_content, no_docstring) +class PropertyDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter): # type: ignore + """ + Specialized Documenter subclass for properties. + """ + objtype = 'property' + directivetype = 'method' + member_order = 60 + + # before AttributeDocumenter + priority = AttributeDocumenter.priority + 1 + + @classmethod + def can_document_member(cls, member, membername, isattr, parent): + # type: (Any, str, bool, Any) -> bool + return inspect.isproperty(member) and isinstance(parent, ClassDocumenter) + + def document_members(self, all_members=False): + # type: (bool) -> None + pass + + def get_real_modname(self): + # type: () -> str + return self.get_attr(self.parent or self.object, '__module__', None) \ + or self.modname + + def add_directive_header(self, sig): + # type: (str) -> None + super().add_directive_header(sig) + self.add_line(' :property:', self.get_sourcename()) + + class InstanceAttributeDocumenter(AttributeDocumenter): """ Specialized Documenter subclass for attributes that cannot be imported @@ -1456,6 +1507,7 @@ def setup(app): app.add_autodocumenter(DecoratorDocumenter) app.add_autodocumenter(MethodDocumenter) app.add_autodocumenter(AttributeDocumenter) + app.add_autodocumenter(PropertyDocumenter) app.add_autodocumenter(InstanceAttributeDocumenter) app.add_config_value('autoclass_content', 'class', True) diff --git a/sphinx/ext/autodoc/directive.py b/sphinx/ext/autodoc/directive.py index 8b41d7fe1..6b002b101 100644 --- a/sphinx/ext/autodoc/directive.py +++ b/sphinx/ext/autodoc/directive.py @@ -6,10 +6,14 @@ :license: BSD, see LICENSE for details. """ +import warnings + from docutils import nodes +from docutils.parsers.rst.states import Struct from docutils.statemachine import StringList from docutils.utils import assemble_option_dict +from sphinx.deprecation import RemovedInSphinx40Warning from sphinx.ext.autodoc import Options, get_documenters from sphinx.util import logging from sphinx.util.docutils import SphinxDirective, switch_source_input @@ -17,7 +21,7 @@ from sphinx.util.nodes import nested_parse_with_titles if False: # For type annotation - from typing import Callable, Dict, List, Set, Type # NOQA + from typing import Any, Callable, Dict, List, Set, Type # NOQA from docutils.parsers.rst.state import RSTState # NOQA from docutils.utils import Reporter # NOQA from sphinx.config import Config # NOQA @@ -30,7 +34,8 @@ logger = logging.getLogger(__name__) # common option names for autodoc directives AUTODOC_DEFAULT_OPTIONS = ['members', 'undoc-members', 'inherited-members', 'show-inheritance', 'private-members', 'special-members', - 'ignore-module-all', 'exclude-members', 'member-order'] + 'ignore-module-all', 'exclude-members', 'member-order', + 'imported-members'] class DummyOptionSpec(dict): @@ -49,8 +54,8 @@ class DummyOptionSpec(dict): class DocumenterBridge: """A parameters container for Documenters.""" - def __init__(self, env, reporter, options, lineno): - # type: (BuildEnvironment, Reporter, Options, int) -> None + def __init__(self, env, reporter, options, lineno, state=None): + # type: (BuildEnvironment, Reporter, Options, int, Any) -> None self.env = env self.reporter = reporter self.genopt = options @@ -58,6 +63,16 @@ class DocumenterBridge: self.filename_set = set() # type: Set[str] self.result = StringList() + if state: + self.state = state + else: + # create fake object for self.state.document.settings.tab_width + warnings.warn('DocumenterBridge requires a state object on instantiation.', + RemovedInSphinx40Warning) + settings = Struct(tab_width=8) + document = Struct(settings=settings) + self.state = Struct(document=document) + def warn(self, msg): # type: (str) -> None logger.warning(msg, location=(self.env.docname, self.lineno)) @@ -130,7 +145,7 @@ class AutodocDirective(SphinxDirective): return [] # generate the output - params = DocumenterBridge(self.env, reporter, documenter_options, lineno) + params = DocumenterBridge(self.env, reporter, documenter_options, lineno, self.state) documenter = doccls(params, self.arguments[0]) documenter.generate(more_content=self.content) if not params.result: diff --git a/sphinx/ext/autosummary/__init__.py b/sphinx/ext/autosummary/__init__.py index 6f4159aab..825fa197c 100644 --- a/sphinx/ext/autosummary/__init__.py +++ b/sphinx/ext/autosummary/__init__.py @@ -175,7 +175,7 @@ _app = None # type: Sphinx class FakeDirective(DocumenterBridge): def __init__(self): # type: () -> None - super().__init__({}, None, Options(), 0) # type: ignore + super().__init__({}, None, Options(), 0, None) # type: ignore def get_documenter(app, obj, parent): @@ -236,7 +236,7 @@ class Autosummary(SphinxDirective): def run(self): # type: () -> List[nodes.Node] self.bridge = DocumenterBridge(self.env, self.state.document.reporter, - Options(), self.lineno) + Options(), self.lineno, self.state) names = [x.strip().split()[0] for x in self.content if x.strip() and re.search(r'^[~a-zA-Z_]', x.strip()[0])] @@ -734,12 +734,13 @@ def process_generate_options(app): return depth_limit = app.config.autosummary_depth_limit - + imported_members = app.config.autosummary_imported_members with mock(app.config.autosummary_mock_imports): generate_autosummary_docs(genfiles, builder=app.builder, warn=logger.warning, info=logger.info, suffix=suffix, base_path=app.srcdir, - app=app, depth_limit=depth_limit) + app=app, imported_members=imported_members, + depth_limit=depth_limit) def setup(app): @@ -766,4 +767,6 @@ def setup(app): app.add_config_value('autosummary_depth_limit', 0, 'env', [int]) app.add_config_value('autosummary_mock_imports', lambda config: config.autodoc_mock_imports, 'env') + app.add_config_value('autosummary_imported_members', [], False, [bool]) + return {'version': sphinx.__display_version__, 'parallel_read_safe': True} diff --git a/sphinx/ext/autosummary/generate.py b/sphinx/ext/autosummary/generate.py index d17ab869f..c89457920 100644 --- a/sphinx/ext/autosummary/generate.py +++ b/sphinx/ext/autosummary/generate.py @@ -41,7 +41,7 @@ from sphinx.util.rst import escape as rst_escape if False: # For type annotation - from typing import Any, Callable, Dict, List, Tuple, Type, Union # NOQA + from typing import Any, Callable, Dict, List, Set, Tuple, Type, Union # NOQA from sphinx.builders import Builder # NOQA from sphinx.ext.autodoc import Documenter # NOQA @@ -198,8 +198,8 @@ def generate_autosummary_docs(sources, # type: List[str] except TemplateNotFound: template = template_env.get_template('autosummary/base.rst') - def get_members(obj, typ, include_public=[], imported=True): - # type: (Any, str, List[str], bool) -> Tuple[List[str], List[str]] + def get_members(obj, types, include_public=[], imported=True): + # type: (Any, Set[str], List[str], bool) -> Tuple[List[str], List[str]] # NOQA items = [] # type: List[str] for name in dir(obj): try: @@ -207,7 +207,7 @@ def generate_autosummary_docs(sources, # type: List[str] except AttributeError: continue documenter = get_documenter(app, value, obj) - if documenter.objtype == typ: + if documenter.objtype in types: if imported or getattr(value, '__module__', None) == obj.__name__: # skip imported members if expected items.append(name) @@ -237,11 +237,11 @@ def generate_autosummary_docs(sources, # type: List[str] if doc.objtype == 'module': ns['members'] = dir(obj) ns['functions'], ns['all_functions'] = \ - get_members(obj, 'function', imported=imported_members) + get_members(obj, {'function'}, imported=imported_members) ns['classes'], ns['all_classes'] = \ - get_members(obj, 'class', imported=imported_members) + get_members(obj, {'class'}, imported=imported_members) ns['exceptions'], ns['all_exceptions'] = \ - get_members(obj, 'exception', imported=imported_members) + get_members(obj, {'exception'}, imported=imported_members) if add_package_children: ns['modules'], ns['all_modules'] = \ get_package_members(obj, 'module') @@ -252,9 +252,9 @@ def generate_autosummary_docs(sources, # type: List[str] ns['inherited_members'] = \ set(dir(obj)) - set(obj.__dict__.keys()) ns['methods'], ns['all_methods'] = \ - get_members(obj, 'method', ['__init__']) + get_members(obj, {'method'}, ['__init__']) ns['attributes'], ns['all_attributes'] = \ - get_members(obj, 'attribute') + get_members(obj, {'attribute', 'property'}) parts = name.split('.') if doc.objtype in ('method', 'attribute'): diff --git a/sphinx/ext/ifconfig.py b/sphinx/ext/ifconfig.py index bad5953d3..1768acf18 100644 --- a/sphinx/ext/ifconfig.py +++ b/sphinx/ext/ifconfig.py @@ -23,6 +23,7 @@ from docutils import nodes import sphinx from sphinx.util.docutils import SphinxDirective +from sphinx.util.nodes import nested_parse_with_titles if False: # For type annotation @@ -48,8 +49,7 @@ class IfConfig(SphinxDirective): node.document = self.state.document self.set_source_info(node) node['expr'] = self.arguments[0] - self.state.nested_parse(self.content, self.content_offset, - node, match_titles=True) + nested_parse_with_titles(self.state, self.content, node) return [node] diff --git a/sphinx/ext/imgmath.py b/sphinx/ext/imgmath.py index eb0d35c25..3efbf4b25 100644 --- a/sphinx/ext/imgmath.py +++ b/sphinx/ext/imgmath.py @@ -21,12 +21,15 @@ from subprocess import CalledProcessError, PIPE from docutils import nodes import sphinx +from sphinx import package_dir +from sphinx.deprecation import RemovedInSphinx40Warning, deprecated_alias from sphinx.errors import SphinxError from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util.math import get_node_equation_number, wrap_displaymath from sphinx.util.osutil import ensuredir from sphinx.util.png import read_png_depth, write_png_depth +from sphinx.util.template import LaTeXRenderer if False: # For type annotation @@ -38,6 +41,8 @@ if False: logger = logging.getLogger(__name__) +templates_path = path.join(package_dir, 'templates', 'imgmath') + class MathExtError(SphinxError): category = 'Math extension error' @@ -85,21 +90,54 @@ DOC_BODY_PREVIEW = r''' ''' depth_re = re.compile(br'\[\d+ depth=(-?\d+)\]') +depthsvg_re = re.compile(br'.*, depth=(.*)pt') +depthsvgcomment_re = re.compile(r'') -def generate_latex_macro(math, config): - # type: (str, Config) -> str +def read_svg_depth(filename): + # type: (str) -> int + """Read the depth from comment at last line of SVG file + """ + with open(filename, 'r') as f: + for line in f: + pass + # Only last line is checked + matched = depthsvgcomment_re.match(line) + if matched: + return int(matched.group(1)) + return None + + +def write_svg_depth(filename, depth): + # type: (str, int) -> None + """Write the depth to SVG file as a comment at end of file + """ + with open(filename, 'a') as f: + f.write('\n' % depth) + + +def generate_latex_macro(image_format, math, config, confdir=''): + # type: (str, str, Config, str) -> str """Generate LaTeX macro.""" - fontsize = config.imgmath_font_size - baselineskip = int(round(fontsize * 1.2)) + variables = { + 'fontsize': config.imgmath_font_size, + 'baselineskip': int(round(config.imgmath_font_size * 1.2)), + 'preamble': config.imgmath_latex_preamble, + 'tightpage': '' if image_format == 'png' else ',tightpage', + 'math': math + } - latex = DOC_HEAD + config.imgmath_latex_preamble if config.imgmath_use_preview: - latex += DOC_BODY_PREVIEW % (fontsize, baselineskip, math) + template_name = 'preview.tex_t' else: - latex += DOC_BODY % (fontsize, baselineskip, math) + template_name = 'template.tex_t' - return latex + for template_dir in config.templates_path: + template = path.join(confdir, template_dir, template_name) + if path.exists(template): + return LaTeXRenderer().render(template, variables) + + return LaTeXRenderer(templates_path).render(template_name, variables) def ensure_tempdir(builder): @@ -197,8 +235,18 @@ def convert_dvi_to_svg(dvipath, builder): command.extend(builder.config.imgmath_dvisvgm_args) command.append(dvipath) - convert_dvi_to_image(command, name) - return filename, None + stdout, stderr = convert_dvi_to_image(command, name) + + depth = None + if builder.config.imgmath_use_preview: + for line in stderr.splitlines(): # not stdout ! + matched = depthsvg_re.match(line) + if matched: + depth = round(float(matched.group(1)) * 100 / 72.27) # assume 100ppi + write_svg_depth(filename, depth) + break + + return filename, depth def render_math(self, math): @@ -220,13 +268,19 @@ def render_math(self, math): if image_format not in SUPPORT_FORMAT: raise MathExtError('imgmath_image_format must be either "png" or "svg"') - latex = generate_latex_macro(math, self.builder.config) + latex = generate_latex_macro(image_format, + math, + self.builder.config, + self.builder.confdir) filename = "%s.%s" % (sha1(latex.encode()).hexdigest(), image_format) relfn = posixpath.join(self.builder.imgpath, 'math', filename) outfn = path.join(self.builder.outdir, self.builder.imagedir, 'math', filename) if path.isfile(outfn): - depth = read_png_depth(outfn) + if image_format == 'png': + depth = read_png_depth(outfn) + elif image_format == 'svg': + depth = read_svg_depth(outfn) return relfn, depth # if latex or dvipng (dvisvgm) has failed once, don't bother to try again @@ -332,6 +386,15 @@ def html_visit_displaymath(self, node): raise nodes.SkipNode +deprecated_alias('sphinx.ext.imgmath', + { + 'DOC_BODY': DOC_BODY, + 'DOC_BODY_PREVIEW': DOC_BODY_PREVIEW, + 'DOC_HEAD': DOC_HEAD, + }, + RemovedInSphinx40Warning) + + def setup(app): # type: (Sphinx) -> Dict[str, Any] app.add_html_math_renderer('imgmath', diff --git a/sphinx/ext/napoleon/docstring.py b/sphinx/ext/napoleon/docstring.py index 00a41afe8..0fea99fb8 100644 --- a/sphinx/ext/napoleon/docstring.py +++ b/sphinx/ext/napoleon/docstring.py @@ -100,7 +100,7 @@ class GoogleDocstring: """ - _name_rgx = re.compile(r"^\s*(:(?P\w+):`(?P[a-zA-Z0-9_.-]+)`|" + _name_rgx = re.compile(r"^\s*((?::(?P\S+):)?`(?P[a-zA-Z0-9_.-]+)`|" r" (?P[a-zA-Z0-9_.-]+))\s*", re.X) def __init__(self, docstring, config=None, app=None, what='', name='', @@ -700,9 +700,9 @@ class GoogleDocstring: fields = self._consume_fields(parse_type=False, prefer_type=True) lines = [] # type: List[str] for _name, _type, _desc in fields: - m = self._name_rgx.match(_type).groupdict() - if m['role']: - _type = m['name'] + m = self._name_rgx.match(_type) + if m and m.group('name'): + _type = m.group('name') _type = ' ' + _type if _type else '' _desc = self._strip_empty(_desc) _descs = ' ' + '\n '.join(_desc) if any(_desc) else '' diff --git a/sphinx/ext/todo.py b/sphinx/ext/todo.py index 1922bb49c..f43520036 100644 --- a/sphinx/ext/todo.py +++ b/sphinx/ext/todo.py @@ -86,7 +86,7 @@ def process_todos(app, doctree): if not hasattr(env, 'todo_all_todos'): env.todo_all_todos = [] # type: ignore for node in doctree.traverse(todo_node): - app.emit('todo-defined', node) + app.events.emit('todo-defined', node) newnode = node.deepcopy() newnode['ids'] = [] diff --git a/sphinx/highlighting.py b/sphinx/highlighting.py index c194738f8..995ca65d3 100644 --- a/sphinx/highlighting.py +++ b/sphinx/highlighting.py @@ -27,6 +27,7 @@ if False: # For type annotation from typing import Any, Dict # NOQA from pygments.formatter import Formatter # NOQA + from pygments.style import Style # NOQA logger = logging.getLogger(__name__) @@ -64,16 +65,8 @@ class PygmentsBridge: def __init__(self, dest='html', stylename='sphinx'): # type: (str, str) -> None self.dest = dest - if stylename is None or stylename == 'sphinx': - style = SphinxStyle - elif stylename == 'none': - style = NoneStyle - elif '.' in stylename: - module, stylename = stylename.rsplit('.', 1) - style = getattr(__import__(module, None, None, ['__name__']), - stylename) - else: - style = get_style_by_name(stylename) + + style = self.get_style(stylename) self.formatter_args = {'style': style} # type: Dict[str, Any] if dest == 'html': self.formatter = self.html_formatter @@ -81,16 +74,25 @@ class PygmentsBridge: self.formatter = self.latex_formatter self.formatter_args['commandprefix'] = 'PYG' + def get_style(self, stylename): + # type: (str) -> Style + if stylename is None or stylename == 'sphinx': + return SphinxStyle + elif stylename == 'none': + return NoneStyle + elif '.' in stylename: + module, stylename = stylename.rsplit('.', 1) + return getattr(__import__(module, None, None, ['__name__']), stylename) + else: + return get_style_by_name(stylename) + def get_formatter(self, **kwargs): # type: (Any) -> Formatter kwargs.update(self.formatter_args) return self.formatter(**kwargs) - def highlight_block(self, source, lang, opts=None, location=None, force=False, **kwargs): - # type: (str, str, Any, Any, bool, Any) -> str - if not isinstance(source, str): - source = source.decode() - + def get_lexer(self, source, lang, opts=None, location=None): + # type: (str, str, Any, Any) -> Lexer # find out which lexer to use if lang in ('py', 'python'): if source.startswith('>>>'): @@ -121,6 +123,15 @@ class PygmentsBridge: else: lexer.add_filter('raiseonerror') + return lexer + + def highlight_block(self, source, lang, opts=None, location=None, force=False, **kwargs): + # type: (str, str, Any, Any, bool, Any) -> str + if not isinstance(source, str): + source = source.decode() + + lexer = self.get_lexer(source, lang, opts, location) + # highlight via Pygments formatter = self.get_formatter(**kwargs) try: @@ -136,6 +147,7 @@ class PygmentsBridge: type='misc', subtype='highlighting_failure', location=location) hlsource = highlight(source, lexers['none'], formatter) + if self.dest == 'html': return hlsource else: diff --git a/sphinx/io.py b/sphinx/io.py index 105ed4397..354121c86 100644 --- a/sphinx/io.py +++ b/sphinx/io.py @@ -14,6 +14,7 @@ from docutils.core import Publisher from docutils.io import FileInput, NullOutput from docutils.parsers.rst import Parser as RSTParser from docutils.readers import standalone +from docutils.transforms.references import DanglingReferences from docutils.writers import UnfilteredWriter from sphinx.transforms import ( @@ -60,7 +61,15 @@ class SphinxBaseReader(standalone.Reader): def get_transforms(self): # type: () -> List[Type[Transform]] - return super().get_transforms() + self.transforms + transforms = super().get_transforms() + self.transforms + + # remove transforms which is not needed for Sphinx + unused = [DanglingReferences] + for transform in unused: + if transform in transforms: + transforms.remove(transform) + + return transforms def new_document(self): # type: () -> nodes.document diff --git a/sphinx/pycode/parser.py b/sphinx/pycode/parser.py index bf80f4367..f9489e91a 100644 --- a/sphinx/pycode/parser.py +++ b/sphinx/pycode/parser.py @@ -381,8 +381,17 @@ class VariableCommentPicker(ast.NodeVisitor): self.context.pop() self.current_function = None + def visit_AsyncFunctionDef(self, node): + # type: (ast.AsyncFunctionDef) -> None + """Handles AsyncFunctionDef node and set context.""" + self.visit_FunctionDef(node) # type: ignore + class DefinitionFinder(TokenProcessor): + """Python source code parser to detect location of functions, + classes and methods. + """ + def __init__(self, lines): # type: (List[str]) -> None super().__init__(lines) @@ -393,6 +402,7 @@ class DefinitionFinder(TokenProcessor): def add_definition(self, name, entry): # type: (str, Tuple[str, int, int]) -> None + """Add a location of definition.""" if self.indents and self.indents[-1][0] == 'def' and entry[0] == 'def': # ignore definition of inner function pass @@ -401,6 +411,7 @@ class DefinitionFinder(TokenProcessor): def parse(self): # type: () -> None + """Parse the code to obtain location of definitions.""" while True: token = self.fetch_token() if token is None: @@ -422,6 +433,7 @@ class DefinitionFinder(TokenProcessor): def parse_definition(self, typ): # type: (str) -> None + """Parse AST of definition.""" name = self.fetch_token() self.context.append(name.value) funcname = '.'.join(self.context) @@ -443,6 +455,7 @@ class DefinitionFinder(TokenProcessor): def finalize_block(self): # type: () -> None + """Finalize definition block.""" definition = self.indents.pop() if definition[0] != 'other': typ, funcname, start_pos = definition diff --git a/sphinx/templates/apidoc/module.rst b/sphinx/templates/apidoc/module.rst new file mode 100644 index 000000000..249027855 --- /dev/null +++ b/sphinx/templates/apidoc/module.rst @@ -0,0 +1,9 @@ +{%- if show_headings %} +{{- [basename, "module"] | join(' ') | e | heading }} + +{% endif -%} +.. automodule:: {{ qualname }} +{%- for option in automodule_options %} + :{{ option }}: +{%- endfor %} + diff --git a/sphinx/templates/apidoc/package.rst b/sphinx/templates/apidoc/package.rst new file mode 100644 index 000000000..0026af34c --- /dev/null +++ b/sphinx/templates/apidoc/package.rst @@ -0,0 +1,52 @@ +{%- macro automodule(modname, options) -%} +.. automodule:: {{ modname }} +{%- for option in options %} + :{{ option }}: +{%- endfor %} +{%- endmacro %} + +{%- macro toctree(docnames) -%} +.. toctree:: +{% for docname in docnames %} + {{ docname }} +{%- endfor %} +{%- endmacro %} + +{%- if is_namespace %} +{{- [pkgname, "namespace"] | join(" ") | e | heading }} +{% else %} +{{- [pkgname, "package"] | join(" ") | e | heading }} +{% endif %} + +{%- if modulefirst and not is_namespace %} +{{ automodule(pkgname, automodule_options) }} +{% endif %} + +{%- if subpackages %} +Subpackages +----------- + +{{ toctree(subpackages) }} +{% endif %} + +{%- if submodules %} +Submodules +---------- +{% if separatemodules %} +{{ toctree(submodules) }} +{%- else %} +{%- for submodule in submodules %} +{% if show_headings %} +{{- [submodule, "module"] | join(" ") | e | heading(2) }} +{% endif %} +{{ automodule(submodule, automodule_options) }} +{%- endfor %} +{% endif %} +{% endif %} + +{%- if not modulefirst and not is_namespace %} +Module contents +--------------- + +{{ automodule(pkgname, automodule_options) }} +{% endif %} diff --git a/sphinx/templates/apidoc/toc.rst b/sphinx/templates/apidoc/toc.rst new file mode 100644 index 000000000..f0877eeb2 --- /dev/null +++ b/sphinx/templates/apidoc/toc.rst @@ -0,0 +1,8 @@ +{{ header | heading }} + +.. toctree:: + :maxdepth: {{ maxdepth }} +{% for docname in docnames %} + {{ docname }} +{%- endfor %} + diff --git a/sphinx/templates/imgmath/preview.tex_t b/sphinx/templates/imgmath/preview.tex_t new file mode 100644 index 000000000..f3fdcda07 --- /dev/null +++ b/sphinx/templates/imgmath/preview.tex_t @@ -0,0 +1,18 @@ +\documentclass[12pt]{article} +\usepackage[utf8]{inputenc} +\usepackage{amsmath} +\usepackage{amsthm} +\usepackage{amssymb} +\usepackage{amsfonts} +\usepackage{anyfontsize} +\usepackage{bm} +\pagestyle{empty} +<%= preamble %> + +\usepackage[active<%= tightpage %>]{preview} + +\begin{document} +\begin{preview} +\fontsize{<%= fontsize %>}{<%= baselineskip %>}\selectfont <%= math %> +\end{preview} +\end{document} diff --git a/sphinx/templates/imgmath/template.tex_t b/sphinx/templates/imgmath/template.tex_t new file mode 100644 index 000000000..92fa8b021 --- /dev/null +++ b/sphinx/templates/imgmath/template.tex_t @@ -0,0 +1,14 @@ +\documentclass[12pt]{article} +\usepackage[utf8]{inputenc} +\usepackage{amsmath} +\usepackage{amsthm} +\usepackage{amssymb} +\usepackage{amsfonts} +\usepackage{anyfontsize} +\usepackage{bm} +\pagestyle{empty} +<%= preamble %> + +\begin{document} +\fontsize{<%= fontsize %>}{<%= baselineskip %>}\selectfont <%= math %> +\end{document} diff --git a/sphinx/templates/latex/longtable.tex_t b/sphinx/templates/latex/longtable.tex_t index ade1a54af..8fe5369df 100644 --- a/sphinx/templates/latex/longtable.tex_t +++ b/sphinx/templates/latex/longtable.tex_t @@ -1,5 +1,5 @@ \begin{savenotes}\sphinxatlongtablestart\begin{longtable} -<%- if table.align == 'center' -%> +<%- if table.align in ('center', 'default') -%> [c] <%- elif table.align == 'left' -%> [l] diff --git a/sphinx/templates/latex/tabular.tex_t b/sphinx/templates/latex/tabular.tex_t index a4f56feb3..a0db7faff 100644 --- a/sphinx/templates/latex/tabular.tex_t +++ b/sphinx/templates/latex/tabular.tex_t @@ -1,6 +1,6 @@ \begin{savenotes}\sphinxattablestart <% if table.align -%> - <%- if table.align == 'center' -%> + <%- if table.align in ('center', 'default') -%> \centering <%- elif table.align == 'left' -%> \raggedright diff --git a/sphinx/templates/latex/tabulary.tex_t b/sphinx/templates/latex/tabulary.tex_t index e3534725b..3236b798a 100644 --- a/sphinx/templates/latex/tabulary.tex_t +++ b/sphinx/templates/latex/tabulary.tex_t @@ -1,6 +1,6 @@ \begin{savenotes}\sphinxattablestart <% if table.align -%> - <%- if table.align == 'center' -%> + <%- if table.align in ('center', 'default') -%> \centering <%- elif table.align == 'left' -%> \raggedright diff --git a/sphinx/templates/quickstart/Makefile.new_t b/sphinx/templates/quickstart/Makefile.new_t index 16a9d482f..1a527578b 100644 --- a/sphinx/templates/quickstart/Makefile.new_t +++ b/sphinx/templates/quickstart/Makefile.new_t @@ -1,9 +1,10 @@ # Minimal makefile for Sphinx documentation # -# You can set these variables from the command line. -SPHINXOPTS = -SPHINXBUILD = sphinx-build +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build SOURCEDIR = {{ rsrcdir }} BUILDDIR = {{ rbuilddir }} @@ -17,3 +18,4 @@ help: # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + diff --git a/sphinx/themes/basic/searchbox.html b/sphinx/themes/basic/searchbox.html index 2ed7fa137..6679ca6b5 100644 --- a/sphinx/themes/basic/searchbox.html +++ b/sphinx/themes/basic/searchbox.html @@ -9,10 +9,10 @@ #} {%- if pagename != "search" and builder != "singlehtml" %}