Merge branch '3.x' into 8103_cached_property

.github/ISSUE_TEMPLATE/config.yml

@@ -2,5 +2,8 @@
 blank_issues_enabled: false  # default: true
 contact_links:
 - name: Question
+  url: https://stackoverflow.com/questions/tagged/python-sphinx
+  about: For Q&A purpose, please use Stackoverflow with the tag python-sphinx
+- name: Discussion
   url: https://groups.google.com/forum/#!forum/sphinx-users
-  about: For Q&A purpose, please use sphinx-users mailing list.
+  about: For general discussion, please use sphinx-users mailing list.

CHANGES

@@ -13,17 +13,24 @@ Deprecated
 Features added
 --------------
 
+* #8100: html: Show a better error message for failures on copying
+  html_static_files
+
 Bugs fixed
 ----------
 
+* #8085: i18n: Add support for having single text domain
+* #8143: autodoc: AttributeError is raised when False value is passed to
+  autodoc_default_options
 * #8103: autodoc: functools.cached_property is not considered as a property
+* #8192: napoleon: description is disappeared when it contains inline literals
 * #8093: The highlight warning has wrong location in some builders (LaTeX,
   singlehtml and so on)
 
 Testing
 --------
 
-Release 3.2.1 (in development)
+Release 3.2.2 (in development)
 ==============================
 
 Dependencies
@@ -41,13 +48,36 @@ Features added
 Bugs fixed
 ----------
 
-* #8074: napoleon: Crashes during processing C-ext module
+* #8188: C, add missing items to internal object types dictionary,
-* #8084: autodoc: KeyError is raised on documenting an attribute of the broken
+  e.g., preventing intersphinx from resolving them.
-  class
+
 
 Testing
 --------
 
+Release 3.2.1 (released Aug 14, 2020)
+=====================================
+
+Features added
+--------------
+
+* #8095: napoleon: Add :confval:`napoleon_preprocess_types` to enable the type
+  preprocessor for numpy style docstrings
+* #8114: C and C++, parse function attributes after parameters and qualifiers.
+
+Bugs fixed
+----------
+
+* #8074: napoleon: Crashes during processing C-ext module
+* #8088: napoleon: "Inline literal start-string without end-string" warning in
+  Numpy style Parameters section
+* #8084: autodoc: KeyError is raised on documenting an attribute of the broken
+  class
+* #8091: autodoc: AttributeError is raised on documenting an attribute on Python
+  3.5.2
+* #8099: autodoc: NameError is raised when target code uses ``TYPE_CHECKING``
+* C++, fix parsing of template template paramters, broken by the fix of #7944
+
 Release 3.2.0 (released Aug 08, 2020)
 =====================================
 

Makefile

@@ -64,10 +64,6 @@ type-check:
 doclinter:
 	python utils/doclinter.py CHANGES *.rst doc/
 
-.PHONY: pylint
-pylint:
-	@pylint --rcfile utils/pylintrc sphinx
-
 .PHONY: test
 test:
 	@$(PYTHON) -m pytest -v $(TEST)

doc/develop.rst

@@ -140,14 +140,14 @@ started with writing your own extensions.
 .. _slideshare: https://www.slideshare.net/
 .. _TikZ/PGF LaTeX package: https://sourceforge.net/projects/pgf/
 .. _MATLAB: https://www.mathworks.com/products/matlab.html
-.. _swf: https://bitbucket.org/klorenz/sphinxcontrib-swf
+.. _swf: https://github.com/sphinx-contrib/swf
-.. _findanything: https://bitbucket.org/klorenz/sphinxcontrib-findanything
+.. _findanything: https://github.com/sphinx-contrib/findanything
-.. _cmakedomain: https://bitbucket.org/klorenz/sphinxcontrib-cmakedomain
+.. _cmakedomain: https://github.com/sphinx-contrib/cmakedomain
 .. _GNU Make: https://www.gnu.org/software/make/
-.. _makedomain: https://bitbucket.org/klorenz/sphinxcontrib-makedomain
+.. _makedomain: https://github.com/sphinx-contrib/makedomain
 .. _inlinesyntaxhighlight: https://sphinxcontrib-inlinesyntaxhighlight.readthedocs.io/
 .. _CMake: https://cmake.org
-.. _domaintools: https://bitbucket.org/klorenz/sphinxcontrib-domaintools
+.. _domaintools: https://github.com/sphinx-contrib/domaintools
 .. _restbuilder: https://pypi.org/project/sphinxcontrib-restbuilder/
 .. _Lasso: http://www.lassosoft.com/
 .. _beamer: https://pypi.org/project/sphinxcontrib-beamer/

doc/extdev/appapi.rst

@@ -177,17 +177,18 @@ type for that event::
    9. (if running in parallel mode, for each process) event.env-merged-info(app, env, docnames, other)
    10. event.env-updated(app, env)
    11. event.env-get-updated(app, env)
-   11. event.env-check-consistency(app, env)
+   12. event.env-check-consistency(app, env)
 
+   # The updated-docs list can be builder dependent, but generally includes all new/changed documents,
+   # plus any output from `env-get-updated`, and then all "parent" documents in the ToC tree
    # For builders that output a single page, they are first joined into a single doctree before post-transforms/doctree-resolved
-   for docname in docnames:
+   for docname in updated-docs:
-      12. apply post-transforms (by priority): docutils.document -> docutils.document
+      13. apply post-transforms (by priority): docutils.document -> docutils.document
-      13. event.doctree-resolved(app, doctree, docname)
+      14. event.doctree-resolved(app, doctree, docname)
           - (for any reference node that fails to resolve) event.missing-reference(env, node, contnode)
 
-   14. Generate output files
+   15. Generate output files
-
+   16. event.build-finished(app, exception)
-   15. event.build-finished(app, exception)
 
 Here is a more detailed list of these events.
 

doc/internals/contributing.rst

@@ -12,6 +12,9 @@ Getting help
 
 The Sphinx community maintains a number of mailing lists and IRC channels.
 
+Stack Overflow with tag `python-sphinx`_
+    Questions and answers about use and development.
+
 sphinx-users <sphinx-users@googlegroups.com>
     Mailing list for user support.
 
@@ -21,6 +24,7 @@ sphinx-dev <sphinx-dev@googlegroups.com>
 #sphinx-doc on irc.freenode.net
     IRC channel for development questions and user support.
 
+.. _python-sphinx: https://stackoverflow.com/questions/tagged/python-sphinx
 
 Bug Reports and Feature Requests
 --------------------------------

doc/usage/configuration.rst

@@ -756,9 +756,15 @@ documentation on :ref:`intl` for details.
    If true, a document's text domain is its docname if it is a top-level
    project file and its very base directory otherwise.
 
+   If set to string, all document's text domain is this string, making all
+   documents use single text domain.
+
    By default, the document ``markup/code.rst`` ends up in the ``markup`` text
    domain.  With this option set to ``False``, it is ``markup/code``.
 
+   .. versionchanged:: 3.3
+      The string value is now accepted.
+
 .. confval:: gettext_uuid
 
    If true, Sphinx generates uuid information for version tracking in message

doc/usage/quickstart.rst

@@ -15,7 +15,7 @@ Much of Sphinx's power comes from the richness of its default plain-text markup
 format, :doc:`reStructuredText </usage/restructuredtext/index>`, along with
 it's :doc:`significant extensibility capabilities </development/index>`.
 
-The goal of this document is to give you a quick taste of what Sphinx it is and
+The goal of this document is to give you a quick taste of what Sphinx is and
 how you might use it. When you're done here, you can check out the
 :doc:`installation guide </usage/installation>` followed by the intro to the
 default markup format used by Sphinx, :doc:`reStucturedText

package-lock.json

@@ -385,12 +385,6 @@
       "integrity": "sha1-Aljq5NPQwJdN4cFpGI7wBR0dGYg=",
       "dev": true
     },
-    "eventemitter3": {
-      "version": "3.1.2",
-      "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-3.1.2.tgz",
-      "integrity": "sha512-tvtQIeLVHjDkJYnzf2dgVMxfuSGJeM/7UCG17TT4EumTfNtF+0nebF/4zWOIkCreAbtNqhGEboB6BWrwqNaw4Q==",
-      "dev": true
-    },
     "extend": {
       "version": "3.0.2",
       "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz",
@@ -535,14 +529,22 @@
       }
     },
     "http-proxy": {
-      "version": "1.17.0",
+      "version": "1.18.1",
-      "resolved": "https://registry.npmjs.org/http-proxy/-/http-proxy-1.17.0.tgz",
+      "resolved": "https://registry.npmjs.org/http-proxy/-/http-proxy-1.18.1.tgz",
-      "integrity": "sha512-Taqn+3nNvYRfJ3bGvKfBSRwy1v6eePlm3oc/aWVxZp57DQr5Eq3xhKJi7Z4hZpS8PC3H4qI+Yly5EmFacGuA/g==",
+      "integrity": "sha512-7mz/721AbnJwIVbnaSv1Cz3Am0ZLT/UBwkC92VlxhXv/k/BBQfM2fXElQNC27BVGr0uwUpplYPQM9LnaBMR5NQ==",
       "dev": true,
       "requires": {
-        "eventemitter3": "^3.0.0",
+        "eventemitter3": "^4.0.0",
         "follow-redirects": "^1.0.0",
         "requires-port": "^1.0.0"
+      },
+      "dependencies": {
+        "eventemitter3": {
+          "version": "4.0.7",
+          "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-4.0.7.tgz",
+          "integrity": "sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==",
+          "dev": true
+        }
       }
     },
     "iconv-lite": {

sphinx/application.py

@@ -1004,7 +1004,7 @@ class Sphinx:
         logger.debug('[app] adding lexer: %r', (alias, lexer))
         if isinstance(lexer, Lexer):
             warnings.warn('app.add_lexer() API changed; '
-                          'Please give lexer class instead instance',
+                          'Please give lexer class instead of instance',
                           RemovedInSphinx40Warning, stacklevel=2)
             lexers[alias] = lexer
         else:

sphinx/builders/gettext.py

@@ -316,7 +316,7 @@ class MessageCatalogBuilder(I18nBuilder):
 def setup(app: Sphinx) -> Dict[str, Any]:
     app.add_builder(MessageCatalogBuilder)
 
-    app.add_config_value('gettext_compact', True, 'gettext')
+    app.add_config_value('gettext_compact', True, 'gettext', Any)
     app.add_config_value('gettext_location', True, 'gettext')
     app.add_config_value('gettext_uuid', False, 'gettext')
     app.add_config_value('gettext_auto_build', True, 'env')

sphinx/builders/html/__init__.py

@@ -751,18 +751,27 @@ class StandaloneHTMLBuilder(Builder):
                 copyfile(jsfile, path.join(self.outdir, '_static', '_stemmer.js'))
 
     def copy_theme_static_files(self, context: Dict) -> None:
+        def onerror(filename: str, error: Exception) -> None:
+            logger.warning(__('Failed to copy a file in html_static_file: %s: %r'),
+                           filename, error)
+
         if self.theme:
             for entry in self.theme.get_theme_dirs()[::-1]:
                 copy_asset(path.join(entry, 'static'),
                            path.join(self.outdir, '_static'),
-                           excluded=DOTFILES, context=context, renderer=self.templates)
+                           excluded=DOTFILES, context=context,
+                           renderer=self.templates, onerror=onerror)
 
     def copy_html_static_files(self, context: Dict) -> None:
+        def onerror(filename: str, error: Exception) -> None:
+            logger.warning(__('Failed to copy a file in html_static_file: %s: %r'),
+                           filename, error)
+
         excluded = Matcher(self.config.exclude_patterns + ["**/.*"])
         for entry in self.config.html_static_path:
             copy_asset(path.join(self.confdir, entry),
                        path.join(self.outdir, '_static'),
-                       excluded, context=context, renderer=self.templates)
+                       excluded, context=context, renderer=self.templates, onerror=onerror)
 
     def copy_html_logo(self) -> None:
         if self.config.html_logo:

sphinx/directives/code.py

@@ -72,7 +72,7 @@ def dedent_lines(lines: List[str], dedent: int, location: Tuple[str, int] = None
         return lines
 
     if any(s[:dedent].strip() for s in lines):
-        logger.warning(__('Over dedent has detected'), location=location)
+        logger.warning(__('non-whitespace stripped by dedent'), location=location)
 
     new_lines = []
     for line in lines:

sphinx/domains/c.py

@@ -32,7 +32,7 @@ from sphinx.transforms import SphinxTransform
 from sphinx.transforms.post_transforms import ReferencesResolver
 from sphinx.util import logging
 from sphinx.util.cfamily import (
-    NoOldIdError, ASTBaseBase, ASTBaseParenExprList,
+    NoOldIdError, ASTBaseBase, ASTAttribute, ASTBaseParenExprList,
     verify_description_mode, StringifyTransform,
     BaseParser, DefinitionError, UnsupportedMultiCharacterCharLiteral,
     identifier_re, anon_identifier_re, integer_literal_re, octal_literal_re,
@@ -652,8 +652,9 @@ class ASTFunctionParameter(ASTBase):
 
 
 class ASTParameters(ASTBase):
-    def __init__(self, args: List[ASTFunctionParameter]) -> None:
+    def __init__(self, args: List[ASTFunctionParameter], attrs: List[ASTAttribute]) -> None:
         self.args = args
+        self.attrs = attrs
 
     @property
     def function_params(self) -> List[ASTFunctionParameter]:
@@ -669,6 +670,9 @@ class ASTParameters(ASTBase):
             first = False
             res.append(str(a))
         res.append(')')
+        for attr in self.attrs:
+            res.append(' ')
+            res.append(transform(attr))
         return ''.join(res)
 
     def describe_signature(self, signode: TextElement, mode: str,
@@ -683,6 +687,9 @@ class ASTParameters(ASTBase):
                 arg.describe_signature(param, 'markType', env, symbol=symbol)
             paramlist += param
         signode += paramlist
+        for attr in self.attrs:
+            signode += nodes.Text(' ')
+            attr.describe_signature(signode)
 
 
 class ASTDeclSpecsSimple(ASTBaseBase):
@@ -1785,7 +1792,7 @@ class Symbol:
 
         if not declaration:
             if Symbol.debug_lookup:
-                Symbol.debug_print("no delcaration")
+                Symbol.debug_print("no declaration")
                 Symbol.debug_indent -= 2
             # good, just a scope creation
             # TODO: what if we have more than one symbol?
@@ -2572,7 +2579,15 @@ class DefinitionParser(BaseParser):
                     self.fail(
                         'Expecting "," or ")" in parameters, '
                         'got "%s".' % self.current_char)
-        return ASTParameters(args)
+
+        attrs = []
+        while True:
+            attr = self._parse_attribute()
+            if attr is None:
+                break
+            attrs.append(attr)
+
+        return ASTParameters(args, attrs)
 
     def _parse_decl_specs_simple(self, outer: str, typed: bool) -> ASTDeclSpecsSimple:
         """Just parse the simple ones."""
@@ -3592,6 +3607,10 @@ class CDomain(Domain):
         'macro': ObjType(_('macro'), 'macro'),
         'type': ObjType(_('type'), 'type'),
         'var': ObjType(_('variable'), 'data'),
+        'enum': ObjType(_('enum'), 'enum'),
+        'enumerator': ObjType(_('enumerator'), 'enumerator'),
+        'struct': ObjType(_('struct'), 'struct'),
+        'union': ObjType(_('union'), 'union'),
     }
 
     directives = {

sphinx/domains/cpp.py

@@ -1879,7 +1879,8 @@ class ASTNoexceptSpec(ASTBase):
 class ASTParametersQualifiers(ASTBase):
     def __init__(self, args: List[ASTFunctionParameter], volatile: bool, const: bool,
                  refQual: str, exceptionSpec: ASTNoexceptSpec, trailingReturn: "ASTType",
-                 override: bool, final: bool, initializer: str) -> None:
+                 override: bool, final: bool, attrs: List[ASTAttribute],
+                 initializer: str) -> None:
         self.args = args
         self.volatile = volatile
         self.const = const
@@ -1888,6 +1889,7 @@ class ASTParametersQualifiers(ASTBase):
         self.trailingReturn = trailingReturn
         self.override = override
         self.final = final
+        self.attrs = attrs
         self.initializer = initializer
 
     @property
@@ -1947,6 +1949,9 @@ class ASTParametersQualifiers(ASTBase):
             res.append(' final')
         if self.override:
             res.append(' override')
+        for attr in self.attrs:
+            res.append(' ')
+            res.append(transform(attr))
         if self.initializer:
             res.append(' = ')
             res.append(self.initializer)
@@ -1988,6 +1993,9 @@ class ASTParametersQualifiers(ASTBase):
             _add_anno(signode, 'final')
         if self.override:
             _add_anno(signode, 'override')
+        for attr in self.attrs:
+            signode += nodes.Text(' ')
+            attr.describe_signature(signode)
         if self.initializer:
             _add_text(signode, '= ' + str(self.initializer))
 
@@ -4284,7 +4292,7 @@ class Symbol:
 
         if not declaration:
             if Symbol.debug_lookup:
-                Symbol.debug_print("no delcaration")
+                Symbol.debug_print("no declaration")
                 Symbol.debug_indent -= 2
             # good, just a scope creation
             # TODO: what if we have more than one symbol?
@@ -5709,6 +5717,13 @@ class DefinitionParser(BaseParser):
             override = self.skip_word_and_ws(
                 'override')  # they can be permuted
 
+        attrs = []
+        while True:
+            attr = self._parse_attribute()
+            if attr is None:
+                break
+            attrs.append(attr)
+
         self.skip_ws()
         initializer = None
         if self.skip_string('='):
@@ -5725,7 +5740,7 @@ class DefinitionParser(BaseParser):
 
         return ASTParametersQualifiers(
             args, volatile, const, refQual, exceptionSpec, trailingReturn,
-            override, final, initializer)
+            override, final, attrs, initializer)
 
     def _parse_decl_specs_simple(self, outer: str, typed: bool) -> ASTDeclSpecsSimple:
         """Just parse the simple ones."""
@@ -6251,6 +6266,7 @@ class DefinitionParser(BaseParser):
     # ==========================================================================
 
     def _parse_template_paramter(self) -> ASTTemplateParam:
+        self.skip_ws()
         if self.skip_word('template'):
             # declare a tenplate template parameter
             nestedParams = self._parse_template_parameter_list()

sphinx/ext/autodoc/__init__.py

@@ -94,6 +94,9 @@ def members_option(arg: Any) -> Union[object, List[str]]:
     """Used to convert the :members: option to auto directives."""
     if arg is None or arg is True:
         return ALL
+    elif arg is False:
+        return None
+    else:
         return [x.strip() for x in arg.split(',') if x.strip()]
 
 
@@ -172,7 +175,7 @@ def merge_members_option(options: Dict) -> None:
 
     members = options.setdefault('members', [])
     for key in {'private-members', 'special-members'}:
-        if key in options and options[key] is not ALL:
+        if key in options and options[key] not in (ALL, None):
             for member in options[key]:
                 if member not in members:
                     members.append(member)
@@ -1608,11 +1611,17 @@ class DataDocumenter(ModuleLevelDocumenter):
             # obtain annotation for this data
             try:
                 annotations = get_type_hints(self.parent)
+            except NameError:
+                # Failed to evaluate ForwardRef (maybe TYPE_CHECKING)
+                annotations = safe_getattr(self.parent, '__annotations__', {})
             except TypeError:
                 annotations = {}
             except KeyError:
                 # a broken class found (refs: https://github.com/sphinx-doc/sphinx/issues/8084)
                 annotations = {}
+            except AttributeError:
+                # AttributeError is raised on 3.5.2 (fixed by 3.5.3)
+                annotations = {}
 
             if self.objpath[-1] in annotations:
                 objrepr = stringify_typehint(annotations.get(self.objpath[-1]))
@@ -1981,11 +1990,17 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter):
             # obtain type annotation for this attribute
             try:
                 annotations = get_type_hints(self.parent)
+            except NameError:
+                # Failed to evaluate ForwardRef (maybe TYPE_CHECKING)
+                annotations = safe_getattr(self.parent, '__annotations__', {})
             except TypeError:
                 annotations = {}
             except KeyError:
                 # a broken class found (refs: https://github.com/sphinx-doc/sphinx/issues/8084)
                 annotations = {}
+            except AttributeError:
+                # AttributeError is raised on 3.5.2 (fixed by 3.5.3)
+                annotations = {}
 
             if self.objpath[-1] in annotations:
                 objrepr = stringify_typehint(annotations.get(self.objpath[-1]))

sphinx/ext/napoleon/__init__.py

@@ -41,6 +41,7 @@ class Config:
         napoleon_use_param = True
         napoleon_use_rtype = True
         napoleon_use_keyword = True
+        napoleon_preprocess_types = False
         napoleon_type_aliases = None
         napoleon_custom_sections = None
 
@@ -237,9 +238,12 @@ class Config:
 
             :returns: *bool* -- True if successful, False otherwise
 
+    napoleon_preprocess_types : :obj:`bool` (Defaults to False)
+        Enable the type preprocessor for numpy style docstrings.
+
     napoleon_type_aliases : :obj:`dict` (Defaults to None)
         Add a mapping of strings to string, translating types in numpy
-        style docstrings.
+        style docstrings. Only works if ``napoleon_preprocess_types = True``.
 
     napoleon_custom_sections : :obj:`list` (Defaults to None)
         Add a list of custom sections to include, expanding the list of parsed sections.
@@ -268,6 +272,7 @@ class Config:
         'napoleon_use_param': (True, 'env'),
         'napoleon_use_rtype': (True, 'env'),
         'napoleon_use_keyword': (True, 'env'),
+        'napoleon_preprocess_types': (False, 'env'),
         'napoleon_type_aliases': (None, 'env'),
         'napoleon_custom_sections': (None, 'env')
     }

sphinx/ext/napoleon/docstring.py

@@ -36,7 +36,7 @@ _numpy_section_regex = re.compile(r'^[=\-`:\'"~^_*+#<>]{2,}\s*$')
 _single_colon_regex = re.compile(r'(?<!:):(?!:)')
 _xref_or_code_regex = re.compile(
     r'((?::(?:[a-zA-Z0-9]+[\-_+:.])*[a-zA-Z0-9]+:`.+?`)|'
-    r'(?:``.+``))')
+    r'(?:``.+?``))')
 _xref_regex = re.compile(
     r'(?:(?::(?:[a-zA-Z0-9]+[\-_+:.])*[a-zA-Z0-9]+:)?`.+?`)'
 )
@@ -1104,6 +1104,7 @@ class NumpyDocstring(GoogleDocstring):
             _name, _type = line, ''
         _name, _type = _name.strip(), _type.strip()
         _name = self._escape_args_and_kwargs(_name)
+        if self._config.napoleon_preprocess_types:
             _type = _convert_numpy_type_spec(
                 _type,
                 location=self._get_location(),

sphinx/registry.py

@@ -401,7 +401,7 @@ class SphinxComponentRegistry:
 
     def load_extension(self, app: "Sphinx", extname: str) -> None:
         """Load a Sphinx extension."""
-        if extname in app.extensions:  # alread loaded
+        if extname in app.extensions:  # already loaded
             return
         if extname in EXTENSION_BLACKLIST:
             logger.warning(__('the extension %r was already merged with Sphinx since '

sphinx/util/cfamily.py

@@ -391,7 +391,7 @@ class BaseParser:
                       % startPos)
         return self.definition[startPos:self.pos]
 
-    def _parse_attribute(self) -> ASTAttribute:
+    def _parse_attribute(self) -> Optional[ASTAttribute]:
         self.skip_ws()
         # try C++11 style
         startPos = self.pos

sphinx/util/fileutil.py

@@ -10,7 +10,7 @@
 
 import os
 import posixpath
-from typing import Dict
+from typing import Callable, Dict
 
 from docutils.utils import relative_path
 
@@ -56,7 +56,8 @@ def copy_asset_file(source: str, destination: str,
 
 
 def copy_asset(source: str, destination: str, excluded: PathMatcher = lambda path: False,
-               context: Dict = None, renderer: "BaseRenderer" = None) -> None:
+               context: Dict = None, renderer: "BaseRenderer" = None,
+               onerror: Callable[[str, Exception], None] = None) -> None:
     """Copy asset files to destination recursively.
 
     On copying, it expands the template variables if context argument is given and
@@ -67,6 +68,7 @@ def copy_asset(source: str, destination: str, excluded: PathMatcher = lambda pat
     :param excluded: The matcher to determine the given path should be copied or not
     :param context: The template variables.  If not given, template files are simply copied
     :param renderer: The template engine.  If not given, SphinxRenderer is used by default
+    :param onerror: The error handler.
     """
     if not os.path.exists(source):
         return
@@ -90,6 +92,12 @@ def copy_asset(source: str, destination: str, excluded: PathMatcher = lambda pat
 
         for filename in files:
             if not excluded(posixpath.join(reldir, filename)):
+                try:
                     copy_asset_file(posixpath.join(root, filename),
                                     posixpath.join(destination, reldir),
                                     context, renderer)
+                except Exception as exc:
+                    if onerror:
+                        onerror(posixpath.join(root, filename), exc)
+                    else:
+                        raise

sphinx/util/i18n.py

@@ -14,7 +14,7 @@ import warnings
 from collections import namedtuple
 from datetime import datetime, timezone
 from os import path
-from typing import Callable, Generator, List, Set, Tuple
+from typing import Callable, Generator, List, Set, Tuple, Union
 
 import babel.dates
 from babel.messages.mofile import write_mo
@@ -128,8 +128,10 @@ def find_catalog(docname: str, compaction: bool) -> str:
     return ret
 
 
-def docname_to_domain(docname: str, compation: bool) -> str:
+def docname_to_domain(docname: str, compation: Union[bool, str]) -> str:
     """Convert docname to domain for catalogs."""
+    if isinstance(compation, str):
+        return compation
     if compation:
         return docname.split(SEP, 1)[0]
     else:

sphinx/util/template.py

@@ -85,7 +85,7 @@ class LaTeXRenderer(SphinxRenderer):
         self.env.block_start_string = '<%'
         self.env.block_end_string = '%>'
         self.env.comment_start_string = '<#'
-        self.env.comment_end_string = '<#'
+        self.env.comment_end_string = '#>'
 
 
 class ReSTRenderer(SphinxRenderer):

tests/roots/test-ext-autodoc/target/TYPE_CHECKING.py

@@ -0,0 +1,8 @@
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from io import StringIO
+
+
+class Foo:
+    attr1: "StringIO"

tests/test_build_gettext.py

@@ -174,3 +174,21 @@ def test_gettext_template_msgid_order_in_sphinxpot(app):
          'msgid "This is Template 2\\.".*'),
         result,
         flags=re.S)
+
+
+@pytest.mark.sphinx(
+    'gettext', srcdir='root-gettext',
+    confoverrides={'gettext_compact': 'documentation'})
+def test_build_single_pot(app):
+    app.builder.build_all()
+
+    assert (app.outdir / 'documentation.pot').isfile()
+
+    result = (app.outdir / 'documentation.pot').read_text()
+    assert re.search(
+        ('msgid "Todo".*'
+         'msgid "Like footnotes.".*'
+         'msgid "The minute.".*'
+         'msgid "Generated section".*'),
+        result,
+        flags=re.S)

tests/test_build_html.py

@@ -10,8 +10,10 @@
 
 import os
 import re
+from distutils.version import LooseVersion
 from itertools import cycle, chain
 
+import pygments
 import pytest
 from html5lib import HTMLParser
 
@@ -1591,4 +1593,8 @@ def test_html_codeblock_linenos_style_inline(app):
     app.build()
     content = (app.outdir / 'index.html').read_text()
 
+    pygments_version = tuple(LooseVersion(pygments.__version__).version)
+    if pygments_version > (2, 7):
+        assert '<span class="linenos">1</span>' in content
+    else:
         assert '<span class="lineno">1 </span>' in content

tests/test_domain_c.py

@@ -497,17 +497,16 @@ def test_attributes():
         parse('member', 'paren_attr({]}) int f')
 
     # position: decl specs
-    check('function', 'static inline __attribute__(()) void f()',
+    check('function', 'static inline __attribute__(()) void f()', {1: 'f'},
-          {1: 'f'},
           output='__attribute__(()) static inline void f()')
-    check('function', '[[attr1]] [[attr2]] void f()',
+    check('function', '[[attr1]] [[attr2]] void f()', {1: 'f'})
-          {1: 'f'},
-          output='[[attr1]] [[attr2]] void f()')
     # position: declarator
     check('member', 'int *[[attr]] i', {1: 'i'})
     check('member', 'int *const [[attr]] volatile i', {1: 'i'},
           output='int *[[attr]] volatile const i')
     check('member', 'int *[[attr]] *i', {1: 'i'})
+    # position: parameters
+    check('function', 'void f() [[attr1]] [[attr2]]', {1: 'f'})
 
     # issue michaeljones/breathe#500
     check('function', 'LIGHTGBM_C_EXPORT int LGBM_BoosterFree(int handle)',

tests/test_domain_cpp.py

@@ -764,6 +764,7 @@ def test_templates():
     check('class', "template<template<typename> typename> {key}A", {2: "II0E0E1A"})
     check('class', "template<template<typename> typename ...T> {key}A", {2: "II0EDpE1A"})
     check('class', "template<template<typename> typename...> {key}A", {2: "II0EDpE1A"})
+    check('class', "template<typename T, template<typename> typename...> {key}A", {2: "I0I0EDpE1A"})
 
     check('class', "template<int> {key}A", {2: "I_iE1A"})
     check('class', "template<int T> {key}A", {2: "I_iE1A"})
@@ -937,15 +938,15 @@ def test_attributes():
     check('function', 'static inline __attribute__(()) void f()',
           {1: 'f', 2: '1fv'},
           output='__attribute__(()) static inline void f()')
-    check('function', '[[attr1]] [[attr2]] void f()',
+    check('function', '[[attr1]] [[attr2]] void f()', {1: 'f', 2: '1fv'})
-          {1: 'f', 2: '1fv'},
-          output='[[attr1]] [[attr2]] void f()')
     # position: declarator
     check('member', 'int *[[attr]] i', {1: 'i__iP', 2: '1i'})
     check('member', 'int *const [[attr]] volatile i', {1: 'i__iPVC', 2: '1i'},
           output='int *[[attr]] volatile const i')
     check('member', 'int &[[attr]] i', {1: 'i__iR', 2: '1i'})
     check('member', 'int *[[attr]] *i', {1: 'i__iPP', 2: '1i'})
+    # position: parameters and qualifiers
+    check('function', 'void f() [[attr1]] [[attr2]]', {1: 'f', 2: '1fv'})
 
 
 def test_xref_parsing():

tests/test_ext_autodoc.py

@@ -1760,6 +1760,28 @@ def test_autodoc_Annotated(app):
     ]
 
 
+@pytest.mark.skipif(sys.version_info < (3, 6), reason='py36+ is required.')
+@pytest.mark.sphinx('html', testroot='ext-autodoc')
+def test_autodoc_TYPE_CHECKING(app):
+    options = {"members": None,
+               "undoc-members": None}
+    actual = do_autodoc(app, 'module', 'target.TYPE_CHECKING', options)
+    assert list(actual) == [
+        '',
+        '.. py:module:: target.TYPE_CHECKING',
+        '',
+        '',
+        '.. py:class:: Foo()',
+        '   :module: target.TYPE_CHECKING',
+        '',
+        '',
+        '   .. py:attribute:: Foo.attr1',
+        '      :module: target.TYPE_CHECKING',
+        '      :type: StringIO',
+        '',
+    ]
+
+
 @pytest.mark.sphinx('html', testroot='pycode-egg')
 def test_autodoc_for_egged_code(app):
     options = {"members": None,

tests/test_ext_napoleon_docstring.py

@@ -66,19 +66,19 @@ Sample namedtuple subclass
 
    Quick description of attr1
 
-   :type: :class:`Arbitrary type`
+   :type: Arbitrary type
 
 .. attribute:: attr2
 
    Quick description of attr2
 
-   :type: :class:`Another arbitrary type`
+   :type: Another arbitrary type
 
 .. attribute:: attr3
 
    Adds a newline after the type
 
-   :type: :class:`Type`
+   :type: Type
 """
 
         self.assertEqual(expected, actual)
@@ -1311,12 +1311,34 @@ class NumpyDocstringTest(BaseDocstringTest):
         config = Config(
             napoleon_use_param=False,
             napoleon_use_rtype=False,
-            napoleon_use_keyword=False)
+            napoleon_use_keyword=False,
+            napoleon_preprocess_types=True)
         for docstring, expected in self.docstrings:
             actual = str(NumpyDocstring(dedent(docstring), config))
             expected = dedent(expected)
             self.assertEqual(expected, actual)
 
+    def test_type_preprocessor(self):
+        docstring = dedent("""
+        Single line summary
+
+        Parameters
+        ----------
+        arg1:str
+            Extended
+            description of arg1
+        """)
+
+        config = Config(napoleon_preprocess_types=False, napoleon_use_param=False)
+        actual = str(NumpyDocstring(docstring, config))
+        expected = dedent("""
+        Single line summary
+
+        :Parameters: **arg1** (*str*) -- Extended
+                     description of arg1
+        """)
+        self.assertEqual(expected, actual)
+
     def test_parameters_with_class_reference(self):
         docstring = """\
 Parameters
@@ -1352,7 +1374,7 @@ x1, x2 : array_like
         config = Config(napoleon_use_param=False)
         actual = str(NumpyDocstring(docstring, config))
         expected = """\
-:Parameters: **x1, x2** (:class:`array_like`) -- Input arrays, description of ``x1``, ``x2``.
+:Parameters: **x1, x2** (*array_like*) -- Input arrays, description of ``x1``, ``x2``.
 """
         self.assertEqual(expected, actual)
 
@@ -1360,9 +1382,9 @@ x1, x2 : array_like
         actual = str(NumpyDocstring(dedent(docstring), config))
         expected = """\
 :param x1: Input arrays, description of ``x1``, ``x2``.
-:type x1: :class:`array_like`
+:type x1: array_like
 :param x2: Input arrays, description of ``x1``, ``x2``.
-:type x2: :class:`array_like`
+:type x2: array_like
 """
         self.assertEqual(expected, actual)
 
@@ -1377,7 +1399,7 @@ param1 : MyClass instance
         config = Config(napoleon_use_param=False)
         actual = str(NumpyDocstring(docstring, config))
         expected = """\
-:Parameters: **param1** (:class:`MyClass instance`)
+:Parameters: **param1** (*MyClass instance*)
 """
         self.assertEqual(expected, actual)
 
@@ -1385,7 +1407,7 @@ param1 : MyClass instance
         actual = str(NumpyDocstring(dedent(docstring), config))
         expected = """\
 :param param1:
-:type param1: :class:`MyClass instance`
+:type param1: MyClass instance
 """
         self.assertEqual(expected, actual)
 
@@ -1474,7 +1496,7 @@ arg_ : type
 
         expected = """
 :ivar arg_: some description
-:vartype arg_: :class:`type`
+:vartype arg_: type
 """
 
         config = Config(napoleon_use_ivar=True)
@@ -1494,7 +1516,7 @@ arg_ : type
 
         expected = """
 :ivar arg\\_: some description
-:vartype arg\\_: :class:`type`
+:vartype arg\\_: type
 """
 
         config = Config(napoleon_use_ivar=True)
@@ -1862,59 +1884,59 @@ definition_after_normal_text : int
         expected = """One line summary.
 
 :param no_list:
-:type no_list: :class:`int`
+:type no_list: int
 :param one_bullet_empty:
                          *
-:type one_bullet_empty: :class:`int`
+:type one_bullet_empty: int
 :param one_bullet_single_line:
                                - first line
-:type one_bullet_single_line: :class:`int`
+:type one_bullet_single_line: int
 :param one_bullet_two_lines:
                              +   first line
                                  continued
-:type one_bullet_two_lines: :class:`int`
+:type one_bullet_two_lines: int
 :param two_bullets_single_line:
                                 -  first line
                                 -  second line
-:type two_bullets_single_line: :class:`int`
+:type two_bullets_single_line: int
 :param two_bullets_two_lines:
                               * first line
                                 continued
                               * second line
                                 continued
-:type two_bullets_two_lines: :class:`int`
+:type two_bullets_two_lines: int
 :param one_enumeration_single_line:
                                     1.  first line
-:type one_enumeration_single_line: :class:`int`
+:type one_enumeration_single_line: int
 :param one_enumeration_two_lines:
                                   1)   first line
                                        continued
-:type one_enumeration_two_lines: :class:`int`
+:type one_enumeration_two_lines: int
 :param two_enumerations_one_line:
                                   (iii) first line
                                   (iv) second line
-:type two_enumerations_one_line: :class:`int`
+:type two_enumerations_one_line: int
 :param two_enumerations_two_lines:
                                    a. first line
                                       continued
                                    b. second line
                                       continued
-:type two_enumerations_two_lines: :class:`int`
+:type two_enumerations_two_lines: int
 :param one_definition_one_line:
                                 item 1
                                     first line
-:type one_definition_one_line: :class:`int`
+:type one_definition_one_line: int
 :param one_definition_two_lines:
                                  item 1
                                      first line
                                      continued
-:type one_definition_two_lines: :class:`int`
+:type one_definition_two_lines: int
 :param two_definitions_one_line:
                                  item 1
                                      first line
                                  item 2
                                      second line
-:type two_definitions_one_line: :class:`int`
+:type two_definitions_one_line: int
 :param two_definitions_two_lines:
                                   item 1
                                       first line
@@ -1922,14 +1944,14 @@ definition_after_normal_text : int
                                   item 2
                                       second line
                                       continued
-:type two_definitions_two_lines: :class:`int`
+:type two_definitions_two_lines: int
 :param one_definition_blank_line:
                                   item 1
 
                                       first line
 
                                       extra first line
-:type one_definition_blank_line: :class:`int`
+:type one_definition_blank_line: int
 :param two_definitions_blank_lines:
                                     item 1
 
@@ -1942,12 +1964,12 @@ definition_after_normal_text : int
                                         second line
 
                                         extra second line
-:type two_definitions_blank_lines: :class:`int`
+:type two_definitions_blank_lines: int
 :param definition_after_normal_text: text line
 
                                      item 1
                                          first line
-:type definition_after_normal_text: :class:`int`
+:type definition_after_normal_text: int
 """
         config = Config(napoleon_use_param=True)
         actual = str(NumpyDocstring(docstring, config))
@@ -2041,7 +2063,7 @@ definition_after_normal_text : int
                item 1
                    first line
 """
-        config = Config(napoleon_use_param=False)
+        config = Config(napoleon_use_param=False, napoleon_preprocess_types=True)
         actual = str(NumpyDocstring(docstring, config))
         self.assertEqual(expected, actual)
 
@@ -2222,6 +2244,7 @@ definition_after_normal_text : int
         config = Config(
             napoleon_use_param=True,
             napoleon_use_rtype=True,
+            napoleon_preprocess_types=True,
             napoleon_type_aliases=translations,
         )
         actual = str(NumpyDocstring(docstring, config))

tox.ini

@@ -26,6 +26,7 @@ extras =
     test
 setenv =
     PYTHONWARNINGS = all,ignore::ImportWarning:importlib._bootstrap_external,ignore::DeprecationWarning:site,ignore::DeprecationWarning:distutils
+    PYTEST_ADDOPTS = --color yes
 commands=
     pytest --durations 25 {posargs}
 
@@ -40,16 +41,6 @@ extras =
 commands =
     flake8 {posargs}
 
-[testenv:pylint]
-basepython = python3
-description =
-    Run source code analyzer.
-deps =
-    pylint
-    {[testenv]deps}
-commands =
-    pylint --rcfile utils/pylintrc sphinx
-
 [testenv:coverage]
 basepython = python3
 description =

utils/pylintrc

@@ -1,301 +0,0 @@
-# lint Python modules using external checkers.
-# 
-# This is the main checker controlling the other ones and the reports
-# generation. It is itself both a raw checker and an astng checker in order
-# to:
-# * handle message activation / deactivation at the module level
-# * handle some basic but necessary stats'data (number of classes, methods...)
-# 
-[MASTER]
-
-# Specify a configuration file.
-#rcfile=
-
-# Profiled execution.
-profile=no
-
-# Add <file or directory> to the black list. It should be a base name, not a
-# path. You may set this option multiple times.
-ignore=.svn
-
-# Pickle collected data for later comparisons.
-persistent=yes
-
-# Set the cache size for astng objects.
-cache-size=500
-
-# List of plugins (as comma separated values of python modules names) to load,
-# usually to register additional checkers.
-load-plugins=
-
-
-[MESSAGES CONTROL]
-
-# Enable only checker(s) with the given id(s). This option conflict with the
-# disable-checker option
-#enable-checker=
-
-# Enable all checker(s) except those with the given id(s). This option conflict
-# with the disable-checker option
-#disable-checker=
-
-# Enable all messages in the listed categories.
-#enable-msg-cat=
-
-# Disable all messages in the listed categories.
-#disable-msg-cat=
-
-# Enable the message(s) with the given id(s).
-#enable-msg=
-
-# Disable the message(s) with the given id(s).
-disable-msg=C0323,W0142,C0301,C0103,C0111,E0213,C0302,C0203,W0703,R0201,W0613,W0612,W0622
-
-
-[REPORTS]
-
-# set the output format. Available formats are text, parseable, colorized and
-# html
-output-format=colorized
-
-# Include message's id in output
-include-ids=yes
-
-# Put messages in a separate file for each module / package specified on the
-# command line instead of printing them on stdout. Reports (if any) will be
-# written in a file name "pylint_global.[txt|html]".
-files-output=no
-
-# Tells wether to display a full report or only the messages
-reports=yes
-
-# Python expression which should return a note less than 10 (10 is the highest
-# note).You have access to the variables errors warning, statement which
-# respectively contain the number of errors / warnings messages and the total
-# number of statements analyzed. This is used by the global evaluation report
-# (R0004).
-evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
-
-# Add a comment according to your evaluation note. This is used by the global
-# evaluation report (R0004).
-comment=no
-
-# Enable the report(s) with the given id(s).
-#enable-report=
-
-# Disable the report(s) with the given id(s).
-#disable-report=
-
-
-# checks for
-# * unused variables / imports
-# * undefined variables
-# * redefinition of variable from builtins or from an outer scope
-# * use of variable before assigment
-# 
-[VARIABLES]
-
-# Tells wether we should check for unused import in __init__ files.
-init-import=no
-
-# A regular expression matching names used for dummy variables (i.e. not used).
-dummy-variables-rgx=_|dummy
-
-# List of additional names supposed to be defined in builtins. Remember that
-# you should avoid to define new builtins when possible.
-additional-builtins=
-
-
-# try to find bugs in the code using type inference
-# 
-[TYPECHECK]
-
-# Tells wether missing members accessed in mixin class should be ignored. A
-# mixin class is detected if its name ends with "mixin" (case insensitive).
-ignore-mixin-members=yes
-
-# When zope mode is activated, consider the acquired-members option to ignore
-# access to some undefined attributes.
-zope=no
-
-# List of members which are usually get through zope's acquisition mecanism and
-# so shouldn't trigger E0201 when accessed (need zope=yes to be considered).
-acquired-members=REQUEST,acl_users,aq_parent
-
-
-# checks for :
-# * doc strings
-# * modules / classes / functions / methods / arguments / variables name
-# * number of arguments, local variables, branchs, returns and statements in
-# functions, methods
-# * required module attributes
-# * dangerous default values as arguments
-# * redefinition of function / method / class
-# * uses of the global statement
-# 
-[BASIC]
-
-# Required attributes for module, separated by a comma
-required-attributes=
-
-# Regular expression which should only match functions or classes name which do
-# not require a docstring
-no-docstring-rgx=__.*__
-
-# Regular expression which should only match correct module names
-module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
-
-# Regular expression which should only match correct module level names
-const-rgx=(([A-Z_][A-Z1-9_]*)|(__.*__))$
-
-# Regular expression which should only match correct class names
-class-rgx=[A-Z_][a-zA-Z0-9]+$
-
-# Regular expression which should only match correct function names
-function-rgx=[a-z_][a-z0-9_]{2,30}$
-
-# Regular expression which should only match correct method names
-method-rgx=[a-z_][a-z0-9_]{2,30}$
-
-# Regular expression which should only match correct instance attribute names
-attr-rgx=[a-z_][a-z0-9_]{2,30}$
-
-# Regular expression which should only match correct argument names
-argument-rgx=[a-z_][a-z0-9_]{2,30}$
-
-# Regular expression which should only match correct variable names
-variable-rgx=[a-z_][a-z0-9_]{2,30}$
-
-# Regular expression which should only match correct list comprehension /
-# generator expression variable names
-inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
-
-# Good variable names which should always be accepted, separated by a comma
-good-names=i,j,k,ex,Run,_
-
-# Bad variable names which should always be refused, separated by a comma
-bad-names=foo,bar,baz,toto,tutu,tata
-
-# List of builtins function names that should not be used, separated by a comma
-bad-functions=apply,input
-
-
-# checks for sign of poor/misdesign:
-# * number of methods, attributes, local variables...
-# * size, complexity of functions, methods
-# 
-[DESIGN]
-
-# Maximum number of arguments for function / method
-max-args=12
-
-# Maximum number of locals for function / method body
-max-locals=30
-
-# Maximum number of return / yield for function / method body
-max-returns=12
-
-# Maximum number of branch for function / method body
-max-branchs=30
-
-# Maximum number of statements in function / method body
-max-statements=60
-
-# Maximum number of parents for a class (see R0901).
-max-parents=7
-
-# Maximum number of attributes for a class (see R0902).
-max-attributes=20
-
-# Minimum number of public methods for a class (see R0903).
-min-public-methods=0
-
-# Maximum number of public methods for a class (see R0904).
-max-public-methods=20
-
-
-# checks for
-# * external modules dependencies
-# * relative / wildcard imports
-# * cyclic imports
-# * uses of deprecated modules
-# 
-[IMPORTS]
-
-# Deprecated modules which should not be used, separated by a comma
-deprecated-modules=regsub,string,TERMIOS,Bastion,rexec
-
-# Create a graph of every (i.e. internal and external) dependencies in the
-# given file (report R0402 must not be disabled)
-import-graph=
-
-# Create a graph of external dependencies in the given file (report R0402 must
-# not be disabled)
-ext-import-graph=
-
-# Create a graph of internal dependencies in the given file (report R0402 must
-# not be disabled)
-int-import-graph=
-
-
-# checks for :
-# * methods without self as first argument
-# * overridden methods signature
-# * access only to existant members via self
-# * attributes not defined in the __init__ method
-# * supported interfaces implementation
-# * unreachable code
-# 
-[CLASSES]
-
-# List of interface methods to ignore, separated by a comma. This is used for
-# instance to not check methods defines in Zope's Interface base class.
-ignore-iface-methods=isImplementedBy,deferred,extends,names,namesAndDescriptions,queryDescriptionFor,getBases,getDescriptionFor,getDoc,getName,getTaggedValue,getTaggedValueTags,isEqualOrExtendedBy,setTaggedValue,isImplementedByInstancesOf,adaptWith,is_implemented_by
-
-# List of method names used to declare (i.e. assign) instance attributes.
-defining-attr-methods=__init__,__new__,setUp
-
-
-# checks for similarities and duplicated code. This computation may be
-# memory / CPU intensive, so you should disable it if you experiments some
-# problems.
-# 
-[SIMILARITIES]
-
-# Minimum lines number of a similarity.
-min-similarity-lines=10
-
-# Ignore comments when computing similarities.
-ignore-comments=yes
-
-# Ignore docstrings when computing similarities.
-ignore-docstrings=yes
-
-
-# checks for:
-# * warning notes in the code
-# * PEP 263: source code with non ascii character but no encoding declaration
-# 
-[MISCELLANEOUS]
-
-# List of note tags to take in consideration, separated by a comma.
-notes=FIXME,XXX,TODO
-
-
-# checks for :
-# * unauthorized constructions
-# * strict indentation
-# * line length
-# * use of <> instead of !=
-# 
-[FORMAT]
-
-# Maximum number of characters on a single line.
-max-line-length=90
-
-# Maximum number of lines in a module
-max-module-lines=1000
-
-# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
-# tab).
-indent-string='    '