Merge branch 'master' into cpp-attributes

.travis.yml

@@ -13,7 +13,7 @@ python:
   - "pypy"
 env:
   global:
-    - TEST=-v
+    - TEST='-v --with-timer --timer-top-n 25'
   matrix:
     - DOCUTILS=0.11
     - DOCUTILS=0.12
@@ -26,11 +26,11 @@ addons:
     - texlive-fonts-recommended
     - texlive-fonts-extra
 install:
-  - pip install -U pip
+  - pip install -U pip setuptools
   - pip install docutils==$DOCUTILS
   - pip install -r test-reqs.txt
-before_script: flake8
+before_script:
+  -  if [[ $TRAVIS_PYTHON_VERSION != '2.6' ]]; then flake8; fi
 script:
-
-  -  if [[ $TRAVIS_PYTHON_VERSION == '3.5' ]]; then make test-async; fi
+  -  if [[ $TRAVIS_PYTHON_VERSION == '3.5' ]]; then make style-check test-async; fi
   -  if [[ $TRAVIS_PYTHON_VERSION != '3.5' ]]; then make test; fi

AUTHORS

@@ -41,6 +41,7 @@ Other contributors, listed alphabetically, are:
 * Martin Mahner -- nature theme
 * Will Maier -- directory HTML builder
 * Jacob Mason -- websupport library (GSOC project)
+* Glenn Matthews -- python domain signature improvements
 * Roland Meister -- epub builder
 * Ezio Melotti -- collapsible sidebar JavaScript
 * Daniel Neuhäuser -- JavaScript domain, Python 3 support (GSOC)

CHANGES

@@ -13,13 +13,37 @@ Incompatible changes
   it was shipped with Sphinx since 1.3.4).
 * latex, literal blocks in tables do not use OriginalVerbatim but
   sphinxVerbatimintable which handles captions and wraps lines(ref #2704).
-
+* latex, replace ``pt`` by TeX equivalent ``bp`` if found in ``width`` or
+  ``height`` attribute of an image.
+* latex, if ``width`` or ``height`` attribute of an image is given with no unit,
+  use ``px`` rather than ignore it.
+* latex: Separate stylesheets of pygments to independent .sty file
+* #2454: The filename of sourcelink is now changed.  The value of
+  `html_sourcelink_suffix` will be appended to the original filename (like
+  ``index.rst.txt``).
+* ``sphinx.util.copy_static_entry()`` is now deprecated.
+  Use ``sphinx.util.fileutil.copy_asset()`` instead.
+* ``sphinx.util.osutil.filecopy()`` skips copying if the file has not been changed
+  (ref: #2510, #2753)
+* Internet Explorer 6-8, Opera 12.1x or Safari 5.1+ support is dropped
+  because jQuery version is updated from 1.11.0 to 3.1.0 (ref: #2634, #2773)
+* QtHelpBuilder doens't generate search page (ref: #2352)
+* QtHelpBuilder uses ``nonav`` theme instead of default one
+  to improve readability.
+* latex: To provide good default settings to Japanese docs, Sphinx uses ``jsbooks``
+  as a docclass by default if the ``language`` is ``ja``.
+* latex: To provide good default settings to Japanese docs, Sphinx uses
+  ``jreport`` and ``jsbooks`` as a docclass by default if the ``language`` is
+  ``ja``.
+* ``sphinx-quickstart`` now allows a project version is empty
+* Fix :download: role on epub/qthelp builder. They ignore the role because they don't support it.
+* ``sphinx.ext.viewcode`` doesn't work on epub building by default. ``viewcode_enable_epub`` option
+* ``sphinx.ext.viewcode`` disabled on singlehtml builder.
 
 Features added
 --------------
 
 * Add ``:caption:`` option for sphinx.ext.inheritance_diagram.
-* #894: Add ``lualatexpdf`` and ``xelatexpdf`` as a make target to build PDF using lualatex or xelatex
 * #2471: Add config variable for default doctest flags.
 * Convert linkcheck builder to requests for better encoding handling
 * #2463, #2516: Add keywords of "meta" directive to search index
@@ -37,54 +61,112 @@ Features added
   and other admonition types, allowing full customizability from the
   ``'preamble'`` key or an input file (ref: feature request #2674, #2685)
 * latex, better computes column widths of some tables (as a result, there will
-  be slight changes as tables now correctly fill the line with; ref: #2708)
+  be slight changes as tables now correctly fill the line width; ref: #2708)
 * latex, sphinxVerbatim environment is more easily customizable (ref: #2704).
   In addition to already existing VerbatimColor and VerbatimBorderColor:
 
   - two lengths ``\sphinxverbatimsep`` and ``\sphinxverbatimborder``,
   - booleans ``\ifsphinxverbatimwithframe`` and ``\ifsphinxverbatimwrapslines``.
+
 * latex, captions for literal blocks inside tables are handled, and long code
   lines wrapped to fit table cell (ref: #2704)
 * #2597: Show warning messages as darkred
+* latex, allow image dimensions using px unit (default is 96px=1in)
+* Show warnings if invalid dimension units found
+* #2650: Add ``--pdb`` option to setup.py command
+* latex, make the use of ``\small`` for code listings customizable (ref #2721)
+* #2663: Add ``--warning-is-error`` option to setup.py command
+* Show warnings if deprecated latex options are used
+* Add sphinx.config.ENUM to check the config values is in candidates
+* math: Add hyperlink marker to each equations in HTML output
+* Add new theme ``nonav`` that doesn't include any navigation links.
+  This is for any help generator like qthelp.
+* #2680: `sphinx.ext.todo` now emits warnings if `todo_emit_warnings` enabled.
+  Also, it emits an additional event named `todo-defined` to handle the TODO
+  entries in 3rd party extensions.
+* Python domain signature parser now uses the xref mixin for 'exceptions',
+  allowing exception classes to be autolinked.
+* #2513: Add `latex_engine` to switch the LaTeX engine by conf.py
 
 Bugs fixed
 ----------
 
 * #2707: (latex) the column width is badly computed for tabular
+* #2799: Sphinx installs roles and directives automatically on importing sphinx
+  module.  Now Sphinx installs them on running application.
 
 Documentation
 -------------
 
 
-Release 1.4.5 (in development)
+Release 1.4.6 (in development)
 ==============================
 
+Bugs fixed
+----------
+
+* applehelp: Sphinx crashes if ``hiutil`` or ``codesign`` commands not found
+* Fix ``make clean`` abort issue when build dir contains regular files like ``DS_Store``.
+* Reduce epubcheck warnings/errors:
+
+  * Fix DOCTYPE to html5
+  * Change extension from .html to .xhtml.
+  * Disable search page on epub results
+
+* #2778: Fix autodoc crashes if obj.__dict__ is a property method and raises exception
+* Fix duplicated toc in epub3 output.
+
+Release 1.4.5 (released Jul 13, 2016)
+=====================================
+
 Incompatible changes
 --------------------
 
 * latex, inclusion of non-inline images from image directive resulted in
   non-coherent whitespaces depending on original image width; new behaviour
-  by necessity from earlier one in some cases. (ref: #2672)
+  by necessity differs from earlier one in some cases. (ref: #2672)
 * latex, use of ``\includegraphics`` to refer to Sphinx custom variant is
-  deprecated; in future it will revert to original LaTeX macro, custom one has
-  already alternative name ``\sphinxincludegraphics``.
+  deprecated; in future it will revert to original LaTeX macro, custom one
+  already has alternative name ``\sphinxincludegraphics``.
 
 Features added
 --------------
 
 * new config option ``latex_keep_old_macro_names``, defaults to True. If False,
-  let macros (for text styling) be defined only with ``\sphinx``-prefixed names.
+  lets macros (for text styling) be defined only with ``\sphinx``-prefixed names.
 * latex writer allows user customization of "shadowed" boxes (topics), via
-  three length variables
+  three length variables.
+* woff-format web font files now supported by the epub builder.
 
 Bugs fixed
 ----------
 
+* jsdump fix for python 3: fixes the HTML search on python > 3
 * #2676: (latex) Error with verbatim text in captions since Sphinx 1.4.4
 * #2629: memoir class crashes LaTeX. Fixed ``by latex_keep_old_macro_names=False`` (ref 2675)
 * #2684: `sphinx.ext.intersphinx` crashes with six-1.4.1
 * #2679: ``float`` package needed for ``'figure_align': 'H'`` latex option
 * #2671: image directive may lead to inconsistent spacing in pdf
+* #2705: ``toctree`` generates empty bullet_list if ``:titlesonly:`` specified
+* #2479: `sphinx.ext.viewcode` uses python2 highlighter by default
+* #2700: HtmlHelp builder has hard coded index.html
+* latex, since 1.4.4 inline literal text is followed by spurious space
+* #2722: C++, fix id generation for var/member declarations to include namespaces.
+* latex, images (from image directive) in lists or quoted blocks did not obey
+  indentation (fixed together with #2671)
+* #2733: since Sphinx-1.4.4 ``make latexpdf`` generates lots of hyperref warnings
+* #2731: `sphinx.ext.autodoc` does not access propertymethods which raises any
+  exceptions
+* #2666: C++, properly look up nested names involving constructors.
+* #2579: Could not refer a label including both spaces and colons via
+  `sphinx.ext.intersphinx`
+* #2718: Sphinx crashes if the document file is not readable
+* #2699: hyperlinks in help HTMLs are broken if `html_file_suffix` is set
+* #2723: extra spaces in latex pdf output from multirow cell
+* #2735: latexpdf ``Underfull \hbox (badness 10000)`` warnings from title page
+* #2667: latex crashes if resized images appeared in section title
+* #2763: (html) Provide default value for required ``alt`` attribute for image
+  tags of SVG source, required to validate and now consistent w/ other formats.
 
 
 Release 1.4.4 (released Jun 12, 2016)
@@ -110,7 +192,6 @@ Bugs fixed
 * #2639: Sphinx now bundles iftex.sty
 * Failed to build PDF with framed.sty 0.95
 * Sphinx now bundles needspace.sty
-* #2666: C++, properly look up nested names involving constructors.
 
 
 Release 1.4.3 (released Jun 5, 2016)

MANIFEST.in

@@ -13,6 +13,7 @@ include sphinx-build.py
 include sphinx-quickstart.py
 include sphinx-apidoc.py
 
+recursive-include sphinx/templates *
 recursive-include sphinx/texinputs *
 recursive-include sphinx/themes *
 recursive-include sphinx/locale *

Makefile

@@ -6,10 +6,11 @@ PYTHON ?= python
 DONT_CHECK = -i build -i dist -i sphinx/style/jquery.js \
              -i sphinx/pycode/pgen2 -i sphinx/util/smartypants.py \
              -i .ropeproject -i doc/_build -i tests/path.py \
-             -i tests/coverage.py -i env -i utils/convert.py \
+             -i tests/coverage.py -i utils/convert.py \
              -i tests/typing_test_data.py \
              -i tests/test_autodoc_py35.py \
              -i tests/build \
+             -i tests/roots/test-warnings/undecodable.rst \
              -i sphinx/search/da.py \
              -i sphinx/search/de.py \
              -i sphinx/search/en.py \

README.rst

@@ -33,6 +33,12 @@ Install from cloned source as editable::
    pip install -e .
 
 
+Release signatures
+==================
+
+Releases are signed with `498D6B9E <https://pgp.mit.edu/pks/lookup?op=vindex&search=0x102C2C17498D6B9E>`_
+
+
 Reading the docs
 ================
 

doc/conf.py

@@ -47,7 +47,7 @@ epub_fix_images = False
 epub_max_image_width = 0
 epub_show_urls = 'inline'
 epub_use_index = False
-epub_guide = (('toc', 'contents.html', u'Table of Contents'),)
+epub_guide = (('toc', 'contents.xhtml', u'Table of Contents'),)
 
 latex_documents = [('contents', 'sphinx.tex', 'Sphinx Documentation',
                     'Georg Brandl', 'manual', 1)]

doc/config.rst

@@ -870,6 +870,13 @@ that use Sphinx's HTMLWriter class.
 
    .. versionadded:: 0.6
 
+.. confval:: html_sourcelink_suffix
+
+   Suffix to be appended to source links (see :confval:`html_show_sourcelink`),
+   unless they have this suffix already.  Default is ``'.txt'``.
+
+   .. versionadded:: 1.5
+
 .. confval:: html_use_opensearch
 
    If nonempty, an `OpenSearch <http://www.opensearch.org/Home>`_ description file will be
@@ -1462,6 +1469,16 @@ Options for LaTeX output
 
 These options influence LaTeX output. See further :doc:`latex`.
 
+.. confval:: latex_engine
+
+   The LaTeX engine to build the docs.  The setting can have the following
+   values:
+
+   * pdflatex -- PDFLaTeX (default)
+   * xelatex -- XeLaTeX
+   * lualatex -- LuaLaTeX
+   * platex -- pLaTeX (default if `language` is 'ja')
+
 .. confval:: latex_documents
 
    This value determines how to group the document tree into LaTeX source files.
@@ -1601,6 +1618,15 @@ These options influence LaTeX output. See further :doc:`latex`.
      ``'pointsize'``
         Point size option of the document class (``'10pt'``, ``'11pt'`` or
         ``'12pt'``), default ``'10pt'``.
+     ``'pxunit'``
+        the value of the ``px`` when used in image attributes ``width`` and
+        ``height``. The default value is ``'49336sp'`` which achieves
+        ``96px=1in`` (``1in = 72.27*65536 = 4736286.72sp``, and all dimensions
+        in TeX are internally integer multiples of ``sp``). To obtain for
+        example ``100px=1in``, one can use ``'0.01in'`` but it is more precise
+        to use ``'47363sp'``. To obtain ``72px=1in``, use ``'1bp'``.
+
+        .. versionadded:: 1.5
      ``'babel'``
         "babel" package inclusion, default ``'\\usepackage{babel}'``.
      ``'fontpkg'``
@@ -1693,6 +1719,11 @@ These options influence LaTeX output. See further :doc:`latex`.
 
    .. versionadded:: 1.0
 
+   .. versionchanged:: 1.5
+
+      In Japanese docs(`language` is ``ja``), ``'jreport'`` is used for
+      ``'howto'`` and ``'jsbooks'`` is used for ``'manual'`` by default.
+
 .. confval:: latex_additional_files
 
    A list of file names, relative to the configuration directory, to copy to the

doc/domains.rst

@@ -359,6 +359,18 @@ single word, like this::
    :param int priority: The priority of the message, can be a number 1-5
 
 
+.. versionadded:: 1.5
+
+Container types such as lists and dictionaries can be linked automatically
+using the following syntax::
+
+   :type priorities: list(int)
+   :type priorities: list[int]
+   :type mapping: dict(str, int)
+   :type mapping: dict[str, int]
+   :type point: tuple(float, float)
+   :type point: tuple[float, float]
+
 .. _python-roles:
 
 Cross-referencing Python objects

doc/ext/doctest.rst

@@ -182,7 +182,7 @@ The doctest extension uses the following configuration values:
 .. confval:: doctest_default_flags
 
    By default, these options are enabled:
-   
+
    - ``ELLIPSIS``, allowing you to put ellipses in the expected output that
      match anything in the actual output;
    - ``IGNORE_EXCEPTION_DETAIL``, causing everything following the leftmost

doc/ext/example_google.py

@@ -211,14 +211,14 @@ class ExampleClass(object):
             param1 (str): Description of `param1`.
             param2 (:obj:`int`, optional): Description of `param2`. Multiple
                 lines are supported.
-            param3 (:obj:`list` of :obj:`str`): Description of `param3`.
+            param3 (list(str)): Description of `param3`.
 
         """
         self.attr1 = param1
         self.attr2 = param2
         self.attr3 = param3  #: Doc comment *inline* with attribute
 
-        #: list of str: Doc comment *before* attribute, with type specified
+        #: list(str): Doc comment *before* attribute, with type specified
         self.attr4 = ['attr4']
 
         self.attr5 = None
@@ -231,7 +231,7 @@ class ExampleClass(object):
 
     @property
     def readwrite_property(self):
-        """:obj:`list` of :obj:`str`: Properties with both a getter and setter
+        """list(str): Properties with both a getter and setter
         should only be documented in their getter method.
 
         If the setter method contains notable behavior, it should be

doc/ext/example_google.rst

@@ -9,7 +9,9 @@ Example Google Style Python Docstrings
 
    :ref:`example_numpy`
 
-Download: :download:`example_google.py <example_google.py>`
+.. only:: builder_html
+
+   Download: :download:`example_google.py <example_google.py>`
 
 .. literalinclude:: example_google.py
    :language: python

doc/ext/example_numpy.py

@@ -260,7 +260,7 @@ class ExampleClass(object):
         ----------
         param1 : str
             Description of `param1`.
-        param2 : :obj:`list` of :obj:`str`
+        param2 : list(str)
             Description of `param2`. Multiple
             lines are supported.
         param3 : :obj:`int`, optional
@@ -271,7 +271,7 @@ class ExampleClass(object):
         self.attr2 = param2
         self.attr3 = param3  #: Doc comment *inline* with attribute
 
-        #: list of str: Doc comment *before* attribute, with type specified
+        #: list(str): Doc comment *before* attribute, with type specified
         self.attr4 = ["attr4"]
 
         self.attr5 = None
@@ -284,7 +284,7 @@ class ExampleClass(object):
 
     @property
     def readwrite_property(self):
-        """:obj:`list` of :obj:`str`: Properties with both a getter and setter
+        """list(str): Properties with both a getter and setter
         should only be documented in their getter method.
 
         If the setter method contains notable behavior, it should be

doc/ext/example_numpy.rst

@@ -9,7 +9,9 @@ Example NumPy Style Python Docstrings
 
    :ref:`example_google`
 
-Download: :download:`example_numpy.py <example_numpy.py>`
+.. only:: builder_html
+
+   Download: :download:`example_numpy.py <example_numpy.py>`
 
 .. literalinclude:: example_numpy.py
    :language: python

doc/ext/inheritance.rst

@@ -40,7 +40,7 @@ It adds this directive:
       included.
 
    .. versionchanged:: 1.5
-      Added ``caption`` option 
+      Added ``caption`` option
 
 
 New config values are:

doc/ext/napoleon.rst

@@ -213,24 +213,6 @@ Google style with Python 3 type annotations::
         """
         return True
 
-Google style with `Python 2/3 compatible annotations`_::
-
-    def func(arg1, arg2):
-        # type: (int, str) -> bool
-        """Summary line.
-
-        Extended description of function.
-
-        Args:
-            arg1: Description of arg1
-            arg2: Description of arg2
-
-        Returns:
-            Description of return value
-
-        """
-        return True
-
 Google style with types in docstrings::
 
     def func(arg1, arg2):
@@ -248,6 +230,10 @@ Google style with types in docstrings::
         """
         return True
 
+.. Note::
+   `Python 2/3 compatible annotations`_ aren't currently
+   supported by Sphinx and won't show up in the docs.
+
 .. _PEP 484:
    https://www.python.org/dev/peps/pep-0484/
 

doc/ext/todo.rst

@@ -34,6 +34,13 @@ There is also an additional config value:
    If this is ``True``, :rst:dir:`todo` and :rst:dir:`todolist` produce output,
    else they produce nothing.  The default is ``False``.
 
+.. confval:: todo_emit_warnings
+
+   If this is ``True``, :rst:dir:`todo` emits a warning for each TODO entries.
+   The default is ``False``.
+
+   .. versionadded:: 1.5
+
 .. confval:: todo_link_only
 
    If this is ``True``, :rst:dir:`todolist` produce output without file path and line,
@@ -41,3 +48,11 @@ There is also an additional config value:
 
    .. versionadded:: 1.4
 
+autodoc provides the following an additional event:
+
+.. event:: todo-defined (app, node)
+
+   .. versionadded:: 1.5
+
+   Emitted when a todo is defined. *node* is the defined ``sphinx.ext.todo.todo_node``
+   node.

doc/ext/viewcode.rst

@@ -15,6 +15,11 @@ a highlighted version of the source code, and a link will be added to all object
 descriptions that leads to the source code of the described object.  A link back
 from the source to the description will also be inserted.
 
+This extension works only on HTML related builders like ``html``,
+``applehelp``, ``devhelp``, ``htmlhelp``, ``qthelp`` and so on except
+``singlehtml``. By default ``epub`` builder doesn't
+support this extension (see :confval:`viewcode_enable_epub`).
+
 There is an additional config value:
 
 .. confval:: viewcode_import
@@ -34,3 +39,26 @@ There is an additional config value:
       main routine is protected by a ``if __name__ == '__main__'`` condition.
 
    .. versionadded:: 1.3
+
+.. confval:: viewcode_enable_epub
+
+   If this is ``True``, viewcode extension is also enabled even if you use
+   epub builders. This extension generates pages outside toctree, but this
+   is not preferred as epub format.
+
+   Until 1.4.x, this extension is always enabled. If you want to generate
+   epub as same as 1.4.x, you should set ``True``, but epub format checker's
+   score becomes worse.
+
+   The default is ``False``.
+
+   .. versionadded:: 1.5
+
+   .. warning::
+
+      Not all epub readers support pages generated by viewcode extension.
+      These readers ignore links to pages are not under toctree.
+
+      Some reader's rendering result are corrupted and
+      `epubcheck <https://github.com/IDPF/epubcheck>`_'s score
+      becomes worse even if the reader supports.

doc/install.rst

@@ -135,7 +135,7 @@ install.
 .. note::
 
    ``pip`` has been contained in the Python official installation after version
-    of Python-3.4.0 or Python-2.7.9.
+   of Python-3.4.0 or Python-2.7.9.
 
 
 Installing Sphinx with pip

doc/latex.rst

@@ -120,6 +120,14 @@ Let us illustrate here what can be modified by the second method.
      were not wrapped), they now use ``sphinxVerbatimintable``.
   .. versionadded:: 1.5
      the two customizable lengths, the ``sphinxVerbatimintable``.
+- by default the Sphinx style file ``sphinx.sty`` includes the command
+  ``\fvset{fontsize=\small}`` as part of its configuration of
+  ``fancyvrb.sty``. The user may override this for example via
+  ``\fvset{fontsize=auto}`` which will use for listings the ambient
+  font size. Refer to ``fancyvrb.sty``'s documentation for further keys.
+
+  .. versionadded:: 1.5
+     formerly, the use of ``\small`` for code listings was not customizable.
 - miscellaneous colours: *TitleColor*, *InnerLinkColor*, *OuterLinkColor*,
   *VerbatimColor* (this is a background colour), *VerbatimBorderColor*.
 - the ``\sphinxAtStartFootnote`` is inserted between footnote numbers and their

doc/markup/inline.rst

@@ -200,6 +200,12 @@ Referencing downloadable files
    The ``example.py`` file will be copied to the output directory, and a
    suitable link generated to it.
 
+   Not to show unavailable download links, you should wrap whole paragraphs that
+   have this role::
+
+      .. only:: builder_html
+
+         See :download:`this example script <../example.py>`.
 
 Cross-referencing figures by figure number
 ------------------------------------------

doc/rest.rst

@@ -363,8 +363,9 @@ directory on building (e.g. the ``_static`` directory for HTML output.)
 
 Interpretation of image size options (``width`` and ``height``) is as follows:
 if the size has no unit or the unit is pixels, the given size will only be
-respected for output channels that support pixels (i.e. not in LaTeX output).
-Other units (like ``pt`` for points) will be used for HTML and LaTeX output.
+respected for output channels that support pixels. Other units (like ``pt``
+for points) will be used for HTML and LaTeX output (the latter replaces ``pt``
+by ``bp`` as this is the TeX unit such that ``72bp=1in``).
 
 Sphinx extends the standard docutils behavior by allowing an asterisk for the
 extension::
@@ -386,6 +387,9 @@ Note that image file names should not contain spaces.
 .. versionchanged:: 0.6
    Image paths can now be absolute.
 
+.. versionchanged:: 1.5
+   latex target supports pixels (default is ``96px=1in``).
+
 
 Footnotes
 ---------

setup.cfg

@@ -26,4 +26,4 @@ universal = 1
 [flake8]
 max-line-length=95
 ignore=E113,E116,E221,E226,E241,E251,E901
-exclude=tests/*,build/*,sphinx/search/*,sphinx/pycode/pgen2/*
+exclude=tests/*,build/*,sphinx/search/*,sphinx/pycode/pgen2/*,doc/ext/example*.py

setup.py

@@ -138,11 +138,8 @@ else:
                                                  domain + '.js'))
 
             for js_file, (locale, po_file) in zip(js_files, po_files):
-                infile = open(po_file, 'r')
-                try:
+                with open(po_file, 'r') as infile:
                     catalog = read_po(infile, locale)
-                finally:
-                    infile.close()
 
                 if catalog.fuzzy and not self.use_fuzzy:
                     continue
@@ -159,8 +156,7 @@ else:
                             msgid = msgid[0]
                         jscatalog[msgid] = message.string
 
-                outfile = open(js_file, 'wb')
-                try:
+                with open(js_file, 'wb') as outfile:
                     outfile.write('Documentation.addTranslations(')
                     dump(dict(
                         messages=jscatalog,
@@ -168,8 +164,6 @@ else:
                         locale=str(catalog.locale)
                     ), outfile, sort_keys=True)
                     outfile.write(');')
-                finally:
-                    outfile.close()
 
     cmdclass['compile_catalog'] = compile_catalog_plusjs
 

sphinx/application.py

@@ -32,12 +32,11 @@ from sphinx.roles import XRefRole
 from sphinx.config import Config
 from sphinx.errors import SphinxError, SphinxWarning, ExtensionError, \
     VersionRequirementError, ConfigError
-from sphinx.domains import ObjType, BUILTIN_DOMAINS
+from sphinx.domains import ObjType
 from sphinx.domains.std import GenericObject, Target, StandardDomain
-from sphinx.builders import BUILTIN_BUILDERS
 from sphinx.environment import BuildEnvironment
 from sphinx.io import SphinxStandaloneReader
-from sphinx.util import pycompat  # noqa: imported for side-effects
+from sphinx.util import pycompat  # noqa: F401
 from sphinx.util import import_object
 from sphinx.util.tags import Tags
 from sphinx.util.osutil import ENOENT
@@ -65,6 +64,36 @@ events = {
     'html-page-context': 'pagename, context, doctree or None',
     'build-finished': 'exception',
 }
+builtin_extensions = (
+    'sphinx.builders.applehelp',
+    'sphinx.builders.changes',
+    'sphinx.builders.epub',
+    'sphinx.builders.epub3',
+    'sphinx.builders.devhelp',
+    'sphinx.builders.dummy',
+    'sphinx.builders.gettext',
+    'sphinx.builders.html',
+    'sphinx.builders.htmlhelp',
+    'sphinx.builders.latex',
+    'sphinx.builders.linkcheck',
+    'sphinx.builders.manpage',
+    'sphinx.builders.qthelp',
+    'sphinx.builders.texinfo',
+    'sphinx.builders.text',
+    'sphinx.builders.websupport',
+    'sphinx.builders.xml',
+    'sphinx.domains.c',
+    'sphinx.domains.cpp',
+    'sphinx.domains.javascript',
+    'sphinx.domains.python',
+    'sphinx.domains.rst',
+    'sphinx.domains.std',
+    'sphinx.directives',
+    'sphinx.directives.code',
+    'sphinx.directives.other',
+    'sphinx.directives.patches',
+    'sphinx.roles',
+)
 
 CONFIG_FILENAME = 'conf.py'
 ENV_PICKLE_FILENAME = 'environment.pickle'
@@ -87,9 +116,9 @@ class Sphinx(object):
         self._additional_source_parsers = {}
         self._listeners = {}
         self._setting_up_extension = ['?']
-        self.domains = BUILTIN_DOMAINS.copy()
+        self.domains = {}
         self.buildername = buildername
-        self.builderclasses = BUILTIN_BUILDERS.copy()
+        self.builderclasses = {}
         self.builder = None
         self.env = None
         self.enumerable_nodes = {}
@@ -152,6 +181,10 @@ class Sphinx(object):
         if self.confdir is None:
             self.confdir = self.srcdir
 
+        # load all built-in extension modules
+        for extension in builtin_extensions:
+            self.setup_extension(extension)
+
         # extension loading support for alabaster theme
         # self.config.html_theme is not set from conf.py at here
         # for now, sphinx always load a 'alabaster' extension.
@@ -243,8 +276,8 @@ class Sphinx(object):
 
     def _init_env(self, freshenv):
         if freshenv:
-            self.env = BuildEnvironment(self.srcdir, self.doctreedir,
-                                        self.config)
+            self.env = BuildEnvironment(self.srcdir, self.doctreedir, self.config)
+            self.env.set_warnfunc(self.warn)
             self.env.find_files(self.config)
             for domain in self.domains.keys():
                 self.env.domains[domain] = self.domains[domain](self.env)
@@ -253,6 +286,7 @@ class Sphinx(object):
                 self.info(bold('loading pickled environment... '), nonl=True)
                 self.env = BuildEnvironment.frompickle(
                     self.srcdir, self.config, path.join(self.doctreedir, ENV_PICKLE_FILENAME))
+                self.env.set_warnfunc(self.warn)
                 self.env.domains = {}
                 for domain in self.domains.keys():
                     # this can raise if the data version doesn't fit
@@ -265,8 +299,6 @@ class Sphinx(object):
                     self.info('failed: %s' % err)
                 return self._init_env(freshenv=True)
 
-        self.env.set_warnfunc(self.warn)
-
     def _init_builder(self, buildername):
         if buildername is None:
             print('No builder selected, using default: html', file=self._status)
@@ -275,11 +307,6 @@ class Sphinx(object):
             raise SphinxError('Builder name %s not registered' % buildername)
 
         builderclass = self.builderclasses[buildername]
-        if isinstance(builderclass, tuple):
-            # builtin builder
-            mod, cls = builderclass
-            builderclass = getattr(
-                __import__('sphinx.builders.' + mod, None, None, [cls]), cls)
         self.builder = builderclass(self)
         self.emit('builder-inited')
 
@@ -571,13 +598,9 @@ class Sphinx(object):
             raise ExtensionError('Builder class %s has no "name" attribute'
                                  % builder)
         if builder.name in self.builderclasses:
-            if isinstance(self.builderclasses[builder.name], tuple):
-                raise ExtensionError('Builder %r is a builtin builder' %
-                                     builder.name)
-            else:
-                raise ExtensionError(
-                    'Builder %r already exists (in module %s)' % (
-                        builder.name, self.builderclasses[builder.name].__module__))
+            raise ExtensionError(
+                'Builder %r already exists (in module %s)' % (
+                    builder.name, self.builderclasses[builder.name].__module__))
         self.builderclasses[builder.name] = builder
 
     def add_config_value(self, name, default, rebuild, types=()):
@@ -778,8 +801,7 @@ class Sphinx(object):
 
     def add_latex_package(self, packagename, options=None):
         self.debug('[app] adding latex package: %r', packagename)
-        from sphinx.builders.latex import LaTeXBuilder
-        LaTeXBuilder.usepackages.append((packagename, options))
+        self.builder.usepackages.append((packagename, options))
 
     def add_lexer(self, alias, lexer):
         self.debug('[app] adding lexer: %r', (alias, lexer))

sphinx/builders/__init__.py

@@ -451,29 +451,3 @@ class Builder(object):
         except AttributeError:
             optname = '%s_%s' % (default, option)
             return getattr(self.config, optname)
-
-BUILTIN_BUILDERS = {
-    'dummy':      ('dummy', 'DummyBuilder'),
-    'html':       ('html', 'StandaloneHTMLBuilder'),
-    'dirhtml':    ('html', 'DirectoryHTMLBuilder'),
-    'singlehtml': ('html', 'SingleFileHTMLBuilder'),
-    'pickle':     ('html', 'PickleHTMLBuilder'),
-    'json':       ('html', 'JSONHTMLBuilder'),
-    'web':        ('html', 'PickleHTMLBuilder'),
-    'htmlhelp':   ('htmlhelp', 'HTMLHelpBuilder'),
-    'devhelp':    ('devhelp', 'DevhelpBuilder'),
-    'qthelp':     ('qthelp', 'QtHelpBuilder'),
-    'applehelp':  ('applehelp', 'AppleHelpBuilder'),
-    'epub':       ('epub', 'EpubBuilder'),
-    'epub3':      ('epub3', 'Epub3Builder'),
-    'latex':      ('latex', 'LaTeXBuilder'),
-    'text':       ('text', 'TextBuilder'),
-    'man':        ('manpage', 'ManualPageBuilder'),
-    'texinfo':    ('texinfo', 'TexinfoBuilder'),
-    'changes':    ('changes', 'ChangesBuilder'),
-    'linkcheck':  ('linkcheck', 'CheckExternalLinksBuilder'),
-    'websupport': ('websupport', 'WebSupportBuilder'),
-    'gettext':    ('gettext', 'MessageCatalogBuilder'),
-    'xml':        ('xml', 'XMLBuilder'),
-    'pseudoxml':  ('xml', 'PseudoXMLBuilder'),
-}

sphinx/builders/applehelp.py

@@ -13,14 +13,16 @@ from __future__ import print_function
 import codecs
 import pipes
 
-from os import path
+from os import path, environ
+import shlex
 
 from sphinx.builders.html import StandaloneHTMLBuilder
-from sphinx.util import copy_static_entry
-from sphinx.util.osutil import copyfile, ensuredir
+from sphinx.config import string_classes
+from sphinx.util.osutil import copyfile, ensuredir, make_filename
 from sphinx.util.console import bold
+from sphinx.util.fileutil import copy_asset
 from sphinx.util.pycompat import htmlescape
-from sphinx.util.matching import compile_matchers
+from sphinx.util.matching import Matcher
 from sphinx.errors import SphinxError
 
 import plistlib
@@ -84,6 +86,7 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
         super(AppleHelpBuilder, self).init()
         # the output files for HTML help must be .html only
         self.out_suffix = '.html'
+        self.link_suffix = '.html'
 
         if self.config.applehelp_bundle_id is None:
             raise SphinxError('You must set applehelp_bundle_id before '
@@ -104,17 +107,15 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
         self.finish_tasks.add_task(self.build_helpbook)
 
     def copy_localized_files(self):
-        source_dir = path.join(self.confdir,
-                               self.config.applehelp_locale + '.lproj')
+        source_dir = path.join(self.confdir, self.config.applehelp_locale + '.lproj')
         target_dir = self.outdir
 
         if path.isdir(source_dir):
             self.info(bold('copying localized files... '), nonl=True)
 
-            ctx = self.globalcontext.copy()
-            matchers = compile_matchers(self.config.exclude_patterns)
-            copy_static_entry(source_dir, target_dir, self, ctx,
-                              exclude_matchers=matchers)
+            excluded = Matcher(self.config.exclude_patterns + ['**/.*'])
+            copy_asset(source_dir, target_dir, excluded,
+                       context=self.globalcontext, renderer=self.templates)
 
             self.info('done')
 
@@ -213,16 +214,19 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
             self.warn('you will need to index this help book with:\n  %s'
                       % (' '.join([pipes.quote(arg) for arg in args])))
         else:
-            p = subprocess.Popen(args,
-                                 stdout=subprocess.PIPE,
-                                 stderr=subprocess.STDOUT)
+            try:
+                p = subprocess.Popen(args,
+                                     stdout=subprocess.PIPE,
+                                     stderr=subprocess.STDOUT)
 
-            output = p.communicate()[0]
+                output = p.communicate()[0]
 
-            if p.returncode != 0:
-                raise AppleHelpIndexerFailed(output)
-            else:
-                self.info('done')
+                if p.returncode != 0:
+                    raise AppleHelpIndexerFailed(output)
+                else:
+                    self.info('done')
+            except OSError:
+                raise AppleHelpIndexerFailed('Command not found: %s' % args[0])
 
         # If we've been asked to, sign the bundle
         if self.config.applehelp_codesign_identity:
@@ -244,13 +248,48 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
                 self.warn('you will need to sign this help book with:\n  %s'
                           % (' '.join([pipes.quote(arg) for arg in args])))
             else:
-                p = subprocess.Popen(args,
-                                     stdout=subprocess.PIPE,
-                                     stderr=subprocess.STDOUT)
+                try:
+                    p = subprocess.Popen(args,
+                                         stdout=subprocess.PIPE,
+                                         stderr=subprocess.STDOUT)
 
-                output = p.communicate()[0]
+                    output = p.communicate()[0]
 
-                if p.returncode != 0:
-                    raise AppleHelpCodeSigningFailed(output)
-                else:
-                    self.info('done')
+                    if p.returncode != 0:
+                        raise AppleHelpCodeSigningFailed(output)
+                    else:
+                        self.info('done')
+                except OSError:
+                    raise AppleHelpCodeSigningFailed('Command not found: %s' % args[0])
+
+
+def setup(app):
+    app.setup_extension('sphinx.builders.html')
+    app.add_builder(AppleHelpBuilder)
+
+    app.add_config_value('applehelp_bundle_name',
+                         lambda self: make_filename(self.project), 'applehelp')
+    app.add_config_value('applehelp_bundle_id', None, 'applehelp', string_classes)
+    app.add_config_value('applehelp_dev_region', 'en-us', 'applehelp')
+    app.add_config_value('applehelp_bundle_version', '1', 'applehelp')
+    app.add_config_value('applehelp_icon', None, 'applehelp', string_classes)
+    app.add_config_value('applehelp_kb_product',
+                         lambda self: '%s-%s' % (make_filename(self.project), self.release),
+                         'applehelp')
+    app.add_config_value('applehelp_kb_url', None, 'applehelp', string_classes)
+    app.add_config_value('applehelp_remote_url', None, 'applehelp', string_classes)
+    app.add_config_value('applehelp_index_anchors', False, 'applehelp', string_classes)
+    app.add_config_value('applehelp_min_term_length', None, 'applehelp', string_classes)
+    app.add_config_value('applehelp_stopwords',
+                         lambda self: self.language or 'en', 'applehelp')
+    app.add_config_value('applehelp_locale', lambda self: self.language or 'en', 'applehelp')
+    app.add_config_value('applehelp_title', lambda self: self.project + ' Help', 'applehelp')
+    app.add_config_value('applehelp_codesign_identity',
+                         lambda self: environ.get('CODE_SIGN_IDENTITY', None),
+                         'applehelp'),
+    app.add_config_value('applehelp_codesign_flags',
+                         lambda self: shlex.split(environ.get('OTHER_CODE_SIGN_FLAGS', '')),
+                         'applehelp'),
+    app.add_config_value('applehelp_indexer_path', '/usr/bin/hiutil', 'applehelp')
+    app.add_config_value('applehelp_codesign_path', '/usr/bin/codesign', 'applehelp')
+    app.add_config_value('applehelp_disable_external_tools', False, None)

sphinx/builders/changes.py

@@ -15,12 +15,12 @@ from os import path
 from six import iteritems
 
 from sphinx import package_dir
-from sphinx.util import copy_static_entry
 from sphinx.locale import _
 from sphinx.theming import Theme
 from sphinx.builders import Builder
 from sphinx.util.osutil import ensuredir, os_path
 from sphinx.util.console import bold
+from sphinx.util.fileutil import copy_asset_file
 from sphinx.util.pycompat import htmlescape
 
 
@@ -138,12 +138,10 @@ class ChangesBuilder(Builder):
                 f.write(self.templates.render('changes/rstsource.html', ctx))
         themectx = dict(('theme_' + key, val) for (key, val) in
                         iteritems(self.theme.get_options({})))
-        copy_static_entry(path.join(package_dir, 'themes', 'default',
-                                    'static', 'default.css_t'),
-                          self.outdir, self, themectx)
-        copy_static_entry(path.join(package_dir, 'themes', 'basic',
-                                    'static', 'basic.css'),
-                          self.outdir, self)
+        copy_asset_file(path.join(package_dir, 'themes', 'default', 'static', 'default.css_t'),
+                        self.outdir, context=themectx, renderer=self.templates)
+        copy_asset_file(path.join(package_dir, 'themes', 'basic', 'static', 'basic.css'),
+                        self.outdir)
 
     def hl(self, text, version):
         text = htmlescape(text)
@@ -154,3 +152,7 @@ class ChangesBuilder(Builder):
 
     def finish(self):
         pass
+
+
+def setup(app):
+    app.add_builder(ChangesBuilder)

sphinx/builders/devhelp.py

@@ -18,6 +18,7 @@ from os import path
 from docutils import nodes
 
 from sphinx import addnodes
+from sphinx.util.osutil import make_filename
 from sphinx.builders.html import StandaloneHTMLBuilder
 
 try:
@@ -59,6 +60,7 @@ class DevhelpBuilder(StandaloneHTMLBuilder):
     def init(self):
         StandaloneHTMLBuilder.init(self)
         self.out_suffix = '.html'
+        self.link_suffix = '.html'
 
     def handle_finish(self):
         self.build_devhelp(self.outdir, self.config.devhelp_basename)
@@ -129,3 +131,10 @@ class DevhelpBuilder(StandaloneHTMLBuilder):
         # Dump the XML file
         with comp_open(path.join(outdir, outname + '.devhelp'), 'w') as f:
             tree.write(f, 'utf-8')
+
+
+def setup(app):
+    app.setup_extension('sphinx.builders.html')
+    app.add_builder(DevhelpBuilder)
+
+    app.add_config_value('devhelp_basename', lambda self: make_filename(self.project), None)

sphinx/builders/dummy.py

@@ -34,3 +34,7 @@ class DummyBuilder(Builder):
 
     def finish(self):
         pass
+
+
+def setup(app):
+    app.add_builder(DummyBuilder)

sphinx/builders/epub.py

@@ -29,7 +29,7 @@ from docutils import nodes
 from sphinx import addnodes
 from sphinx.builders.html import StandaloneHTMLBuilder
 from sphinx.util.i18n import format_date
-from sphinx.util.osutil import ensuredir, copyfile, EEXIST
+from sphinx.util.osutil import ensuredir, copyfile, make_filename, EEXIST
 from sphinx.util.smartypants import sphinx_smarty_pants as ssp
 from sphinx.util.console import brown
 
@@ -113,7 +113,7 @@ COVER_TEMPLATE = u'''\
     <meta name="cover" content="%(cover)s"/>
 '''
 
-COVERPAGE_NAME = u'epub-cover.html'
+COVERPAGE_NAME = u'epub-cover.xhtml'
 
 FILE_TEMPLATE = u'''\
     <item id="%(id)s"
@@ -128,6 +128,10 @@ GUIDE_TEMPLATE = u'''\
 
 TOCTREE_TEMPLATE = u'toctree-l%d'
 
+DOCTYPE = u'''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
+  "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
+'''
+
 LINK_TARGET_TEMPLATE = u' [%(uri)s]'
 
 FOOTNOTE_LABEL_TEMPLATE = u'#%d'
@@ -143,7 +147,7 @@ GUIDE_TITLES = {
 }
 
 MEDIA_TYPES = {
-    '.html': 'application/xhtml+xml',
+    '.xhtml': 'application/xhtml+xml',
     '.css': 'text/css',
     '.png': 'image/png',
     '.gif': 'image/gif',
@@ -152,6 +156,7 @@ MEDIA_TYPES = {
     '.jpeg': 'image/jpeg',
     '.otf': 'application/x-font-otf',
     '.ttf': 'application/x-font-ttf',
+    '.woff': 'application/font-woff',
 }
 
 VECTOR_GRAPHICS_EXTENSIONS = ('.svg',)
@@ -183,6 +188,11 @@ class EpubBuilder(StandaloneHTMLBuilder):
     add_permalinks = False
     # don't add sidebar etc.
     embedded = True
+    # disable download role
+    download_support = False
+
+    # don't generate search index or include search page
+    search = False
 
     mimetype_template = MIMETYPE_TEMPLATE
     container_template = CONTAINER_TEMPLATE
@@ -197,6 +207,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
     spine_template = SPINE_TEMPLATE
     guide_template = GUIDE_TEMPLATE
     toctree_template = TOCTREE_TEMPLATE
+    doctype = DOCTYPE
     link_target_template = LINK_TARGET_TEMPLATE
     css_link_target_class = CSS_LINK_TARGET_CLASS
     guide_titles = GUIDE_TITLES
@@ -206,7 +217,8 @@ class EpubBuilder(StandaloneHTMLBuilder):
     def init(self):
         StandaloneHTMLBuilder.init(self)
         # the output files for epub must be .html only
-        self.out_suffix = '.html'
+        self.out_suffix = '.xhtml'
+        self.link_suffix = '.xhtml'
         self.playorder = 0
         self.tocid = 0
 
@@ -276,7 +288,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
         """
         refnodes.insert(0, {
             'level': 1,
-            'refuri': self.esc(self.config.master_doc + '.html'),
+            'refuri': self.esc(self.config.master_doc + self.out_suffix),
             'text': ssp(self.esc(
                 self.env.titles[self.config.master_doc].astext()))
         })
@@ -471,6 +483,9 @@ class EpubBuilder(StandaloneHTMLBuilder):
             else:
                 super(EpubBuilder, self).copy_image_files()
 
+    def copy_download_files(self):
+        pass
+
     def handle_page(self, pagename, addctx, templatename='page.html',
                     outfilename=None, event_arg=None):
         """Create a rendered page.
@@ -480,6 +495,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
         """
         if pagename.startswith('genindex'):
             self.fix_genindex(addctx['genindexentries'])
+        addctx['doctype'] = self.doctype
         StandaloneHTMLBuilder.handle_page(self, pagename, addctx, templatename,
                                           outfilename, event_arg)
 
@@ -760,3 +776,33 @@ class EpubBuilder(StandaloneHTMLBuilder):
             fp = path.join(outdir, file)
             epub.write(fp, file, zipfile.ZIP_DEFLATED)
         epub.close()
+
+
+def setup(app):
+    app.setup_extension('sphinx.builders.html')
+    app.add_builder(EpubBuilder)
+
+    # config values
+    app.add_config_value('epub_basename', lambda self: make_filename(self.project), None)
+    app.add_config_value('epub_theme', 'epub', 'html')
+    app.add_config_value('epub_theme_options', {}, 'html')
+    app.add_config_value('epub_title', lambda self: self.html_title, 'html')
+    app.add_config_value('epub_author', 'unknown', 'html')
+    app.add_config_value('epub_language', lambda self: self.language or 'en', 'html')
+    app.add_config_value('epub_publisher', 'unknown', 'html')
+    app.add_config_value('epub_copyright', lambda self: self.copyright, 'html')
+    app.add_config_value('epub_identifier', 'unknown', 'html')
+    app.add_config_value('epub_scheme', 'unknown', 'html')
+    app.add_config_value('epub_uid', 'unknown', 'env')
+    app.add_config_value('epub_cover', (), 'env')
+    app.add_config_value('epub_guide', (), 'env')
+    app.add_config_value('epub_pre_files', [], 'env')
+    app.add_config_value('epub_post_files', [], 'env')
+    app.add_config_value('epub_exclude_files', [], 'env')
+    app.add_config_value('epub_tocdepth', 3, 'env')
+    app.add_config_value('epub_tocdup', True, 'env')
+    app.add_config_value('epub_tocscope', 'default', 'env')
+    app.add_config_value('epub_fix_images', False, 'env')
+    app.add_config_value('epub_max_image_width', 0, 'env')
+    app.add_config_value('epub_show_urls', 'inline', 'html')
+    app.add_config_value('epub_use_index', lambda self: self.html_use_index, 'html')

sphinx/builders/epub3.py

@@ -13,6 +13,7 @@
 import codecs
 from os import path
 
+from sphinx.config import string_classes
 from sphinx.builders.epub import EpubBuilder
 
 
@@ -73,7 +74,6 @@ PACKAGE_DOC_TEMPLATE = u'''\
 %(files)s
   </manifest>
   <spine toc="ncx" page-progression-direction="%(page_progression_direction)s">
-    <itemref idref="nav" />
 %(spine)s
   </spine>
   <guide>
@@ -82,6 +82,9 @@ PACKAGE_DOC_TEMPLATE = u'''\
 </package>
 '''
 
+DOCTYPE = u'''<!DOCTYPE html>
+'''
+
 # The epub3 publisher
 
 
@@ -99,6 +102,7 @@ class Epub3Builder(EpubBuilder):
     navlist_template = NAVLIST_TEMPLATE
     navlist_indent = NAVLIST_INDENT
     content_template = PACKAGE_DOC_TEMPLATE
+    doctype = DOCTYPE
 
     # Finish by building the epub file
     def handle_finish(self):
@@ -209,3 +213,12 @@ class Epub3Builder(EpubBuilder):
 
         # Add nav.xhtml to epub file
         self.files.append(outname)
+
+
+def setup(app):
+    app.setup_extension('sphinx.builders.epub')
+    app.add_builder(Epub3Builder)
+
+    app.add_config_value('epub3_description', '', 'epub3', string_classes)
+    app.add_config_value('epub3_contributor', 'unknown', 'epub3', string_classes)
+    app.add_config_value('epub3_page_progression_direction', 'ltr', 'epub3', string_classes)

sphinx/builders/gettext.py

@@ -233,3 +233,13 @@ class MessageCatalogBuilder(I18nBuilder):
                         replace('"', r'\"'). \
                         replace('\n', '\\n"\n"')
                     pofile.write('msgid "%s"\nmsgstr ""\n\n' % message)
+
+
+def setup(app):
+    app.add_builder(MessageCatalogBuilder)
+
+    app.add_config_value('gettext_compact', True, 'gettext')
+    app.add_config_value('gettext_location', True, 'gettext')
+    app.add_config_value('gettext_uuid', False, 'gettext')
+    app.add_config_value('gettext_auto_build', True, 'env')
+    app.add_config_value('gettext_additional_targets', [], 'env')

sphinx/builders/html.py

@@ -27,13 +27,15 @@ from docutils.frontend import OptionParser
 from docutils.readers.doctree import Reader as DoctreeReader
 
 from sphinx import package_dir, __display_version__
-from sphinx.util import jsonimpl, copy_static_entry, copy_extra_entry
+from sphinx.util import jsonimpl
 from sphinx.util.i18n import format_date
 from sphinx.util.osutil import SEP, os_path, relative_uri, ensuredir, \
     movefile, copyfile
 from sphinx.util.nodes import inline_all_toctrees
-from sphinx.util.matching import patmatch, compile_matchers
-from sphinx.locale import _
+from sphinx.util.fileutil import copy_asset
+from sphinx.util.matching import patmatch, Matcher, DOTFILES
+from sphinx.config import string_classes
+from sphinx.locale import _, l_
 from sphinx.search import js_index
 from sphinx.theming import Theme
 from sphinx.builders import Builder
@@ -80,6 +82,7 @@ class StandaloneHTMLBuilder(Builder):
     add_permalinks = True
     embedded = False  # for things like HTML help or Qt help: suppresses sidebar
     search = True  # for things like HTML help and Apple help: suppress search
+    download_support = True  # enable download role
 
     # This is a class attribute because it is mutated by Sphinx.add_javascript.
     script_files = ['_static/jquery.js', '_static/underscore.js',
@@ -339,6 +342,7 @@ class StandaloneHTMLBuilder(Builder):
             show_sphinx = self.config.html_show_sphinx,
             has_source = self.config.html_copy_source,
             show_source = self.config.html_show_sourcelink,
+            sourcelink_suffix = self.config.html_sourcelink_suffix,
             file_suffix = self.out_suffix,
             script_files = self.script_files,
             language = self.config.language,
@@ -402,15 +406,21 @@ class StandaloneHTMLBuilder(Builder):
         # title rendered as HTML
         title = self.env.longtitles.get(docname)
         title = title and self.render_partial(title)['title'] or ''
+
+        # Suffix for the document
+        source_suffix = path.splitext(self.env.doc2path(docname))[1]
+
         # the name for the copied source
-        sourcename = self.config.html_copy_source and docname + '.txt' or ''
+        if self.config.html_copy_source:
+            sourcename = docname + source_suffix
+            if source_suffix != self.config.html_sourcelink_suffix:
+                sourcename += self.config.html_sourcelink_suffix
+        else:
+            sourcename = ''
 
         # metadata for the document
         meta = self.env.metadata.get(docname)
 
-        # Suffix for the document
-        source_suffix = '.' + self.env.doc2path(docname).split('.')[-1]
-
         # local TOC and global TOC tree
         self_toc = self.env.get_toc_for(docname, self)
         toc = self.render_partial(self_toc)['fragment']
@@ -581,9 +591,8 @@ class StandaloneHTMLBuilder(Builder):
         self.info(bold('copying static files... '), nonl=True)
         ensuredir(path.join(self.outdir, '_static'))
         # first, create pygments style file
-        f = open(path.join(self.outdir, '_static', 'pygments.css'), 'w')
-        f.write(self.highlighter.get_stylesheet())
-        f.close()
+        with open(path.join(self.outdir, '_static', 'pygments.css'), 'w') as f:
+            f.write(self.highlighter.get_stylesheet())
         # then, copy translations JavaScript file
         if self.config.language is not None:
             jsfile = self._get_translations_js()
@@ -605,21 +614,19 @@ class StandaloneHTMLBuilder(Builder):
 
         # then, copy over theme-supplied static files
         if self.theme:
-            themeentries = [path.join(themepath, 'static')
-                            for themepath in self.theme.get_dirchain()[::-1]]
-            for entry in themeentries:
-                copy_static_entry(entry, path.join(self.outdir, '_static'),
-                                  self, ctx)
+            for theme_path in self.theme.get_dirchain()[::-1]:
+                entry = path.join(theme_path, 'static')
+                copy_asset(entry, path.join(self.outdir, '_static'), excluded=DOTFILES,
+                           context=ctx, renderer=self.templates)
         # then, copy over all user-supplied static files
-        staticentries = [path.join(self.confdir, spath)
-                         for spath in self.config.html_static_path]
-        matchers = compile_matchers(self.config.exclude_patterns)
-        for entry in staticentries:
+        excluded = Matcher(self.config.exclude_patterns + ["**/.*"])
+        for static_path in self.config.html_static_path:
+            entry = path.join(self.confdir, static_path)
             if not path.exists(entry):
                 self.warn('html_static_path entry %r does not exist' % entry)
                 continue
-            copy_static_entry(entry, path.join(self.outdir, '_static'), self,
-                              ctx, exclude_matchers=matchers)
+            copy_asset(entry, path.join(self.outdir, '_static'), excluded,
+                       context=ctx, renderer=self.templates)
         # copy logo and favicon files if not already in static path
         if self.config.html_logo:
             logobase = path.basename(self.config.html_logo)
@@ -642,14 +649,15 @@ class StandaloneHTMLBuilder(Builder):
     def copy_extra_files(self):
         # copy html_extra_path files
         self.info(bold('copying extra files... '), nonl=True)
-        extraentries = [path.join(self.confdir, epath)
-                        for epath in self.config.html_extra_path]
-        matchers = compile_matchers(self.config.exclude_patterns)
-        for entry in extraentries:
+        excluded = Matcher(self.config.exclude_patterns)
+
+        for extra_path in self.config.html_extra_path:
+            entry = path.join(self.confdir, extra_path)
             if not path.exists(entry):
                 self.warn('html_extra_path entry %r does not exist' % entry)
                 continue
-            copy_extra_entry(entry, self.outdir, matchers)
+
+            copy_asset(entry, self.outdir, excluded)
         self.info('done')
 
     def write_buildinfo(self):
@@ -712,7 +720,12 @@ class StandaloneHTMLBuilder(Builder):
     def index_page(self, pagename, doctree, title):
         # only index pages with title
         if self.indexer is not None and title:
-            self.indexer.feed(pagename, title, doctree)
+            filename = self.env.doc2path(pagename, base=None)
+            try:
+                self.indexer.feed(pagename, filename, title, doctree)
+            except TypeError:
+                # fallback for old search-adapters
+                self.indexer.feed(pagename, title, doctree)
 
     def _get_local_toctree(self, docname, collapse=True, **kwds):
         if 'includehidden' not in kwds:
@@ -1056,6 +1069,7 @@ class SerializingHTMLBuilder(StandaloneHTMLBuilder):
         self.theme = None       # no theme necessary
         self.templates = None   # no template bridge necessary
         self.init_translator_class()
+        self.init_templates()
         self.init_highlighter()
 
     def get_target_uri(self, docname, typ=None):
@@ -1148,3 +1162,52 @@ class JSONHTMLBuilder(SerializingHTMLBuilder):
 
     def init(self):
         SerializingHTMLBuilder.init(self)
+
+
+def setup(app):
+    # builders
+    app.add_builder(StandaloneHTMLBuilder)
+    app.add_builder(DirectoryHTMLBuilder)
+    app.add_builder(SingleFileHTMLBuilder)
+    app.add_builder(PickleHTMLBuilder)
+    app.add_builder(JSONHTMLBuilder)
+
+    # config values
+    app.add_config_value('html_theme', 'alabaster', 'html')
+    app.add_config_value('html_theme_path', [], 'html')
+    app.add_config_value('html_theme_options', {}, 'html')
+    app.add_config_value('html_title',
+                         lambda self: l_('%s %s documentation') % (self.project, self.release),
+                         'html', string_classes)
+    app.add_config_value('html_short_title', lambda self: self.html_title, 'html')
+    app.add_config_value('html_style', None, 'html', string_classes)
+    app.add_config_value('html_logo', None, 'html', string_classes)
+    app.add_config_value('html_favicon', None, 'html', string_classes)
+    app.add_config_value('html_static_path', [], 'html')
+    app.add_config_value('html_extra_path', [], 'html')
+    app.add_config_value('html_last_updated_fmt', None, 'html', string_classes)
+    app.add_config_value('html_use_smartypants', True, 'html')
+    app.add_config_value('html_translator_class', None, 'html', string_classes)
+    app.add_config_value('html_sidebars', {}, 'html')
+    app.add_config_value('html_additional_pages', {}, 'html')
+    app.add_config_value('html_use_modindex', True, 'html')  # deprecated
+    app.add_config_value('html_domain_indices', True, 'html', [list])
+    app.add_config_value('html_add_permalinks', u'\u00B6', 'html')
+    app.add_config_value('html_use_index', True, 'html')
+    app.add_config_value('html_split_index', False, 'html')
+    app.add_config_value('html_copy_source', True, 'html')
+    app.add_config_value('html_show_sourcelink', True, 'html')
+    app.add_config_value('html_sourcelink_suffix', '.txt', 'html')
+    app.add_config_value('html_use_opensearch', '', 'html')
+    app.add_config_value('html_file_suffix', None, 'html', string_classes)
+    app.add_config_value('html_link_suffix', None, 'html', string_classes)
+    app.add_config_value('html_show_copyright', True, 'html')
+    app.add_config_value('html_show_sphinx', True, 'html')
+    app.add_config_value('html_context', {}, 'html')
+    app.add_config_value('html_output_encoding', 'utf-8', 'html')
+    app.add_config_value('html_compact_lists', True, 'html')
+    app.add_config_value('html_secnumber_suffix', '. ', 'html')
+    app.add_config_value('html_search_language', None, 'html', string_classes)
+    app.add_config_value('html_search_options', {}, 'html')
+    app.add_config_value('html_search_scorer', '', None)
+    app.add_config_value('html_scaled_image_link', True, 'html')

sphinx/builders/htmlhelp.py

@@ -18,6 +18,7 @@ from os import path
 from docutils import nodes
 
 from sphinx import addnodes
+from sphinx.util.osutil import make_filename
 from sphinx.builders.html import StandaloneHTMLBuilder
 from sphinx.util.pycompat import htmlescape
 
@@ -63,7 +64,7 @@ Binary Index=No
 Compiled file=%(outname)s.chm
 Contents file=%(outname)s.hhc
 Default Window=%(outname)s
-Default topic=index.html
+Default topic=%(master_doc)s
 Display compile progress=No
 Full text search stop list file=%(outname)s.stp
 Full-text search=Yes
@@ -73,7 +74,7 @@ Title=%(title)s
 
 [WINDOWS]
 %(outname)s="%(title)s","%(outname)s.hhc","%(outname)s.hhk",\
-"index.html","index.html",,,,,0x63520,220,0x10384e,[0,0,1024,768],,,,,,,0
+"%(master_doc)s","%(master_doc)s",,,,,0x63520,220,0x10384e,[0,0,1024,768],,,,,,,0
 
 [FILES]
 '''
@@ -183,6 +184,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
         StandaloneHTMLBuilder.init(self)
         # the output files for HTML help must be .html only
         self.out_suffix = '.html'
+        self.link_suffix = '.html'
         # determine the correct locale setting
         locale = chm_locales.get(self.config.language)
         if locale is not None:
@@ -204,11 +206,14 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
 
         self.info('writing project file...')
         with self.open_file(outdir, outname+'.hhp') as f:
-            f.write(project_template % {'outname': outname,
-                                        'title': self.config.html_title,
-                                        'version': self.config.version,
-                                        'project': self.config.project,
-                                        'lcid': self.lcid})
+            f.write(project_template % {
+                'outname': outname,
+                'title': self.config.html_title,
+                'version': self.config.version,
+                'project': self.config.project,
+                'lcid': self.lcid,
+                'master_doc': self.config.master_doc + self.out_suffix
+            })
             if not outdir.endswith(os.sep):
                 outdir += os.sep
             olen = len(outdir)
@@ -225,7 +230,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
             f.write(contents_header)
             # special books
             f.write('<LI> ' + object_sitemap % (self.config.html_short_title,
-                                                'index.html'))
+                                                self.config.master_doc + self.out_suffix))
             for indexname, indexcls, content, collapse in self.domain_indices:
                 f.write('<LI> ' + object_sitemap % (indexcls.localname,
                                                     '%s.html' % indexname))
@@ -292,3 +297,10 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
                 for title, (refs, subitems, key_) in group:
                     write_index(title, refs, subitems)
             f.write('</UL>\n')
+
+
+def setup(app):
+    app.setup_extension('sphinx.builders.html')
+    app.add_builder(HTMLHelpBuilder)
+
+    app.add_config_value('htmlhelp_basename', lambda self: make_filename(self.project), None)

sphinx/builders/latex.py

@@ -11,7 +11,6 @@
 
 import os
 from os import path
-import warnings
 
 from six import iteritems
 from docutils import nodes
@@ -19,14 +18,16 @@ from docutils.io import FileOutput
 from docutils.utils import new_document
 from docutils.frontend import OptionParser
 
-from sphinx import package_dir, addnodes
+from sphinx import package_dir, addnodes, highlighting
 from sphinx.util import texescape
+from sphinx.config import string_classes, ENUM
 from sphinx.errors import SphinxError
 from sphinx.locale import _
 from sphinx.builders import Builder
 from sphinx.environment import NoUri
 from sphinx.util.nodes import inline_all_toctrees
-from sphinx.util.osutil import SEP, copyfile
+from sphinx.util.fileutil import copy_asset_file
+from sphinx.util.osutil import SEP, make_filename
 from sphinx.util.console import bold, darkgreen
 from sphinx.writers.latex import LaTeXWriter
 
@@ -38,27 +39,12 @@ class LaTeXBuilder(Builder):
     name = 'latex'
     format = 'latex'
     supported_image_types = ['application/pdf', 'image/png', 'image/jpeg']
-    usepackages = []
 
     def init(self):
         self.docnames = []
         self.document_data = []
+        self.usepackages = []
         texescape.init()
-        self.check_options()
-
-    def check_options(self):
-        if self.config.latex_toplevel_sectioning not in (None, 'part', 'chapter', 'section'):
-            self.warn('invalid latex_toplevel_sectioning, ignored: %s' %
-                      self.config.latex_top_sectionlevel)
-            self.config.latex_top_sectionlevel = None
-
-        if self.config.latex_use_parts:
-            warnings.warn('latex_use_parts will be removed at Sphinx-1.5. '
-                          'Use latex_toplevel_sectioning instead.',
-                          DeprecationWarning)
-
-            if self.config.latex_toplevel_sectioning:
-                self.warn('latex_use_parts conflicts with latex_toplevel_sectioning, ignored.')
 
     def get_outdated_docs(self):
         return 'all documents'  # for now
@@ -92,6 +78,16 @@ class LaTeXBuilder(Builder):
                 docname = docname[:-5]
             self.titles.append((docname, entry[2]))
 
+    def write_stylesheet(self):
+        highlighter = highlighting.PygmentsBridge(
+            'latex', self.config.pygments_style, self.config.trim_doctest_flags)
+        stylesheet = path.join(self.outdir, 'sphinxhighlight.sty')
+        with open(stylesheet, 'w') as f:
+            f.write('\\NeedsTeXFormat{LaTeX2e}[1995/12/01]\n')
+            f.write('\\ProvidesPackage{sphinxhighlight}'
+                    '[2016/05/29 stylesheet for highlighting with pygments]\n\n')
+            f.write(highlighter.get_stylesheet())
+
     def write(self, *ignored):
         docwriter = LaTeXWriter(self)
         docsettings = OptionParser(
@@ -100,6 +96,7 @@ class LaTeXBuilder(Builder):
             read_config_files=True).get_default_values()
 
         self.init_document_data()
+        self.write_stylesheet()
 
         for entry in self.document_data:
             docname, targetname, title, author, docclass = entry[:5]
@@ -192,33 +189,109 @@ class LaTeXBuilder(Builder):
             self.info(bold('copying images...'), nonl=1)
             for src, dest in iteritems(self.images):
                 self.info(' '+src, nonl=1)
-                copyfile(path.join(self.srcdir, src),
-                         path.join(self.outdir, dest))
+                copy_asset_file(path.join(self.srcdir, src),
+                                path.join(self.outdir, dest))
             self.info()
 
         # copy TeX support files from texinputs
+        context = {'latex_engine': self.config.latex_engine}
         self.info(bold('copying TeX support files...'))
         staticdirname = path.join(package_dir, 'texinputs')
         for filename in os.listdir(staticdirname):
             if not filename.startswith('.'):
-                copyfile(path.join(staticdirname, filename),
-                         path.join(self.outdir, filename))
+                copy_asset_file(path.join(staticdirname, filename),
+                                self.outdir, context=context)
 
         # copy additional files
         if self.config.latex_additional_files:
             self.info(bold('copying additional files...'), nonl=1)
             for filename in self.config.latex_additional_files:
                 self.info(' '+filename, nonl=1)
-                copyfile(path.join(self.confdir, filename),
-                         path.join(self.outdir, path.basename(filename)))
+                copy_asset_file(path.join(self.confdir, filename), self.outdir)
             self.info()
 
         # the logo is handled differently
         if self.config.latex_logo:
-            logobase = path.basename(self.config.latex_logo)
-            logotarget = path.join(self.outdir, logobase)
             if not path.isfile(path.join(self.confdir, self.config.latex_logo)):
                 raise SphinxError('logo file %r does not exist' % self.config.latex_logo)
-            elif not path.isfile(logotarget):
-                copyfile(path.join(self.confdir, self.config.latex_logo), logotarget)
+            else:
+                copy_asset_file(path.join(self.confdir, self.config.latex_logo), self.outdir)
         self.info('done')
+
+
+def validate_config_values(app):
+    if app.config.latex_toplevel_sectioning not in (None, 'part', 'chapter', 'section'):
+        app.warn('invalid latex_toplevel_sectioning, ignored: %s' %
+                 app.config.latex_toplevel_sectioning)
+        app.config.latex_toplevel_sectioning = None
+
+    if app.config.latex_use_parts:
+        if app.config.latex_toplevel_sectioning:
+            app.warn('latex_use_parts conflicts with latex_toplevel_sectioning, ignored.')
+        else:
+            app.warn('latex_use_parts is deprecated. Use latex_toplevel_sectioning instead.')
+            app.config.latex_toplevel_sectioning = 'parts'
+
+    if app.config.latex_use_modindex is not True:  # changed by user
+        app.warn('latex_use_modeindex is deprecated. Use latex_domain_indices instead.')
+
+    if app.config.latex_preamble:
+        if app.config.latex_elements.get('preamble'):
+            app.warn("latex_preamble conflicts with latex_elements['preamble'], ignored.")
+        else:
+            app.warn("latex_preamble is deprecated. Use latex_elements['preamble'] instead.")
+            app.config.latex_elements['preamble'] = app.config.latex_preamble
+
+    if app.config.latex_paper_size != 'letter':
+        if app.config.latex_elements.get('papersize'):
+            app.warn("latex_paper_size conflicts with latex_elements['papersize'], ignored.")
+        else:
+            app.warn("latex_paper_size is deprecated. "
+                     "Use latex_elements['papersize'] instead.")
+            if app.config.latex_paper_size:
+                app.config.latex_elements['papersize'] = app.config.latex_paper_size + 'paper'
+
+    if app.config.latex_font_size != '10pt':
+        if app.config.latex_elements.get('pointsize'):
+            app.warn("latex_font_size conflicts with latex_elements['pointsize'], ignored.")
+        else:
+            app.warn("latex_font_size is deprecated. Use latex_elements['pointsize'] instead.")
+            app.config.latex_elements['pointsize'] = app.config.latex_font_size
+
+
+def setup(app):
+    app.add_builder(LaTeXBuilder)
+    app.connect('builder-inited', validate_config_values)
+
+    app.add_config_value('latex_engine',
+                         lambda self: 'pdflatex' if self.language != 'ja' else 'platex',
+                         None,
+                         ENUM('pdflatex', 'xelatex', 'lualatex', 'platex'))
+    app.add_config_value('latex_documents',
+                         lambda self: [(self.master_doc, make_filename(self.project) + '.tex',
+                                        self.project, '', 'manual')],
+                         None)
+    app.add_config_value('latex_logo', None, None, string_classes)
+    app.add_config_value('latex_appendices', [], None)
+    app.add_config_value('latex_keep_old_macro_names', True, None)
+    # now deprecated - use latex_toplevel_sectioning
+    app.add_config_value('latex_use_parts', False, None)
+    app.add_config_value('latex_toplevel_sectioning', None, None, [str])
+    app.add_config_value('latex_use_modindex', True, None)  # deprecated
+    app.add_config_value('latex_domain_indices', True, None, [list])
+    app.add_config_value('latex_show_urls', 'no', None)
+    app.add_config_value('latex_show_pagerefs', False, None)
+    # paper_size and font_size are still separate values
+    # so that you can give them easily on the command line
+    app.add_config_value('latex_paper_size', 'letter', None)
+    app.add_config_value('latex_font_size', '10pt', None)
+    app.add_config_value('latex_elements', {}, None)
+    app.add_config_value('latex_additional_files', [], None)
+
+    japanese_default = {'manual': 'jsbook',
+                        'howto': 'jreport'}
+    app.add_config_value('latex_docclass',
+                         lambda self: japanese_default if self.language == 'ja' else {},
+                         None)
+    # now deprecated - use latex_elements
+    app.add_config_value('latex_preamble', '', None)

sphinx/builders/linkcheck.py

@@ -54,6 +54,13 @@ except pkg_resources.DistributionNotFound:
             'install "requests[security]" as a dependency or upgrade to '
             'a python version with SNI support (Python 3 and Python 2.7.9+).'
         )
+except pkg_resources.UnknownExtra:
+    warnings.warn(
+        'Some links may return broken results due to being unable to '
+        'check the Server Name Indication (SNI) in the returned SSL cert '
+        'against the hostname in the url requested. Recommended to '
+        'install requests-2.4.1+.'
+    )
 
 requests_user_agent = [('User-agent', 'Mozilla/5.0 (X11; Linux x86_64; rv:25.0) '
                         'Gecko/20100101 Firefox/25.0')]
@@ -300,3 +307,13 @@ class CheckExternalLinksBuilder(Builder):
     def finish(self):
         for worker in self.workers:
             self.wqueue.put((None, None, None), False)
+
+
+def setup(app):
+    app.add_builder(CheckExternalLinksBuilder)
+
+    app.add_config_value('linkcheck_ignore', [], None)
+    app.add_config_value('linkcheck_retries', 1, None)
+    app.add_config_value('linkcheck_timeout', None, None, [int])
+    app.add_config_value('linkcheck_workers', 5, None)
+    app.add_config_value('linkcheck_anchors', True, None)

sphinx/builders/manpage.py

@@ -19,6 +19,7 @@ from sphinx import addnodes
 from sphinx.builders import Builder
 from sphinx.environment import NoUri
 from sphinx.util.nodes import inline_all_toctrees
+from sphinx.util.osutil import make_filename
 from sphinx.util.console import bold, darkgreen
 from sphinx.writers.manpage import ManualPageWriter
 
@@ -88,3 +89,13 @@ class ManualPageBuilder(Builder):
 
     def finish(self):
         pass
+
+
+def setup(app):
+    app.add_builder(ManualPageBuilder)
+
+    app.add_config_value('man_pages',
+                         lambda self: [(self.master_doc, make_filename(self.project).lower(),
+                                        '%s %s' % (self.project, self.release), [], 1)],
+                         None)
+    app.add_config_value('man_show_urls', False, None)

sphinx/builders/qthelp.py

@@ -21,6 +21,7 @@ from docutils import nodes
 from sphinx import addnodes
 from sphinx.builders.html import StandaloneHTMLBuilder
 from sphinx.util import force_decode
+from sphinx.util.osutil import make_filename
 from sphinx.util.pycompat import htmlescape
 
 
@@ -104,15 +105,25 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
 
     # don't add links
     add_permalinks = False
+
     # don't add sidebar etc.
     embedded = True
+    # disable download role
+    download_support = False
+
+    # don't generate the search index or include the search page
+    search = False
 
     def init(self):
         StandaloneHTMLBuilder.init(self)
         # the output files for HTML help must be .html only
         self.out_suffix = '.html'
+        self.link_suffix = '.html'
         # self.config.html_style = 'traditional.css'
 
+    def get_theme_config(self):
+        return self.config.qthelp_theme, self.config.qthelp_theme_options
+
     def handle_finish(self):
         self.build_qhp(self.outdir, self.config.qthelp_basename)
 
@@ -290,3 +301,12 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
                 keywords.extend(self.build_keywords(subitem[0], subitem[1], []))
 
         return keywords
+
+
+def setup(app):
+    app.setup_extension('sphinx.builders.html')
+    app.add_builder(QtHelpBuilder)
+
+    app.add_config_value('qthelp_basename', lambda self: make_filename(self.project), None)
+    app.add_config_value('qthelp_theme', 'nonav', 'html')
+    app.add_config_value('qthelp_theme_options', {}, 'html')

sphinx/builders/texinfo.py

@@ -22,7 +22,7 @@ from sphinx.locale import _
 from sphinx.builders import Builder
 from sphinx.environment import NoUri
 from sphinx.util.nodes import inline_all_toctrees
-from sphinx.util.osutil import SEP, copyfile
+from sphinx.util.osutil import SEP, copyfile, make_filename
 from sphinx.util.console import bold, darkgreen
 from sphinx.writers.texinfo import TexinfoWriter
 
@@ -225,3 +225,20 @@ class TexinfoBuilder(Builder):
         except (IOError, OSError) as err:
             self.warn("error writing file %s: %s" % (fn, err))
         self.info(' done')
+
+
+def setup(app):
+    app.add_builder(TexinfoBuilder)
+
+    app.add_config_value('texinfo_documents',
+                         lambda self: [(self.master_doc, make_filename(self.project).lower(),
+                                        self.project, '', make_filename(self.project),
+                                        'The %s reference manual.' %
+                                        make_filename(self.project),
+                                        'Python')],
+                         None)
+    app.add_config_value('texinfo_appendices', [], None)
+    app.add_config_value('texinfo_elements', {}, None)
+    app.add_config_value('texinfo_domain_indices', True, None, [list])
+    app.add_config_value('texinfo_show_urls', 'footnote', None)
+    app.add_config_value('texinfo_no_detailmenu', False, None)

sphinx/builders/text.py

@@ -67,3 +67,10 @@ class TextBuilder(Builder):
 
     def finish(self):
         pass
+
+
+def setup(app):
+    app.add_builder(TextBuilder)
+
+    app.add_config_value('text_sectionchars', '*=-~"+`', 'env')
+    app.add_config_value('text_newlines', 'unix', 'env')

sphinx/builders/websupport.py

@@ -165,3 +165,7 @@ class WebSupportBuilder(PickleHTMLBuilder):
 
     def dump_search_index(self):
         self.indexer.finish_indexing()
+
+
+def setup(app):
+    app.add_builder(WebSupportBuilder)

sphinx/builders/xml.py

@@ -95,3 +95,10 @@ class PseudoXMLBuilder(XMLBuilder):
     out_suffix = '.pseudoxml'
 
     _writer_class = PseudoXMLWriter
+
+
+def setup(app):
+    app.add_builder(XMLBuilder)
+    app.add_builder(PseudoXMLBuilder)
+
+    app.add_config_value('xml_pretty', True, 'env')

sphinx/config.py

@@ -10,14 +10,13 @@
 """
 
 import re
-from os import path, environ, getenv
-import shlex
+from os import path, getenv
 
 from six import PY2, PY3, iteritems, string_types, binary_type, text_type, integer_types
 
 from sphinx.errors import ConfigError
 from sphinx.locale import l_
-from sphinx.util.osutil import make_filename, cd
+from sphinx.util.osutil import cd
 from sphinx.util.pycompat import execfile_, NoneType
 from sphinx.util.i18n import format_date
 
@@ -29,10 +28,25 @@ if PY3:
     CONFIG_SYNTAX_ERROR += "\nDid you change the syntax from 2.x to 3.x?"
 CONFIG_EXIT_ERROR = "The configuration file (or one of the modules it imports) " \
                     "called sys.exit()"
+CONFIG_ENUM_WARNING = "The config value `{name}` has to be a one of {candidates}, " \
+                      "but `{current}` is given."
 CONFIG_TYPE_WARNING = "The config value `{name}' has type `{current.__name__}', " \
                       "defaults to `{default.__name__}.'"
 
 
+class ENUM:
+    """represents the config value should be a one of candidates.
+
+    Example:
+        app.add_config_value('latex_show_urls', 'no', ENUM('no', 'footnote', 'inline'))
+    """
+    def __init__(self, *candidates):
+        self.candidates = candidates
+
+    def match(self, value):
+        return value in self.candidates
+
+
 string_classes = [text_type]
 if PY2:
     string_classes.append(binary_type)  # => [str, unicode]
@@ -94,182 +108,6 @@ class Config(object):
                           'table': l_('Table %s'),
                           'code-block': l_('Listing %s')},
                          'env'),
-
-        # HTML options
-        html_theme = ('alabaster', 'html'),
-        html_theme_path = ([], 'html'),
-        html_theme_options = ({}, 'html'),
-        html_title = (lambda self: l_('%s %s documentation') %
-                      (self.project, self.release),
-                      'html', string_classes),
-        html_short_title = (lambda self: self.html_title, 'html'),
-        html_style = (None, 'html', string_classes),
-        html_logo = (None, 'html', string_classes),
-        html_favicon = (None, 'html', string_classes),
-        html_static_path = ([], 'html'),
-        html_extra_path = ([], 'html'),
-        # the real default is locale-dependent
-        html_last_updated_fmt = (None, 'html', string_classes),
-        html_use_smartypants = (True, 'html'),
-        html_translator_class = (None, 'html', string_classes),
-        html_sidebars = ({}, 'html'),
-        html_additional_pages = ({}, 'html'),
-        html_use_modindex = (True, 'html'),  # deprecated
-        html_domain_indices = (True, 'html', [list]),
-        html_add_permalinks = (u'\u00B6', 'html'),
-        html_use_index = (True, 'html'),
-        html_split_index = (False, 'html'),
-        html_copy_source = (True, 'html'),
-        html_show_sourcelink = (True, 'html'),
-        html_use_opensearch = ('', 'html'),
-        html_file_suffix = (None, 'html', string_classes),
-        html_link_suffix = (None, 'html', string_classes),
-        html_show_copyright = (True, 'html'),
-        html_show_sphinx = (True, 'html'),
-        html_context = ({}, 'html'),
-        html_output_encoding = ('utf-8', 'html'),
-        html_compact_lists = (True, 'html'),
-        html_secnumber_suffix = ('. ', 'html'),
-        html_search_language = (None, 'html', string_classes),
-        html_search_options = ({}, 'html'),
-        html_search_scorer = ('', None),
-        html_scaled_image_link = (True, 'html'),
-
-        # HTML help only options
-        htmlhelp_basename = (lambda self: make_filename(self.project), None),
-
-        # Qt help only options
-        qthelp_basename = (lambda self: make_filename(self.project), None),
-
-        # Devhelp only options
-        devhelp_basename = (lambda self: make_filename(self.project), None),
-
-        # Apple help options
-        applehelp_bundle_name = (lambda self: make_filename(self.project),
-                                 'applehelp'),
-        applehelp_bundle_id = (None, 'applehelp', string_classes),
-        applehelp_dev_region = ('en-us', 'applehelp'),
-        applehelp_bundle_version = ('1', 'applehelp'),
-        applehelp_icon = (None, 'applehelp', string_classes),
-        applehelp_kb_product = (lambda self: '%s-%s' %
-                                (make_filename(self.project), self.release),
-                                'applehelp'),
-        applehelp_kb_url = (None, 'applehelp', string_classes),
-        applehelp_remote_url = (None, 'applehelp', string_classes),
-        applehelp_index_anchors = (False, 'applehelp', string_classes),
-        applehelp_min_term_length = (None, 'applehelp', string_classes),
-        applehelp_stopwords = (lambda self: self.language or 'en', 'applehelp'),
-        applehelp_locale = (lambda self: self.language or 'en', 'applehelp'),
-        applehelp_title = (lambda self: self.project + ' Help', 'applehelp'),
-        applehelp_codesign_identity = (lambda self:
-                                       environ.get('CODE_SIGN_IDENTITY', None),
-                                       'applehelp'),
-        applehelp_codesign_flags = (lambda self:
-                                    shlex.split(
-                                        environ.get('OTHER_CODE_SIGN_FLAGS',
-                                                    '')),
-                                    'applehelp'),
-        applehelp_indexer_path = ('/usr/bin/hiutil', 'applehelp'),
-        applehelp_codesign_path = ('/usr/bin/codesign', 'applehelp'),
-        applehelp_disable_external_tools = (False, None),
-
-        # Epub options
-        epub_basename = (lambda self: make_filename(self.project), None),
-        epub_theme = ('epub', 'html'),
-        epub_theme_options = ({}, 'html'),
-        epub_title = (lambda self: self.html_title, 'html'),
-        epub3_description = ('', 'epub3', string_classes),
-        epub_author = ('unknown', 'html'),
-        epub3_contributor = ('unknown', 'epub3', string_classes),
-        epub_language = (lambda self: self.language or 'en', 'html'),
-        epub_publisher = ('unknown', 'html'),
-        epub_copyright = (lambda self: self.copyright, 'html'),
-        epub_identifier = ('unknown', 'html'),
-        epub_scheme = ('unknown', 'html'),
-        epub_uid = ('unknown', 'env'),
-        epub_cover = ((), 'env'),
-        epub_guide = ((), 'env'),
-        epub_pre_files = ([], 'env'),
-        epub_post_files = ([], 'env'),
-        epub_exclude_files = ([], 'env'),
-        epub_tocdepth = (3, 'env'),
-        epub_tocdup = (True, 'env'),
-        epub_tocscope = ('default', 'env'),
-        epub_fix_images = (False, 'env'),
-        epub_max_image_width = (0, 'env'),
-        epub_show_urls = ('inline', 'html'),
-        epub_use_index = (lambda self: self.html_use_index, 'html'),
-        epub3_page_progression_direction = ('ltr', 'epub3', string_classes),
-
-        # LaTeX options
-        latex_documents = (lambda self: [(self.master_doc,
-                                          make_filename(self.project) + '.tex',
-                                          self.project,
-                                          '', 'manual')],
-                           None),
-        latex_logo = (None, None, string_classes),
-        latex_appendices = ([], None),
-        latex_keep_old_macro_names = (True, None),
-        # now deprecated - use latex_toplevel_sectioning
-        latex_use_parts = (False, None),
-        latex_toplevel_sectioning = (None, None, [str]),
-        latex_use_modindex = (True, None),  # deprecated
-        latex_domain_indices = (True, None, [list]),
-        latex_show_urls = ('no', None),
-        latex_show_pagerefs = (False, None),
-        # paper_size and font_size are still separate values
-        # so that you can give them easily on the command line
-        latex_paper_size = ('letter', None),
-        latex_font_size = ('10pt', None),
-        latex_elements = ({}, None),
-        latex_additional_files = ([], None),
-        latex_docclass = ({}, None),
-        # now deprecated - use latex_elements
-        latex_preamble = ('', None),
-
-        # text options
-        text_sectionchars = ('*=-~"+`', 'env'),
-        text_newlines = ('unix', 'env'),
-
-        # manpage options
-        man_pages = (lambda self: [(self.master_doc,
-                                    make_filename(self.project).lower(),
-                                    '%s %s' % (self.project, self.release),
-                                    [], 1)],
-                     None),
-        man_show_urls = (False, None),
-
-        # Texinfo options
-        texinfo_documents = (lambda self: [(self.master_doc,
-                                            make_filename(self.project).lower(),
-                                            self.project, '',
-                                            make_filename(self.project),
-                                            'The %s reference manual.' %
-                                            make_filename(self.project),
-                                            'Python')],
-                             None),
-        texinfo_appendices = ([], None),
-        texinfo_elements = ({}, None),
-        texinfo_domain_indices = (True, None, [list]),
-        texinfo_show_urls = ('footnote', None),
-        texinfo_no_detailmenu = (False, None),
-
-        # linkcheck options
-        linkcheck_ignore = ([], None),
-        linkcheck_retries = (1, None),
-        linkcheck_timeout = (None, None, [int]),
-        linkcheck_workers = (5, None),
-        linkcheck_anchors = (True, None),
-
-        # gettext options
-        gettext_compact = (True, 'gettext'),
-        gettext_location = (True, 'gettext'),
-        gettext_uuid = (False, 'gettext'),
-        gettext_auto_build = (True, 'env'),
-        gettext_additional_targets = ([], 'env'),
-
-        # XML options
-        xml_pretty = (True, 'env'),
     )
 
     def __init__(self, dirname, filename, overrides, tags):
@@ -327,19 +165,24 @@ class Config(object):
             if default is None and not permitted:
                 continue  # neither inferrable nor expliclitly permitted types
             current = self[name]
-            if type(current) is type(default):
-                continue
-            if type(current) in permitted:
-                continue
+            if isinstance(permitted, ENUM):
+                if not permitted.match(current):
+                    warn(CONFIG_ENUM_WARNING.format(
+                        name=name, current=current, candidates=permitted.candidates))
+            else:
+                if type(current) is type(default):
+                    continue
+                if type(current) in permitted:
+                    continue
 
-            common_bases = (set(type(current).__bases__ + (type(current),)) &
-                            set(type(default).__bases__))
-            common_bases.discard(object)
-            if common_bases:
-                continue  # at least we share a non-trivial base class
+                common_bases = (set(type(current).__bases__ + (type(current),)) &
+                                set(type(default).__bases__))
+                common_bases.discard(object)
+                if common_bases:
+                    continue  # at least we share a non-trivial base class
 
-            warn(CONFIG_TYPE_WARNING.format(
-                name=name, current=type(current), default=type(default)))
+                warn(CONFIG_TYPE_WARNING.format(
+                    name=name, current=type(current), default=type(default)))
 
     def check_unicode(self, warn):
         # check all string values for non-ASCII characters in bytestrings,

sphinx/directives/__init__.py

@@ -17,11 +17,6 @@ from docutils.parsers.rst import Directive, directives, roles
 from sphinx import addnodes
 from sphinx.util.docfields import DocFieldTransformer
 
-# import and register directives
-from sphinx.directives.code import *   # noqa
-from sphinx.directives.other import *  # noqa
-from sphinx.directives.patches import *  # noqa
-
 
 # RE to strip backslash escapes
 nl_escape_re = re.compile(r'\\\n')
@@ -216,8 +211,9 @@ class DefaultDomain(Directive):
         return []
 
 
-directives.register_directive('default-role', DefaultRole)
-directives.register_directive('default-domain', DefaultDomain)
-directives.register_directive('describe', ObjectDescription)
-# new, more consistent, name
-directives.register_directive('object', ObjectDescription)
+def setup(app):
+    directives.register_directive('default-role', DefaultRole)
+    directives.register_directive('default-domain', DefaultDomain)
+    directives.register_directive('describe', ObjectDescription)
+    # new, more consistent, name
+    directives.register_directive('object', ObjectDescription)

sphinx/directives/code.py

@@ -342,8 +342,9 @@ class LiteralInclude(Directive):
         return [retnode]
 
 
-directives.register_directive('highlight', Highlight)
-directives.register_directive('highlightlang', Highlight)  # old
-directives.register_directive('code-block', CodeBlock)
-directives.register_directive('sourcecode', CodeBlock)
-directives.register_directive('literalinclude', LiteralInclude)
+def setup(app):
+    directives.register_directive('highlight', Highlight)
+    directives.register_directive('highlightlang', Highlight)  # old
+    directives.register_directive('code-block', CodeBlock)
+    directives.register_directive('sourcecode', CodeBlock)
+    directives.register_directive('literalinclude', LiteralInclude)

sphinx/directives/other.py

@@ -409,24 +409,25 @@ class Include(BaseInclude):
         return BaseInclude.run(self)
 
 
-directives.register_directive('toctree', TocTree)
-directives.register_directive('sectionauthor', Author)
-directives.register_directive('moduleauthor', Author)
-directives.register_directive('codeauthor', Author)
-directives.register_directive('index', Index)
-directives.register_directive('deprecated', VersionChange)
-directives.register_directive('versionadded', VersionChange)
-directives.register_directive('versionchanged', VersionChange)
-directives.register_directive('seealso', SeeAlso)
-directives.register_directive('tabularcolumns', TabularColumns)
-directives.register_directive('centered', Centered)
-directives.register_directive('acks', Acks)
-directives.register_directive('hlist', HList)
-directives.register_directive('only', Only)
-directives.register_directive('include', Include)
+def setup(app):
+    directives.register_directive('toctree', TocTree)
+    directives.register_directive('sectionauthor', Author)
+    directives.register_directive('moduleauthor', Author)
+    directives.register_directive('codeauthor', Author)
+    directives.register_directive('index', Index)
+    directives.register_directive('deprecated', VersionChange)
+    directives.register_directive('versionadded', VersionChange)
+    directives.register_directive('versionchanged', VersionChange)
+    directives.register_directive('seealso', SeeAlso)
+    directives.register_directive('tabularcolumns', TabularColumns)
+    directives.register_directive('centered', Centered)
+    directives.register_directive('acks', Acks)
+    directives.register_directive('hlist', HList)
+    directives.register_directive('only', Only)
+    directives.register_directive('include', Include)
 
-# register the standard rst class directive under a different name
-# only for backwards compatibility now
-directives.register_directive('cssclass', Class)
-# new standard name when default-domain with "class" is in effect
-directives.register_directive('rst-class', Class)
+    # register the standard rst class directive under a different name
+    # only for backwards compatibility now
+    directives.register_directive('cssclass', Class)
+    # new standard name when default-domain with "class" is in effect
+    directives.register_directive('rst-class', Class)

sphinx/directives/patches.py

@@ -35,4 +35,5 @@ class Figure(images.Figure):
         return [figure_node]
 
 
-directives.register_directive('figure', Figure)
+def setup(app):
+    directives.register_directive('figure', Figure)

sphinx/domains/__init__.py

@@ -273,20 +273,3 @@ class Domain(object):
         if primary:
             return type.lname
         return _('%s %s') % (self.label, type.lname)
-
-
-from sphinx.domains.c import CDomain                     # noqa
-from sphinx.domains.cpp import CPPDomain                 # noqa
-from sphinx.domains.std import StandardDomain            # noqa
-from sphinx.domains.python import PythonDomain           # noqa
-from sphinx.domains.javascript import JavaScriptDomain   # noqa
-from sphinx.domains.rst import ReSTDomain                # noqa
-
-BUILTIN_DOMAINS = {
-    'std': StandardDomain,
-    'py': PythonDomain,
-    'c': CDomain,
-    'cpp': CPPDomain,
-    'js': JavaScriptDomain,
-    'rst': ReSTDomain,
-}

sphinx/domains/c.py

@@ -279,6 +279,9 @@ class CDomain(Domain):
                      typ, target, node, contnode):
         # strip pointer asterisk
         target = target.rstrip(' *')
+        # becase TypedField can generate xrefs
+        if target in CObject.stopwords:
+            return contnode
         if target not in self.data['objects']:
             return None
         obj = self.data['objects'][target]
@@ -299,3 +302,7 @@ class CDomain(Domain):
     def get_objects(self):
         for refname, (docname, type) in list(self.data['objects'].items()):
             yield (refname, refname, type, docname, 'c.' + refname, 1)
+
+
+def setup(app):
+    app.add_domain(CDomain)

sphinx/domains/cpp.py

@@ -2018,7 +2018,7 @@ class ASTTypeWithInit(ASTBase):
 
     def get_id_v2(self, objectType=None, symbol=None):
         if objectType == 'member':
-            return symbol.declaration.name.get_id_v2()
+            return symbol.get_full_nested_name().get_id_v2()
         else:
             return self.type.get_id_v2()
 
@@ -4272,3 +4272,7 @@ class CPPDomain(Domain):
             docname = symbol.docname
             newestId = symbol.declaration.get_newest_id()
             yield (name, name, objectType, docname, newestId, 1)
+
+
+def setup(app):
+    app.add_domain(CPPDomain)

sphinx/domains/javascript.py

@@ -234,3 +234,7 @@ class JavaScriptDomain(Domain):
         for refname, (docname, type) in list(self.data['objects'].items()):
             yield refname, refname, type, docname, \
                 refname.replace('$', '_S_'), 1
+
+
+def setup(app):
+    app.add_domain(JavaScriptDomain)

sphinx/domains/python.py

@@ -101,11 +101,36 @@ class PyXrefMixin(object):
                 break
         return result
 
+    def make_xrefs(self, rolename, domain, target, innernode=nodes.emphasis,
+                   contnode=None):
+        delims = '(\s*[\[\]\(\),]\s*)'
+        delims_re = re.compile(delims)
+        sub_targets = re.split(delims, target)
+
+        split_contnode = bool(contnode and contnode.astext() == target)
+
+        results = []
+        for sub_target in sub_targets:
+            if split_contnode:
+                contnode = nodes.Text(sub_target)
+
+            if delims_re.match(sub_target):
+                results.append(contnode or innernode(sub_target, sub_target))
+            else:
+                results.append(self.make_xref(rolename, domain, sub_target,
+                                              innernode, contnode))
+
+        return results
+
 
 class PyField(PyXrefMixin, Field):
     pass
 
 
+class PyGroupedField(PyXrefMixin, GroupedField):
+    pass
+
+
 class PyTypedField(PyXrefMixin, TypedField):
     pass
 
@@ -130,9 +155,9 @@ class PyObject(ObjectDescription):
                      names=('var', 'ivar', 'cvar'),
                      typerolename='obj', typenames=('vartype',),
                      can_collapse=True),
-        GroupedField('exceptions', label=l_('Raises'), rolename='exc',
-                     names=('raises', 'raise', 'exception', 'except'),
-                     can_collapse=True),
+        PyGroupedField('exceptions', label=l_('Raises'), rolename='exc',
+                       names=('raises', 'raise', 'exception', 'except'),
+                       can_collapse=True),
         Field('returnvalue', label=l_('Returns'), has_arg=False,
               names=('returns', 'return')),
         PyField('returntype', label=l_('Return type'), has_arg=False,
@@ -771,3 +796,7 @@ class PythonDomain(Domain):
         for refname, (docname, type) in iteritems(self.data['objects']):
             if type != 'module':  # modules are already handled
                 yield (refname, refname, type, docname, refname, 1)
+
+
+def setup(app):
+    app.add_domain(PythonDomain)

sphinx/domains/rst.py

@@ -156,3 +156,7 @@ class ReSTDomain(Domain):
     def get_objects(self):
         for (typ, name), docname in iteritems(self.data['objects']):
             yield name, name, typ, docname, typ + '-' + name, 1
+
+
+def setup(app):
+    app.add_domain(ReSTDomain)

sphinx/domains/std.py

@@ -14,7 +14,6 @@ import unicodedata
 
 from six import iteritems
 from docutils import nodes
-from docutils.nodes import fully_normalize_name
 from docutils.parsers.rst import directives
 from docutils.statemachine import ViewList
 
@@ -585,106 +584,119 @@ class StandardDomain(Domain):
         newnode.append(innernode)
         return newnode
 
-    def resolve_xref(self, env, fromdocname, builder,
-                     typ, target, node, contnode):
+    def resolve_xref(self, env, fromdocname, builder, typ, target, node, contnode):
         if typ == 'ref':
-            if node['refexplicit']:
-                # reference to anonymous label; the reference uses
-                # the supplied link caption
-                docname, labelid = self.data['anonlabels'].get(target, ('', ''))
-                sectname = node.astext()
-            else:
-                # reference to named label; the final node will
-                # contain the section name after the label
-                docname, labelid, sectname = self.data['labels'].get(target,
-                                                                     ('', '', ''))
-            if not docname:
-                return None
-
-            return self.build_reference_node(fromdocname, builder,
-                                             docname, labelid, sectname, 'ref')
+            resolver = self._resolve_ref_xref
         elif typ == 'numref':
+            resolver = self._resolve_numref_xref
+        elif typ == 'keyword':
+            resolver = self._resolve_keyword_xref
+        elif typ == 'option':
+            resolver = self._resolve_option_xref
+        else:
+            resolver = self._resolve_obj_xref
+
+        return resolver(env, fromdocname, builder, typ, target, node, contnode)
+
+    def _resolve_ref_xref(self, env, fromdocname, builder, typ, target, node, contnode):
+        if node['refexplicit']:
+            # reference to anonymous label; the reference uses
+            # the supplied link caption
             docname, labelid = self.data['anonlabels'].get(target, ('', ''))
-            if not docname:
-                return None
+            sectname = node.astext()
+        else:
+            # reference to named label; the final node will
+            # contain the section name after the label
+            docname, labelid, sectname = self.data['labels'].get(target,
+                                                                 ('', '', ''))
+        if not docname:
+            return None
 
-            if env.config.numfig is False:
-                env.warn(fromdocname, 'numfig is disabled. :numref: is ignored.',
-                         lineno=node.line)
-                return contnode
+        return self.build_reference_node(fromdocname, builder,
+                                         docname, labelid, sectname, 'ref')
 
-            target_node = env.get_doctree(docname).ids.get(labelid)
-            figtype = self.get_figtype(target_node)
-            if figtype is None:
-                return None
+    def _resolve_numref_xref(self, env, fromdocname, builder, typ, target, node, contnode):
+        docname, labelid = self.data['anonlabels'].get(target, ('', ''))
+        if not docname:
+            return None
 
-            try:
-                figure_id = target_node['ids'][0]
-                fignumber = env.toc_fignumbers[docname][figtype][figure_id]
-            except (KeyError, IndexError):
-                # target_node is found, but fignumber is not assigned.
-                # Maybe it is defined in orphaned document.
-                env.warn(fromdocname, "no number is assigned for %s: %s" % (figtype, labelid),
-                         lineno=node.line)
-                return contnode
+        if env.config.numfig is False:
+            env.warn_node('numfig is disabled. :numref: is ignored.', node)
+            return contnode
 
-            title = contnode.astext()
-            if target == fully_normalize_name(title):
+        target_node = env.get_doctree(docname).ids.get(labelid)
+        figtype = self.get_figtype(target_node)
+        if figtype is None:
+            return None
+
+        try:
+            figure_id = target_node['ids'][0]
+            fignumber = env.toc_fignumbers[docname][figtype][figure_id]
+        except (KeyError, IndexError):
+            # target_node is found, but fignumber is not assigned.
+            # Maybe it is defined in orphaned document.
+            env.warn_node("no number is assigned for %s: %s" % (figtype, labelid), node)
+            return contnode
+
+        try:
+            if node['refexplicit']:
+                title = contnode.astext()
+            else:
                 title = env.config.numfig_format.get(figtype, '')
 
-            try:
-                newtitle = title % '.'.join(map(str, fignumber))
-            except TypeError:
-                env.warn(fromdocname, 'invalid numfig_format: %s' % title,
-                         lineno=node.line)
-                return None
+            newtitle = title % '.'.join(map(str, fignumber))
+        except TypeError:
+            env.warn_node('invalid numfig_format: %s' % title, node)
+            return None
 
-            return self.build_reference_node(fromdocname, builder,
-                                             docname, labelid, newtitle, 'numref',
-                                             nodeclass=addnodes.number_reference,
-                                             title=title)
-        elif typ == 'keyword':
-            # keywords are oddballs: they are referenced by named labels
-            docname, labelid, _ = self.data['labels'].get(target, ('', '', ''))
-            if not docname:
-                return None
-            return make_refnode(builder, fromdocname, docname,
-                                labelid, contnode)
-        elif typ == 'option':
-            progname = node.get('std:program')
-            target = target.strip()
-            docname, labelid = self.data['progoptions'].get((progname, target), ('', ''))
-            if not docname:
-                commands = []
-                while ws_re.search(target):
-                    subcommand, target = ws_re.split(target, 1)
-                    commands.append(subcommand)
-                    progname = "-".join(commands)
+        return self.build_reference_node(fromdocname, builder,
+                                         docname, labelid, newtitle, 'numref',
+                                         nodeclass=addnodes.number_reference,
+                                         title=title)
 
-                    docname, labelid = self.data['progoptions'].get((progname, target),
-                                                                    ('', ''))
-                    if docname:
-                        break
-                else:
-                    return None
+    def _resolve_keyword_xref(self, env, fromdocname, builder, typ, target, node, contnode):
+        # keywords are oddballs: they are referenced by named labels
+        docname, labelid, _ = self.data['labels'].get(target, ('', '', ''))
+        if not docname:
+            return None
+        return make_refnode(builder, fromdocname, docname,
+                            labelid, contnode)
 
-            return make_refnode(builder, fromdocname, docname,
-                                labelid, contnode)
-        else:
-            objtypes = self.objtypes_for_role(typ) or []
-            for objtype in objtypes:
-                if (objtype, target) in self.data['objects']:
-                    docname, labelid = self.data['objects'][objtype, target]
+    def _resolve_option_xref(self, env, fromdocname, builder, typ, target, node, contnode):
+        progname = node.get('std:program')
+        target = target.strip()
+        docname, labelid = self.data['progoptions'].get((progname, target), ('', ''))
+        if not docname:
+            commands = []
+            while ws_re.search(target):
+                subcommand, target = ws_re.split(target, 1)
+                commands.append(subcommand)
+                progname = "-".join(commands)
+
+                docname, labelid = self.data['progoptions'].get((progname, target),
+                                                                ('', ''))
+                if docname:
                     break
             else:
-                docname, labelid = '', ''
-            if not docname:
                 return None
-            return make_refnode(builder, fromdocname, docname,
-                                labelid, contnode)
 
-    def resolve_any_xref(self, env, fromdocname, builder, target,
-                         node, contnode):
+        return make_refnode(builder, fromdocname, docname,
+                            labelid, contnode)
+
+    def _resolve_obj_xref(self, env, fromdocname, builder, typ, target, node, contnode):
+        objtypes = self.objtypes_for_role(typ) or []
+        for objtype in objtypes:
+            if (objtype, target) in self.data['objects']:
+                docname, labelid = self.data['objects'][objtype, target]
+                break
+        else:
+            docname, labelid = '', ''
+        if not docname:
+            return None
+        return make_refnode(builder, fromdocname, docname,
+                            labelid, contnode)
+
+    def resolve_any_xref(self, env, fromdocname, builder, target, node, contnode):
         results = []
         ltarget = target.lower()  # :ref: lowercases its target automatically
         for role in ('ref', 'option'):  # do not try "keyword"
@@ -755,3 +767,7 @@ class StandardDomain(Domain):
         else:
             figtype, _ = self.enumerable_nodes.get(node.__class__, (None, None))
             return figtype
+
+
+def setup(app):
+    app.add_domain(StandardDomain)

sphinx/environment.py

@@ -403,8 +403,13 @@ class BuildEnvironment:
             config.html_extra_path +
             ['**/_sources', '.#*', '**/.#*', '*.lproj/**']
         )
-        self.found_docs = set(get_matching_docs(
-            self.srcdir, config.source_suffix, exclude_matchers=matchers))
+        self.found_docs = set()
+        for docname in get_matching_docs(self.srcdir, config.source_suffix,
+                                         exclude_matchers=matchers):
+            if os.access(self.doc2path(docname), os.R_OK):
+                self.found_docs.add(docname)
+            else:
+                self.warn(docname, "document not readable. Ignored.")
 
         # add catalog mo file dependency
         for docname in self.found_docs:
@@ -1438,7 +1443,10 @@ class BuildEnvironment:
                             # nodes with length 1 don't have any children anyway
                             if len(toplevel) > 1:
                                 subtrees = toplevel.traverse(addnodes.toctree)
-                                toplevel[1][:] = subtrees
+                                if subtrees:
+                                    toplevel[1][:] = subtrees
+                                else:
+                                    toplevel.pop(1)
                     # resolve all sub-toctrees
                     for subtocnode in toc.traverse(addnodes.toctree):
                         if not (subtocnode.get('hidden', False) and
@@ -1484,6 +1492,9 @@ class BuildEnvironment:
         _toctree_add_classes(newnode, 1)
         self._toctree_prune(newnode, 1, prune and maxdepth or 0, collapse)
 
+        if len(newnode[-1]) == 0:  # No titles found
+            return None
+
         # set the target paths in the toctrees (they are not known at TOC
         # generation time)
         for refnode in newnode.traverse(nodes.reference):
@@ -1513,9 +1524,9 @@ class BuildEnvironment:
                                                   typ, target, node, contnode)
                 # really hardwired reference types
                 elif typ == 'any':
-                    newnode = self._resolve_any_reference(builder, node, contnode)
+                    newnode = self._resolve_any_reference(builder, refdoc, node, contnode)
                 elif typ == 'doc':
-                    newnode = self._resolve_doc_reference(builder, node, contnode)
+                    newnode = self._resolve_doc_reference(builder, refdoc, node, contnode)
                 elif typ == 'citation':
                     newnode = self._resolve_citation(builder, refdoc, node, contnode)
                 # no new node found? try the missing-reference event
@@ -1563,10 +1574,10 @@ class BuildEnvironment:
             msg = '%r reference target not found: %%(target)s' % typ
         self.warn_node(msg % {'target': target}, node, type='ref', subtype=typ)
 
-    def _resolve_doc_reference(self, builder, node, contnode):
+    def _resolve_doc_reference(self, builder, refdoc, node, contnode):
         # directly reference to document by source name;
         # can be absolute or relative
-        docname = docname_join(node['refdoc'], node['reftarget'])
+        docname = docname_join(refdoc, node['reftarget'])
         if docname in self.all_docs:
             if node['refexplicit']:
                 # reference with explicit title
@@ -1576,7 +1587,7 @@ class BuildEnvironment:
             innernode = nodes.inline(caption, caption)
             innernode['classes'].append('doc')
             newnode = nodes.reference('', '', internal=True)
-            newnode['refuri'] = builder.get_relative_uri(node['refdoc'], docname)
+            newnode['refuri'] = builder.get_relative_uri(refdoc, docname)
             newnode.append(innernode)
             return newnode
 
@@ -1599,13 +1610,12 @@ class BuildEnvironment:
             # transforms.CitationReference.apply.
             del node['ids'][:]
 
-    def _resolve_any_reference(self, builder, node, contnode):
+    def _resolve_any_reference(self, builder, refdoc, node, contnode):
         """Resolve reference generated by the "any" role."""
-        refdoc = node['refdoc']
         target = node['reftarget']
         results = []
         # first, try resolving as :doc:
-        doc_ref = self._resolve_doc_reference(builder, node, contnode)
+        doc_ref = self._resolve_doc_reference(builder, refdoc, node, contnode)
         if doc_ref:
             results.append(('doc', doc_ref))
         # next, do the standard domain (makes this a priority)

sphinx/ext/imgmath.py

@@ -22,6 +22,7 @@ from six import text_type
 from docutils import nodes
 
 import sphinx
+from sphinx.locale import _
 from sphinx.errors import SphinxError, ExtensionError
 from sphinx.util.png import read_png_depth, write_png_depth
 from sphinx.util.osutil import ensuredir, ENOENT, cd
@@ -253,7 +254,9 @@ def html_visit_displaymath(self, node):
     self.body.append(self.starttag(node, 'div', CLASS='math'))
     self.body.append('<p>')
     if node['number']:
-        self.body.append('<span class="eqno">(%s)</span>' % node['number'])
+        self.body.append('<span class="eqno">(%s)' % node['number'])
+        self.add_permalink_ref(node, _('Permalink to this equation'))
+        self.body.append('</span>')
     if fname is None:
         # something failed -- use text-only as a bad substitute
         self.body.append('<span class="math">%s</span></p>\n</div>' %

sphinx/ext/intersphinx.py

@@ -106,7 +106,7 @@ def read_inventory_v2(f, uri, join, bufsize=16*1024):
 
     for line in split_lines(read_chunks()):
         # be careful to handle names with embedded spaces correctly
-        m = re.match(r'(?x)(.+?)\s+(\S*:\S*)\s+(\S+)\s+(\S+)\s+(.*)',
+        m = re.match(r'(?x)(.+?)\s+(\S*:\S*)\s+(-?\d+)\s+(\S+)\s+(.*)',
                      line.rstrip())
         if not m:
             continue

sphinx/ext/jsmath.py

@@ -13,6 +13,7 @@
 from docutils import nodes
 
 import sphinx
+from sphinx.locale import _
 from sphinx.application import ExtensionError
 from sphinx.ext.mathbase import setup_math as mathbase_setup
 
@@ -34,8 +35,9 @@ def html_visit_displaymath(self, node):
         if i == 0:
             # necessary to e.g. set the id property correctly
             if node['number']:
-                self.body.append('<span class="eqno">(%s)</span>' %
-                                 node['number'])
+                self.body.append('<span class="eqno">(%s)' % node['number'])
+                self.add_permalink_ref(node, _('Permalink to this equation'))
+                self.body.append('</span>')
             self.body.append(self.starttag(node, 'div', CLASS='math'))
         else:
             # but only once!

sphinx/ext/mathjax.py

@@ -14,6 +14,7 @@
 from docutils import nodes
 
 import sphinx
+from sphinx.locale import _
 from sphinx.errors import ExtensionError
 from sphinx.ext.mathbase import setup_math as mathbase_setup
 
@@ -35,7 +36,9 @@ def html_visit_displaymath(self, node):
 
     # necessary to e.g. set the id property correctly
     if node['number']:
-        self.body.append('<span class="eqno">(%s)</span>' % node['number'])
+        self.body.append('<span class="eqno">(%s)' % node['number'])
+        self.add_permalink_ref(node, _('Permalink to this equation'))
+        self.body.append('</span>')
     self.body.append(self.builder.config.mathjax_display[0])
     parts = [prt for prt in node['latex'].split('\n\n') if prt.strip()]
     if len(parts) > 1:  # Add alignment if there are more than 1 equation

sphinx/ext/napoleon/docstring.py

@@ -24,7 +24,7 @@ from sphinx.util.pycompat import UnicodeMixin
 
 _directive_regex = re.compile(r'\.\. \S+::')
 _google_section_regex = re.compile(r'^(\s|\w)+:\s*$')
-_google_typed_arg_regex = re.compile(r'\s*(.+?)\s*\(\s*(.+?)\s*\)')
+_google_typed_arg_regex = re.compile(r'\s*(.+?)\s*\(\s*(.*[^\s]+)\s*\)')
 _numpy_section_regex = re.compile(r'^[=\-`:\'"~^_*+#<>]{2,}\s*$')
 _single_colon_regex = re.compile(r'(?<!:):(?!:)')
 _xref_regex = re.compile(r'(:\w+:\S+:`.+?`|:\S+:`.+?`|`.+?`)')
@@ -172,7 +172,7 @@ class GoogleDocstring(UnicodeMixin):
 
         Returns
         -------
-        :obj:`list` of :obj:`str`
+        list(str)
             The lines of the docstring in a list.
 
         """
@@ -254,12 +254,7 @@ class GoogleDocstring(UnicodeMixin):
                 else:
                     _desc = lines[1:]
 
-                match = _google_typed_arg_regex.match(before)
-                if match:
-                    _name = match.group(1)
-                    _type = match.group(2)
-                else:
-                    _type = before
+                _type = before
 
             _desc = self.__class__(_desc, self._config).lines()
             return [(_name, _type, _desc,)]
@@ -817,7 +812,7 @@ class NumpyDocstring(GoogleDocstring):
 
         Returns
         -------
-        :obj:`list` of :obj:`str`
+        list(str)
             The lines of the docstring in a list.
 
     """

sphinx/ext/todo.py

@@ -70,6 +70,8 @@ def process_todos(app, doctree):
     if not hasattr(env, 'todo_all_todos'):
         env.todo_all_todos = []
     for node in doctree.traverse(todo_node):
+        app.emit('todo-defined', node)
+
         try:
             targetnode = node.parent[node.parent.index(node) - 1]
             if not isinstance(targetnode, nodes.target):
@@ -86,6 +88,9 @@ def process_todos(app, doctree):
             'target': targetnode,
         })
 
+        if env.config.todo_emit_warnings:
+            env.warn_node("TODO entry found: %s" % node[1].astext(), node)
+
 
 class TodoList(Directive):
     """
@@ -187,8 +192,10 @@ def depart_todo_node(self, node):
 
 
 def setup(app):
+    app.add_event('todo-defined')
     app.add_config_value('todo_include_todos', False, 'html')
     app.add_config_value('todo_link_only', False, 'html')
+    app.add_config_value('todo_emit_warnings', False, 'html')
 
     app.add_node(todolist)
     app.add_node(todo_node,

sphinx/ext/viewcode.py

@@ -46,6 +46,10 @@ def doctree_read(app, doctree):
     env = app.builder.env
     if not hasattr(env, '_viewcode_modules'):
         env._viewcode_modules = {}
+    if app.builder.name == "singlehtml":
+        return
+    if app.builder.name.startswith("epub") and not env.config.viewcode_enable_epub:
+        return
 
     def has_tag(modname, fullname, docname, refname):
         entry = env._viewcode_modules.get(modname, None)
@@ -139,8 +143,8 @@ def collect_pages(app):
         # construct a page name for the highlighted source
         pagename = '_modules/' + modname.replace('.', '/')
         # highlight the source using the builder's highlighter
-        if env.config.highlight_language == 'python3':
-            lexer = 'python3'
+        if env.config.highlight_language in ('python3', 'default'):
+            lexer = env.config.highlight_language
         else:
             lexer = 'python'
         highlighted = highlighter.highlight_block(code, lexer, linenos=False)
@@ -215,6 +219,7 @@ def collect_pages(app):
 
 def setup(app):
     app.add_config_value('viewcode_import', True, False)
+    app.add_config_value('viewcode_enable_epub', False, False)
     app.connect('doctree-read', doctree_read)
     app.connect('env-merge-info', env_merge_info)
     app.connect('html-collect-pages', collect_pages)

sphinx/make_mode.py

@@ -18,13 +18,12 @@ from __future__ import print_function
 
 import os
 import sys
-import shutil
 from os import path
 from subprocess import call
 
 import sphinx
 from sphinx.util.console import bold, blue
-from sphinx.util.osutil import cd
+from sphinx.util.osutil import cd, rmtree
 
 proj_name = os.getenv('SPHINXPROJ', '<project>')
 
@@ -42,8 +41,6 @@ BUILDERS = [
     ("",      "latex",       "to make LaTeX files, you can set PAPER=a4 or PAPER=letter"),
     ("posix", "latexpdf",    "to make LaTeX files and run them through pdflatex"),
     ("posix", "latexpdfja",  "to make LaTeX files and run them through platex/dvipdfmx"),
-    ("posix", "lualatexpdf", "to make LaTeX files and run them through lualatex"),
-    ("posix", "xelatexpdf",  "to make LaTeX files and run them through xelatex"),
     ("",      "text",        "to make text files"),
     ("",      "man",         "to make manual pages"),
     ("",      "texinfo",     "to make Texinfo files"),
@@ -77,7 +74,7 @@ class Make(object):
             return 1
         print("Removing everything under %r..." % self.builddir)
         for item in os.listdir(self.builddir):
-            shutil.rmtree(self.builddir_join(item))
+            rmtree(self.builddir_join(item))
 
     def build_help(self):
         print(bold("Sphinx v%s" % sphinx.__display_version__))
@@ -173,18 +170,6 @@ class Make(object):
         with cd(self.builddir_join('latex')):
             os.system('make all-pdf-ja')
 
-    def build_lualatexpdf(self):
-        if self.run_generic_build('latex') > 0:
-            return 1
-        with cd(self.builddir_join('latex')):
-            os.system('make PDFLATEX=lualatex all-pdf')
-
-    def build_xelatexpdf(self):
-        if self.run_generic_build('latex') > 0:
-            return 1
-        with cd(self.builddir_join('latex')):
-            os.system('make PDFLATEX=xelatex all-pdf')
-
     def build_text(self):
         if self.run_generic_build('text') > 0:
             return 1