IntenseWebs/sphinx
3
0
Fork 0
mirror of https://github.com/sphinx-doc/sphinx.git synced 2025-02-25 18:55:22 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into literalincludelines

Browse Source
This commit is contained in:
Jean-François B 2017-02-13 19:38:22 +01:00 committed by GitHub
commit a7becf30de
195 changed files with 8707 additions and 3598 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,27 @@
Subject: <what happen when you do on which document project>
### Problem
- <Detail of problem>
#### Procedure to reproduce the problem
```
<Paste your command-line here which cause the problem>
```
#### Error logs / results
```
<Paste your error log here>
```
- <public link of unexpected result if you have>
#### Expected results
<Describe what to actually do>
### Reproducible project / your project
- <link to your project, or attach zipped small project sample>
### Environment info
- OS: <Unix/Linux/Mac/Win/other with version>
- Python version:
- Sphinx version:
- <Extra tools e.g.: Browser, tex or something else>

18
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,18 @@
Subject: <short purpose of this pull request>
### Feature or Bugfix
<!-- please choose -->
- Feature
- Bugfix
### Purpose
- <long purpose of this pull request>
- <Environment if this PR depends on>
### Detail
- <feature1 or bug1>
- <feature2 or bug2>
### Relates
- <URL or Ticket>

5
.travis.yml
View File

@ -1,10 +1,11 @@
language: python
sudo: false
dist: trusty
cache:
directories:
- $HOME/.cache/pip
python:
- "pypy"
- "pypy-5.4.1"
- "2.7"
- "3.4"
- "3.5"
@ -26,7 +27,7 @@ matrix:
env: DOCUTILS=0.12
- python: nightly
env: DOCUTILS=0.12
- python: pypy
- python: "pypy-5.4.1"
env: DOCUTILS=0.12
addons:
apt:

1
AUTHORS
View File

@ -21,6 +21,7 @@ Other contributors, listed alphabetically, are:
* Henrique Bastos -- SVG support for graphviz extension
* Daniel Bültmann -- todo extension
* Jean-François Burnol -- LaTeX improvements
* Marco Buttu -- doctest extension (pyversion option)
* Etienne Desautels -- apidoc module
* Michael Droettboom -- inheritance_diagram extension
* Charles Duffy -- original graphviz extension

92
CHANGES
View File

@ -1,3 +1,85 @@
Release 1.6 (in development)
============================
Incompatible changes
--------------------
* #1061, #2336, #3235: Now generation of autosummary doesn't contain imported
members by default. Thanks to Luc Saffre.
* LaTeX ``\includegraphics`` command isn't overloaded: only ``\sphinxincludegraphics``
has the custom code to fit image to available width if oversized.
* The subclasses of ``sphinx.domains.Index`` should override ``generate()``
method. The default implementation raises NotImplementedError
* LaTeX positioned long tables horizontally centered, and short ones
flushed left (no text flow around table.) The position now defaults to center in
both cases, and it will obey Docutils 0.13 ``:align:`` option (refs #3415, #3377)
Features removed
----------------
* Configuration variables
- epub3_contributor
- epub3_description
- epub3_page_progression_direction
- html_translator_class
- html_use_modindex
- latex_font_size
- latex_paper_size
- latex_preamble
- latex_use_modindex
- latex_use_parts
* ``termsep`` node
* defindex.html template
* LDML format support in `today`, `today_fmt` and `html_last_updated_fmt`
* ``:inline:`` option for the directives of sphinx.ext.graphviz extension
* sphinx.ext.pngmath extension
* ``sphinx.util.compat.make_admonition()``
Features added
--------------
* #3136: Add ``:name:`` option to the directives in ``sphinx.ext.graphviz``
* #2336: Add ``imported_members`` option to ``sphinx-autogen`` command to document
imported members.
* C++, add ``:tparam-line-spec:`` option to templated declarations.
When specified, each template parameter will be rendered on a separate line.
* #3359: Allow sphinx.js in a user locale dir to override sphinx.js from Sphinx
* #3303: Add ``:pyversion:`` option to the doctest directive.
* #3378: (latex) support for ``:widths:`` option of table directives
(refs: #3379, #3381)
* #3402: Allow to suppress "download file not readable" warnings using
:confval:`suppress_warnings`.
* #3377: latex: Add support for Docutils 0.13 ``:align:`` option for tables
(but does not implement text flow around table).
Bugs fixed
----------
Deprecated
----------
* ``sphinx.util.compat.Directive`` class is now deprecated. Please use instead
``docutils.parsers.rsr.Directive``
* ``sphinx.util.compat.docutils_version`` is now deprecated
* #2367: ``Sphinx.warn()``, ``Sphinx.info()`` and other logging methods are now
deprecated. Please use ``sphinx.util.logging`` (:ref:`logging-api`) instead.
* #3318: ``notice`` is now deprecated as LaTeX environment name and will be
removed at Sphinx 1.7. Extension authors please use ``sphinxadmonition``
instead (as Sphinx does since 1.5.)
* ``Sphinx.status_iterator()` and ``Sphinx.old_status_iterator()`` is now
deprecated. Please use ``sphinx.util:status_iterator()`` intead.
* ``BuildEnvironment.set_warnfunc()`` is now deprecated
* Following methods of ``BuildEnvironment`` is now deprecated.
- ``BuildEnvironment.note_toctree()``
- ``BuildEnvironment.get_toc_for()``
- ``BuildEnvironment.get_toctree_for()``
- ``BuildEnvironment.create_index()``
Please use ``sphinx.environment.adapters`` modules instead.
Release 1.5.3 (in development)
==============================
@ -85,6 +167,7 @@ Bugs fixed
* #3295: Could not import extension sphinx.builders.linkcheck
* #3285: autosummary: asterisks are escaped twice
* LaTeX, pass dvipdfm option to geometry package for Japanese documents (ref #3363)
* Fix parselinenos() could not parse left half open range (cf. "-4")
Release 1.5.1 (released Dec 13, 2016)
@ -162,7 +245,7 @@ Incompatible changes
* Fix ``genindex.html``, Sphinx's document template, link address to itself to satisfy xhtml standard.
* Use epub3 builder by default. And the old epub builder is renamed to epub2.
* Fix ``epub`` and ``epub3`` builders that contained links to ``genindex`` even if ``epub_use_index = False``.
* `html_translator_class` is now deprecated.
* ``html_translator_class`` is now deprecated.
Use `Sphinx.set_translator()` API instead.
* Drop python 2.6 and 3.3 support
* Drop epub3 builder's ``epub3_page_progression_direction`` option (use ``epub3_writing_mode``).
@ -331,7 +414,7 @@ Bugs fixed
* `sphinx.ext.autodoc` crashes if target code imports * from mock modules
by `autodoc_mock_imports`.
* #1953: ``Sphinx.add_node`` does not add handlers the translator installed by
`html_translator_class`
``html_translator_class``
* #1797: text builder inserts blank line on top
* #2894: quickstart main() doesn't use argv argument
* #2874: gettext builder could not extract all text under the ``only``
@ -738,7 +821,8 @@ Incompatible changes
``"MMMM dd, YYYY"`` is default format for `today_fmt` and `html_last_updated_fmt`.
However strftime format like ``"%B %d, %Y"`` is also supported for backward
compatibility until Sphinx-1.5. Later format will be disabled from Sphinx-1.5.
* #2327: `latex_use_parts` is deprecated now. Use `latex_toplevel_sectioning` instead.
* #2327: ``latex_use_parts`` is deprecated now. Use `latex_toplevel_sectioning`
instead.
* #2337: Use ``\url{URL}`` macro instead of ``\href{URL}{URL}`` in LaTeX writer.
* #1498: manpage writer: don't make whole of item in definition list bold if it includes strong node.
* #582: Remove hint message from quick search box for html output.
@ -1301,7 +1385,7 @@ Features added
for the ids defined on the node. Thanks to Olivier Heurtier.
* PR#229: Allow registration of other translators. Thanks to Russell Sim.
* Add app.set_translator() API to register or override a Docutils translator
class like `html_translator_class`.
class like ``html_translator_class``.
* PR#267, #1134: add 'diff' parameter to literalinclude. Thanks to Richard Wall
and WAKAYAMA shirou.
* PR#272: Added 'bizstyle' theme. Thanks to Shoji KUMAGAI.

7
Makefile
View File

@ -1,6 +1,6 @@
PYTHON ?= python
.PHONY: all style-check clean clean-pyc clean-patchfiles clean-backupfiles \
.PHONY: all style-check type-check clean clean-pyc clean-patchfiles clean-backupfiles \
clean-generated pylint reindent test covertest build
DONT_CHECK = -i build -i dist -i sphinx/style/jquery.js \
@ -30,11 +30,14 @@ DONT_CHECK = -i build -i dist -i sphinx/style/jquery.js \
-i sphinx/search/tr.py \
-i .tox
all: clean-pyc clean-backupfiles style-check test
all: clean-pyc clean-backupfiles style-check type-check test
style-check:
@$(PYTHON) utils/check_sources.py $(DONT_CHECK) .
type-check:
mypy sphinx/
clean: clean-pyc clean-pycache clean-patchfiles clean-backupfiles clean-generated clean-testfiles clean-buildfiles clean-mypyfiles
clean-pyc:

7
doc/_static/conf.py.txt vendored
View File

@ -79,7 +79,7 @@ language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
# These patterns also affect html_static_path and html_extra_path
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
# The reST default role (used for this markup: `text`) to use for all
@ -268,11 +268,6 @@ latex_documents = [
#
# latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#
# latex_use_parts = False
# If true, show page references after internal links.
#
# latex_show_pagerefs = False

100
doc/config.rst
View File

@ -226,8 +226,10 @@ General configuration
* app.add_role
* app.add_generic_role
* app.add_source_parser
* download.not_readable
* image.data_uri
* image.nonlocal_uri
* image.not_readable
* ref.term
* ref.ref
* ref.numref
@ -382,18 +384,6 @@ Project information
%Y'`` (or, if translation is enabled with :confval:`language`, an equivalent
format for the selected locale).
.. versionchanged:: 1.4
Format specification was changed from strftime to Locale Data Markup
Language. strftime format is also supported for backward compatibility
until Sphinx-1.5.
.. versionchanged:: 1.4.1
Format specification was changed again from Locale Data Markup Language
to strftime. LDML format is also supported for backward compatibility
until Sphinx-1.5.
.. confval:: highlight_language
The default language to highlight source code in. The default is
@ -769,19 +759,6 @@ that use Sphinx's HTMLWriter class.
The empty string is equivalent to ``'%b %d, %Y'`` (or a
locale-dependent equivalent).
.. versionchanged:: 1.4
Format specification was changed from strftime to Locale Data Markup
Language. strftime format is also supported for backward compatibility
until Sphinx-1.5.
.. versionchanged:: 1.4.1
Format specification was changed again from Locale Data Markup Language
to strftime. LDML format is also supported for backward compatibility
until Sphinx-1.5.
.. confval:: html_use_smartypants
If true, `SmartyPants <http://daringfireball.net/projects/smartypants/>`_
@ -881,13 +858,6 @@ that use Sphinx's HTMLWriter class.
.. versionadded:: 1.0
.. confval:: html_use_modindex
If true, add a module index to the HTML documents. Default is ``True``.
.. deprecated:: 1.0
Use :confval:`html_domain_indices`.
.. confval:: html_use_index
If true, add an index to the HTML documents. Default is ``True``.
@ -950,20 +920,6 @@ that use Sphinx's HTMLWriter class.
.. versionadded:: 0.6
.. confval:: html_translator_class
A string with the fully-qualified name of a HTML Translator class, that is, a
subclass of Sphinx's :class:`~sphinx.writers.html.HTMLTranslator`, that is
used to translate document trees to HTML. Default is ``None`` (use the
builtin translator).
.. seealso:: :meth:`~sphinx.application.Sphinx.set_translator`
.. deprecated:: 1.5
Implement your translator as extension and use `Sphinx.set_translator`
instead.
.. confval:: html_show_copyright
If true, "(C) Copyright ..." is shown in the HTML footer. Default is
@ -1539,20 +1495,6 @@ the `Dublin Core metadata <http://dublincore.org/>`_.
.. [#] https://developer.mozilla.org/en-US/docs/Web/CSS/writing-mode
.. confval:: epub3_page_progression_direction
The global direction in which the content flows.
Allowed values are ``'ltr'`` (left-to-right), ``'rtl'`` (right-to-left) and
``'default'``. The default value is ``'ltr'``.
When the ``'default'`` value is specified, the Author is expressing no
preference and the Reading System may chose the rendering direction.
.. versionadded:: 1.4
.. deprecated:: 1.5
Use ``epub_writing_mode`` instead.
.. _latex-options:
Options for LaTeX output
@ -1624,16 +1566,6 @@ These options influence LaTeX output. See further :doc:`latex`.
.. versionadded:: 1.4
.. confval:: latex_use_parts
If true, the topmost sectioning unit is parts, else it is chapters. Default:
``False``.
.. versionadded:: 0.3
.. deprecated:: 1.4
Use :confval:`latex_toplevel_sectioning`.
.. confval:: latex_appendices
A list of document names to append as an appendix to all manuals.
@ -1649,13 +1581,6 @@ These options influence LaTeX output. See further :doc:`latex`.
.. versionadded:: 1.0
.. confval:: latex_use_modindex
If true, add a module index to LaTeX documents. Default is ``True``.
.. deprecated:: 1.0
Use :confval:`latex_domain_indices`.
.. confval:: latex_show_pagerefs
If true, add page references after internal references. This is very useful
@ -1932,27 +1857,6 @@ These options influence LaTeX output. See further :doc:`latex`.
.. versionchanged:: 1.2
This overrides the files which is provided from Sphinx such as sphinx.sty.
.. confval:: latex_preamble
Additional LaTeX markup for the preamble.
.. deprecated:: 0.5
Use the ``'preamble'`` key in the :confval:`latex_elements` value.
.. confval:: latex_paper_size
The output paper size (``'letter'`` or ``'a4'``). Default is ``'letter'``.
.. deprecated:: 0.5
Use the ``'papersize'`` key in the :confval:`latex_elements` value.
.. confval:: latex_font_size
The font size ('10pt', '11pt' or '12pt'). Default is ``'10pt'``.
.. deprecated:: 0.5
Use the ``'pointsize'`` key in the :confval:`latex_elements` value.
.. _text-options:

8
doc/develop.rst
View File

@ -55,17 +55,17 @@ This is the current list of contributed extensions in that repository:
- hyphenator: client-side hyphenation of HTML using hyphenator_
- inlinesyntaxhighlight_: inline syntax highlighting
- lassodomain: a domain for documenting Lasso_ source code
- libreoffice: an extension to include any drawing supported by LibreOffice (e.g. odg, vsd...).
- libreoffice: an extension to include any drawing supported by LibreOffice (e.g. odg, vsd, ...).
- lilypond: an extension inserting music scripts from Lilypond_ in PNG format.
- makedomain_: a domain for `GNU Make`_
- matlabdomain: document MATLAB_ code.
- mockautodoc: mock imports.
- mscgen: embed mscgen-formatted MSC (Message Sequence Chart)s.
- napoleon: supports `Google style`_ and `NumPy style`_ docstrings.
- nicoviceo: embed videos from nicovideo
- nicovideo: embed videos from nicovideo
- nwdiag: embed network diagrams by using nwdiag_
- omegat: support tools to collaborate with OmegaT_ (Sphinx 1.1 needed)
- osaka: convert standard Japanese doc to Osaka dialect (it is joke extension)
- osaka: convert standard Japanese doc to Osaka dialect (this is a joke extension)
- paverutils: an alternate integration of Sphinx with Paver_.
- phpdomain: an extension for PHP support
- plantuml: embed UML diagram by using PlantUML_
@ -113,7 +113,7 @@ own extensions.
.. _Google Analytics: http://www.google.com/analytics/
.. _Google Chart: https://developers.google.com/chart/
.. _Google Maps: https://www.google.com/maps
.. _Google style: http://google-styleguide.googlecode.com/svn/trunk/pyguide.html
.. _Google style: https://google.github.io/styleguide/pyguide.html
.. _NumPy style: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt
.. _hyphenator: https://github.com/mnater/hyphenator
.. _exceltable: http://pythonhosted.org/sphinxcontrib-exceltable/

14
doc/domains.rst
View File

@ -545,6 +545,10 @@ The C++ Domain
The C++ domain (name **cpp**) supports documenting C++ projects.
Directives
~~~~~~~~~~
The following directives are available. All declarations can start with
a visibility statement (``public``, ``private`` or ``protected``).
@ -740,6 +744,16 @@ a visibility statement (``public``, ``private`` or ``protected``).
Holder of elements, to which it can provide access via
:cpp:concept:`Iterator` s.
Options
.......
Some directives support options:
- ``:noindex:``, see :ref:`basic-domain-markup`.
- ``:tparam-line-spec:``, for templated declarations.
If specified, each template parameter will be rendered on a separate line.
Constrained Templates
~~~~~~~~~~~~~~~~~~~~~

5
doc/ext/autodoc.rst
View File

@ -446,6 +446,11 @@ member should be included in the documentation by using the following event:
documentation. The member is excluded if a handler returns ``True``. It is
included if the handler returns ``False``.
If more than one enabled extension handles the ``autodoc-skip-member``
event, autodoc will use the first non-``None`` value returned by a handler.
Handlers should return ``None`` to fall back to the skipping behavior of
autodoc and other enabled extensions.
:param app: the Sphinx application object
:param what: the type of the object which the docstring belongs to (one of
``"module"``, ``"class"``, ``"exception"``, ``"function"``, ``"method"``,

15
doc/ext/doctest.rst
View File

@ -63,7 +63,7 @@ a comma-separated list of group names.
default set of flags is specified by the :confval:`doctest_default_flags`
configuration variable.
This directive supports two options:
This directive supports three options:
* ``hide``, a flag option, hides the doctest block in other builders. By
default it is shown as a highlighted doctest block.
@ -73,6 +73,19 @@ a comma-separated list of group names.
explicit flags per example, with doctest comments, but they will show up in
other builders too.)
* ``pyversion``, a string option, can be used to specify the required Python
version for the example to be tested. For instance, in the following case
the example will be tested only for Python versions greather than 3.3::
.. doctest::
:pyversion: > 3.3
The supported operands are ``<``, ``<=``, ``==``, ``>=``, ``>``, and
comparison is performed by `distutils.version.LooseVersion
<https://www.python.org/dev/peps/pep-0386/#distutils>`__.
.. versionadded:: 1.6
Note that like with standard doctests, you have to use ``<BLANKLINE>`` to
signal a blank line in the expected output. The ``<BLANKLINE>`` is removed
when building presentation output (HTML, LaTeX etc.).

17
doc/ext/graphviz.rst
View File

@ -76,21 +76,9 @@ It adds these directives:
alternate text for HTML output. If not given, the alternate text defaults to
the graphviz code.
.. versionadded:: 1.1
All three directives support an ``inline`` flag that controls paragraph
breaks in the output. When set, the graph is inserted into the current
paragraph. If the flag is not given, paragraph breaks are introduced before
and after the image (the default).
.. versionadded:: 1.1
All three directives support a ``caption`` option that can be used to give a
caption to the diagram. Naturally, diagrams marked as "inline" cannot have a
caption.
.. deprecated:: 1.4
``inline`` option is deprecated.
All three directives generate inline node by default. If ``caption`` is given,
these generate block node instead.
caption to the diagram.
.. versionchanged:: 1.4
All three directives support a ``graphviz_dot`` option that can be switch the
@ -100,6 +88,9 @@ It adds these directives:
All three directives support a ``align`` option to align the graph horizontal.
The values "left", "center", "right" are allowed.
.. versionadded:: 1.6
All three directives support a ``name`` option to set the label to graph.
There are also these new config values:
.. confval:: graphviz_dot

15
doc/extdev/appapi.rst
View File

@ -89,11 +89,6 @@ package.
This allows extensions to use custom translator and define custom
nodes for the translator (see :meth:`add_node`).
This is a API version of :confval:`html_translator_class` for all other
builders. Note that if :confval:`html_translator_class` is specified and
this API is called for html related builders, API overriding takes
precedence.
.. versionadded:: 1.3
.. method:: Sphinx.add_node(node, **kwds)
@ -364,6 +359,12 @@ package.
.. versionadded:: 1.4
.. method:: Sphinx.add_env_collector(collector)
Register an environment collector class (refs: :ref:`collector-api`)
.. versionadded:: 1.6
.. method:: Sphinx.require_sphinx(version)
Compare *version* (which must be a ``major.minor`` version string,
@ -420,6 +421,10 @@ The application object also provides support for emitting leveled messages.
the build; just raise an exception (:exc:`sphinx.errors.SphinxError` or a
custom subclass) to do that.
.. deprecated:: 1.6
Please use :ref:`logging-api` instead.
.. automethod:: Sphinx.warn
.. automethod:: Sphinx.info

9
doc/extdev/collectorapi.rst Normal file
View File

@ -0,0 +1,9 @@
.. _collector-api:
Environment Collector API
-------------------------
.. module:: sphinx.environment.collectors
.. autoclass:: EnvironmentCollector
:members:

2
doc/extdev/index.rst
View File

@ -50,7 +50,9 @@ APIs used for writing extensions
appapi
envapi
builderapi
collectorapi
markupapi
domainapi
parserapi
nodes
logging

77
doc/extdev/logging.rst Normal file
View File

@ -0,0 +1,77 @@
.. _logging-api:
Logging API
===========
.. function:: sphinx.util.logging.getLogger(name)
Returns a logger wrapped by :class:`SphinxLoggerAdapter` with the specified *name*.
Example usage::
from sphinx.util import logging # Load on top of python's logging module
logger = logging.getLogger(__name__)
logger.info('Hello, this is an extension!')
.. class:: SphinxLoggerAdapter(logging.LoggerAdapter)
.. method:: SphinxLoggerAdapter.error(level, msg, *args, **kwargs)
.. method:: SphinxLoggerAdapter.critical(level, msg, *args, **kwargs)
.. method:: SphinxLoggerAdapter.warning(level, msg, *args, **kwargs)
Logs a message on this logger with the specified level.
Basically, the arguments are as with python's logging module.
In addition, Sphinx logger supports following keyword arguments:
**type**, ***subtype***
Categories of warning logs. It is used to suppress
warnings by :confval:`suppress_warnings` setting.
**location**
Where the warning happened. It is used to include
the path and line number in each log. It allows docname,
tuple of docname and line number and nodes::
logger = sphinx.util.logging.getLogger(__name__)
logger.warning('Warning happened!', location='index')
logger.warning('Warning happened!', location=('chapter1/index', 10))
logger.warning('Warning happened!', location=some_node)
**color**
The color of logs. By default, warning level logs are
colored as ``"darkred"``. The others are not colored.
.. method:: SphinxLoggerAdapter.log(level, msg, *args, **kwargs)
.. method:: SphinxLoggerAdapter.info(level, msg, *args, **kwargs)
.. method:: SphinxLoggerAdapter.verbose(level, msg, *args, **kwargs)
.. method:: SphinxLoggerAdapter.debug(level, msg, *args, **kwargs)
Logs a message to this logger with the specified level.
Basically, the arguments are as with python's logging module.
In addition, Sphinx logger supports following keyword arguments:
**nonl**
If true, the logger does not fold lines at the end of the log message.
The default is ``False``.
**color**
The color of logs. By default, debug level logs are
colored as ``"darkgray"``, and debug2 level ones are ``"lightgray"``.
The others are not colored.
.. function:: pending_logging()
Marks all logs as pending::
with pending_logging():
logger.warning('Warning message!') # not flushed yet
some_long_process()
# the warning is flushed here
.. function:: pending_warnings()
Marks warning logs as pending. Similar to :func:`pending_logging`.

1
doc/extdev/nodes.rst
View File

@ -55,4 +55,3 @@ You should not need to generate the nodes below in extensions.
.. autoclass:: start_of_file
.. autoclass:: productionlist
.. autoclass:: production
.. autoclass:: termsep

20
doc/extdev/tutorial.rst
View File

@ -246,7 +246,6 @@ todolist directive has neither content nor arguments that need to be handled.
The ``todo`` directive function looks like this::
from sphinx.util.compat import make_admonition
from sphinx.locale import _
class TodoDirective(Directive):
@ -260,20 +259,20 @@ The ``todo`` directive function looks like this::
targetid = "todo-%d" % env.new_serialno('todo')
targetnode = nodes.target('', '', ids=[targetid])
ad = make_admonition(todo, self.name, [_('Todo')], self.options,
self.content, self.lineno, self.content_offset,
self.block_text, self.state, self.state_machine)
todo_node = todo('\n'.join(self.content))
todo_node += nodes.title(_('Todo'), _('Todo'))
self.state.nested_parse(self.content, self.content_offset, todo_node)
if not hasattr(env, 'todo_all_todos'):
env.todo_all_todos = []
env.todo_all_todos.append({
'docname': env.docname,
'lineno': self.lineno,
'todo': ad[0].deepcopy(),
'todo': todo_node.deepcopy(),
'target': targetnode,
})
return [targetnode] + ad
return [targetnode, todo_node]
Several important things are covered here. First, as you can see, you can refer
to the build environment instance using ``self.state.document.settings.env``.
@ -285,11 +284,10 @@ returns a new unique integer on each call and therefore leads to unique target
names. The target node is instantiated without any text (the first two
arguments).
An admonition is created using a standard docutils function (wrapped in Sphinx
for docutils cross-version compatibility). The first argument gives the node
class, in our case ``todo``. The third argument gives the admonition title (use
``arguments`` here to let the user specify the title). A list of nodes is
returned from ``make_admonition``.
On creating admonition node, the content body of the directive are parsed using
``self.state.nested_parse``. The first argument gives the content body, and
the second one gives content offset. The third argument gives the parent node
of parsed result, in our case the ``todo`` node.
Then, the todo node is added to the environment. This is needed to be able to
create a list of all todo entries throughout the documentation, in the place

2
doc/faq.rst
View File

@ -201,7 +201,7 @@ The following list gives some hints for the creation of epub files:
Error(prcgen):E24011: TOC section scope is not included in the parent chapter:(title)
Error(prcgen):E24001: The table of content could not be built.
.. _Epubcheck: https://code.google.com/archive/p/epubcheck
.. _Epubcheck: https://github.com/IDPF/epubcheck
.. _Calibre: http://calibre-ebook.com/
.. _FBreader: https://fbreader.org/
.. _Bookworm: http://www.oreilly.com/bookworm/index.html

2
doc/intl.rst
View File

@ -74,7 +74,7 @@ Quick guide
^^^^^^^^^^^
`sphinx-intl`_ is a useful tool to work with Sphinx translation flow.
This section describe a easy way to translate with sphinx-intl.
This section describe an easy way to translate with sphinx-intl.
#. Install `sphinx-intl`_ by :command:`pip install sphinx-intl` or
:command:`easy_install sphinx-intl`.

2
doc/invocation.rst
View File

@ -404,7 +404,7 @@ variables to customize behavior:
.. describe:: PAPER
The value for :confval:`latex_paper_size`.
The value for '"papersize"` key of :confval:`latex_elements`.
.. describe:: SPHINXBUILD

2
doc/markup/code.rst
View File

@ -240,7 +240,7 @@ For example::
:rst:dir:`literalinclude` also supports the ``caption`` and ``name`` option.
``caption`` has a additional feature that if you leave the value empty, the shown
``caption`` has an additional feature that if you leave the value empty, the shown
filename will be exactly the one given as an argument.

2
doc/markup/toctree.rst
View File

@ -41,7 +41,7 @@ tables of contents. The ``toctree`` directive is the central element.
* Tables of contents from all those documents are inserted, with a maximum
depth of two, that means one nested heading. ``toctree`` directives in
those documents are also taken into account.
* Sphinx knows that the relative order of the documents ``intro``,
* Sphinx knows the relative order of the documents ``intro``,
``strings`` and so forth, and it knows that they are children of the shown
document, the library index. From this information it generates "next
chapter", "previous chapter" and "parent chapter" links.

2
doc/rest.rst
View File

@ -226,7 +226,7 @@ as long as the text::
Normally, there are no heading levels assigned to certain characters as the
structure is determined from the succession of headings. However, this
convention is used in `Python's Style Guide for documentating
convention is used in `Python's Style Guide for documenting
<https://docs.python.org/devguide/documenting.html#style-guide>`_ which you may
follow:

64
doc/templating.rst
View File

@ -291,31 +291,12 @@ in the future.
The value of :confval:`master_doc`, for usage with :func:`pathto`.
.. data:: next
The next document for the navigation. This variable is either false or has
two attributes `link` and `title`. The title contains HTML markup. For
example, to generate a link to the next page, you can use this snippet::
{% if next %}
<a href="{{ next.link|e }}">{{ next.title }}</a>
{% endif %}
.. data:: pagename
The "page name" of the current file, i.e. either the document name if the
file is generated from a reST source, or the equivalent hierarchical name
relative to the output directory (``[directory/]filename_without_extension``).
.. data:: parents
A list of parent documents for navigation, structured like the :data:`next`
item.
.. data:: prev
Like :data:`next`, but for the previous page.
.. data:: project
The value of :confval:`project`.
@ -369,16 +350,58 @@ In documents that are created from source files (as opposed to
automatically-generated files like the module index, or documents that already
are in HTML form), these variables are also available:
.. data:: body
A string containing the content of the page in HTML form as produced by the HTML builder,
before the theme is applied.
.. data:: display_toc
A boolean that is True if the toc contains more than one entry.
.. data:: meta
Document metadata (a dictionary), see :ref:`metadata`.
.. data:: metatags
A string containing the page's HTML :dudir:`meta` tags.
.. data:: next
The next document for the navigation. This variable is either false or has
two attributes `link` and `title`. The title contains HTML markup. For
example, to generate a link to the next page, you can use this snippet::
{% if next %}
<a href="{{ next.link|e }}">{{ next.title }}</a>
{% endif %}
.. data:: page_source_suffix
The suffix of the file that was rendered. Since we support a list of :confval:`source_suffix`,
this will allow you to properly link to the original source file.
.. data:: parents
A list of parent documents for navigation, structured like the :data:`next`
item.
.. data:: prev
Like :data:`next`, but for the previous page.
.. data:: sourcename
The name of the copied source file for the current document. This is only
nonempty if the :confval:`html_copy_source` value is ``True``.
This has empty value on creating automatically-generated files.
.. data:: title
The page title.
.. data:: toc
The local table of contents for the current page, rendered as HTML bullet
@ -401,7 +424,4 @@ are in HTML form), these variables are also available:
* ``includehidden`` (``False`` by default): if true, the TOC tree will also
contain hidden entries.
.. data:: page_source_suffix
The suffix of the file that was rendered. Since we support a list of :confval:`source_suffix`,
this will allow you to properly link to the original source file.

7
mypy.ini Normal file
View File

@ -0,0 +1,7 @@
[mypy]
python_version = 2.7
silent_imports = True
fast_parser = True
incremental = True
check_untyped_defs = True
warn_unused_ignores = True

1
setup.py
View File

@ -51,6 +51,7 @@ requires = [
'alabaster>=0.7,<0.8',
'imagesize',
'requests>=2.0.0',
'typing',
]
extras_require = {
# Environment Marker works for wheel 0.24 or later

23
sphinx/__init__.py
View File

@ -30,19 +30,19 @@ if 'PYTHONWARNINGS' not in os.environ:
warnings.filterwarnings('ignore', "'U' mode is deprecated",
DeprecationWarning, module='docutils.io')
__version__ = '1.5.3+'
__released__ = '1.5.3' # used when Sphinx builds its own docs
__version__ = '1.6'
__released__ = '1.6+' # used when Sphinx builds its own docs
# version info for better programmatic use
# possible values for 3rd element: 'alpha', 'beta', 'rc', 'final'
# 'final' has 0 as the last element
version_info = (1, 5, 3, 'beta', 0)
version_info = (1, 6, 0, 'beta', 1)
package_dir = path.abspath(path.dirname(__file__))
__display_version__ = __version__ # used for command line version
if __version__.endswith('+'):
# try to find out the changeset hash if checked out from hg, and append
# try to find out the commit hash if checked out from git, and append
# it to __version__ (since we use this value from setup.py, it gets
# automatically propagated to an installed copy as well)
__display_version__ = __version__
@ -54,12 +54,13 @@ if __version__.endswith('+'):
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if out:
__display_version__ += '/' + out.decode().strip()
__display_version__ += '/' + out.decode().strip() # type: ignore
except Exception:
pass
def main(argv=sys.argv):
# type: (List[str]) -> None
if sys.argv[1:2] == ['-M']:
sys.exit(make_main(argv))
else:
@ -67,6 +68,7 @@ def main(argv=sys.argv):
def build_main(argv=sys.argv):
# type: (List[str]) -> int
"""Sphinx build "main" command-line entry."""
if (sys.version_info[:3] < (2, 7, 0) or
(3, 0, 0) <= sys.version_info[:3] < (3, 4, 0)):
@ -99,19 +101,20 @@ def build_main(argv=sys.argv):
return 1
raise
from sphinx.util.compat import docutils_version
if docutils_version < (0, 10):
import sphinx.util.docutils
if sphinx.util.docutils.__version_info__ < (0, 10):
sys.stderr.write('Error: Sphinx requires at least Docutils 0.10 to '
'run.\n')
return 1
return cmdline.main(argv)
return cmdline.main(argv) # type: ignore
def make_main(argv=sys.argv):
# type: (List[str]) -> int
"""Sphinx build "make mode" entry."""
from sphinx import make_mode
return make_mode.run_make_mode(argv[2:])
return make_mode.run_make_mode(argv[2:]) # type: ignore
if __name__ == '__main__':
sys.exit(main(sys.argv))
sys.exit(main(sys.argv)) # type: ignore

28
sphinx/addnodes.py
View File

@ -9,10 +9,11 @@
:license: BSD, see LICENSE for details.
"""
import warnings
from docutils import nodes
from sphinx.deprecation import RemovedInSphinx16Warning
if False:
# For type annotation
from typing import Sequence # NOQA
class translatable(object):
@ -30,14 +31,17 @@ class translatable(object):
"""
def preserve_original_messages(self):
# type: () -> None
"""Preserve original translatable messages."""
raise NotImplementedError
def apply_translated_message(self, original_message, translated_message):
# type: (unicode, unicode) -> None
"""Apply translated message."""
raise NotImplementedError
def extract_original_messages(self):
# type: () -> Sequence[unicode]
"""Extract translation messages.
:returns: list of extracted messages or messages generator
@ -49,14 +53,17 @@ class toctree(nodes.General, nodes.Element, translatable):
"""Node for inserting a "TOC tree"."""
def preserve_original_messages(self):
# type: () -> None
if self.get('caption'):
self['rawcaption'] = self['caption']
def apply_translated_message(self, original_message, translated_message):
# type: (unicode, unicode) -> None
if self.get('rawcaption') == original_message:
self['caption'] = translated_message
def extract_original_messages(self):
# type: () -> List[unicode]
if 'rawcaption' in self:
return [self['rawcaption']]
else:
@ -109,6 +116,7 @@ class desc_type(nodes.Part, nodes.Inline, nodes.TextElement):
class desc_returns(desc_type):
"""Node for a "returns" annotation (a la -> in Python)."""
def astext(self):
# type: () -> unicode
return ' -> ' + nodes.TextElement.astext(self)
@ -130,6 +138,7 @@ class desc_optional(nodes.Part, nodes.Inline, nodes.TextElement):
child_text_separator = ', '
def astext(self):
# type: () -> unicode
return '[' + nodes.TextElement.astext(self) + ']'
@ -273,19 +282,6 @@ class abbreviation(nodes.Inline, nodes.TextElement):
"""Node for abbreviations with explanations."""
class termsep(nodes.Structural, nodes.Element):
"""Separates two terms within a <term> node.
.. versionchanged:: 1.4
sphinx.addnodes.termsep is deprecated. It will be removed at Sphinx-1.6.
"""
def __init__(self, *args, **kw):
warnings.warn('sphinx.addnodes.termsep will be removed at Sphinx-1.6',
RemovedInSphinx16Warning, stacklevel=2)
super(termsep, self).__init__(*args, **kw)
class manpage(nodes.Inline, nodes.TextElement):
"""Node for references to manpages."""

34
sphinx/apidoc.py
View File

@ -25,6 +25,11 @@ from fnmatch import fnmatch
from sphinx.util.osutil import FileAvoidWrite, walk
from sphinx import __display_version__
from sphinx.quickstart import EXTENSIONS
if False:
# For type annotation
from typing import Any, Tuple # NOQA
# automodule options
if 'SPHINX_APIDOC_OPTIONS' in os.environ:
@ -42,6 +47,7 @@ PY_SUFFIXES = set(['.py', '.pyx'])
def makename(package, module):
# type: (unicode, unicode) -> unicode
"""Join package and module with a dot."""
# Both package and module can be None/empty.
if package:
@ -54,6 +60,7 @@ def makename(package, module):
def write_file(name, text, opts):
# type: (unicode, unicode, Any) -> None
"""Write the output file for module/package <name>."""
fname = path.join(opts.destdir, '%s.%s' % (name, opts.suffix))
if opts.dryrun:
@ -68,12 +75,14 @@ def write_file(name, text, opts):
def format_heading(level, text):
# type: (int, unicode) -> unicode
"""Create a heading of <level> [1, 2 or 3 supported]."""
underlining = ['=', '-', '~', ][level - 1] * len(text)
return '%s\n%s\n\n' % (text, underlining)
def format_directive(module, package=None):
# type: (unicode, unicode) -> unicode
"""Create the automodule directive and add the options."""
directive = '.. automodule:: %s\n' % makename(package, module)
for option in OPTIONS:
@ -82,6 +91,7 @@ def format_directive(module, package=None):
def create_module_file(package, module, opts):
# type: (unicode, unicode, Any) -> None
"""Build the text of the file and write the file."""
if not opts.noheadings:
text = format_heading(1, '%s module' % module)
@ -93,6 +103,7 @@ def create_module_file(package, module, opts):
def create_package_file(root, master_package, subroot, py_files, opts, subs, is_namespace):
# type: (unicode, unicode, unicode, List[unicode], Any, List[unicode], bool) -> None
"""Build the text of the file and write the file."""
text = format_heading(1, ('%s package' if not is_namespace else "%s namespace")
% makename(master_package, subroot))
@ -148,13 +159,14 @@ def create_package_file(root, master_package, subroot, py_files, opts, subs, is_
def create_modules_toc_file(modules, opts, name='modules'):
# type: (List[unicode], Any, unicode) -> None
"""Create the module's index."""
text = format_heading(1, '%s' % opts.header)
text += '.. toctree::\n'
text += ' :maxdepth: %s\n\n' % opts.maxdepth
modules.sort()
prev_module = ''
prev_module = '' # type: unicode
for module in modules:
# look if the module is a subpackage and, if yes, ignore it
if module.startswith(prev_module + '.'):
@ -166,6 +178,7 @@ def create_modules_toc_file(modules, opts, name='modules'):
def shall_skip(module, opts):
# type: (unicode, Any) -> bool
"""Check if we want to skip this module."""
# skip if the file doesn't exist and not using implicit namespaces
if not opts.implicit_namespaces and not path.exists(module):
@ -184,6 +197,7 @@ def shall_skip(module, opts):
def recurse_tree(rootpath, excludes, opts):
# type: (unicode, List[unicode], Any) -> List[unicode]
"""
Look for every file in the directory tree and create the corresponding
ReST files.
@ -217,7 +231,7 @@ def recurse_tree(rootpath, excludes, opts):
# remove hidden ('.') and private ('_') directories, as well as
# excluded dirs
if includeprivate:
exclude_prefixes = ('.',)
exclude_prefixes = ('.',) # type: Tuple[unicode, ...]
else:
exclude_prefixes = ('.', '_')
subs[:] = sorted(sub for sub in subs if not sub.startswith(exclude_prefixes) and
@ -247,23 +261,26 @@ def recurse_tree(rootpath, excludes, opts):
def normalize_excludes(rootpath, excludes):
# type: (unicode, List[unicode]) -> List[unicode]
"""Normalize the excluded directory list."""
return [path.abspath(exclude) for exclude in excludes]
def is_excluded(root, excludes):
# type: (unicode, List[unicode]) -> bool
"""Check if the directory is in the exclude list.
Note: by having trailing slashes, we avoid common prefix issues, like
e.g. an exlude "foo" also accidentally excluding "foobar".
"""
for exclude in excludes:
if fnmatch(root, exclude):
if fnmatch(root, exclude): # type: ignore
return True
return False
def main(argv=sys.argv):
# type: (List[str]) -> int
"""Parse and check the command line arguments."""
parser = optparse.OptionParser(
usage="""\
@ -330,6 +347,11 @@ Note: By default this script will not overwrite already created files.""")
'defaults to --doc-version')
parser.add_option('--version', action='store_true', dest='show_version',
help='Show version information and exit')
group = parser.add_option_group('Extension options')
for ext in EXTENSIONS:
group.add_option('--ext-' + ext, action='store_true',
dest='ext_' + ext, default=False,
help='enable %s extension' % ext)
(opts, args) = parser.parse_args(argv[1:])
@ -359,7 +381,7 @@ Note: By default this script will not overwrite already created files.""")
if opts.full:
from sphinx import quickstart as qs
modules.sort()
prev_module = ''
prev_module = '' # type: unicode
text = ''
for module in modules:
if module.startswith(prev_module + '.'):
@ -388,6 +410,10 @@ Note: By default this script will not overwrite already created files.""")
module_path = rootpath,
append_syspath = opts.append_syspath,
)
enabled_exts = {'ext_' + ext: getattr(opts, 'ext_' + ext)
for ext in EXTENSIONS if getattr(opts, 'ext_' + ext)}
d.update(enabled_exts)
if isinstance(opts.header, binary_type):
d['project'] = d['project'].decode('utf-8')
if isinstance(opts.author, binary_type):

490
sphinx/application.py
View File

@ -15,40 +15,53 @@ from __future__ import print_function
import os
import sys
import types
import warnings
import posixpath
import traceback
from os import path
from collections import deque
from six import iteritems, itervalues, text_type
from six import iteritems, itervalues
from six.moves import cStringIO
from docutils import nodes
from docutils.parsers.rst import convert_directive_function, \
directives, roles
import sphinx
from sphinx import package_dir, locale
from sphinx.roles import XRefRole
from sphinx.config import Config
from sphinx.errors import SphinxError, SphinxWarning, ExtensionError, \
VersionRequirementError, ConfigError
from sphinx.errors import SphinxError, ExtensionError, VersionRequirementError, \
ConfigError
from sphinx.domains import ObjType
from sphinx.domains.std import GenericObject, Target, StandardDomain
from sphinx.deprecation import RemovedInSphinx17Warning, RemovedInSphinx20Warning
from sphinx.environment import BuildEnvironment
from sphinx.io import SphinxStandaloneReader
from sphinx.roles import XRefRole
from sphinx.util import pycompat # noqa: F401
from sphinx.util import import_object
from sphinx.util import logging
from sphinx.util import status_iterator, old_status_iterator, display_chunk
from sphinx.util.tags import Tags
from sphinx.util.osutil import ENOENT
from sphinx.util.logging import is_suppressed_warning
from sphinx.util.console import bold, lightgray, darkgray, darkred, darkgreen, \
term_width_line
from sphinx.util.console import bold, darkgreen # type: ignore
from sphinx.util.i18n import find_catalog_source_files
if False:
# For type annotation
from typing import Any, Callable, IO, Iterable, Iterator, Tuple, Type, Union # NOQA
from docutils.parsers import Parser # NOQA
from docutils.transform import Transform # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.domains import Domain, Index # NOQA
from sphinx.environment.collectors import EnvironmentCollector # NOQA
# List of all known core events. Maps name to arguments description.
events = {
'builder-inited': '',
'env-get-outdated': 'env, added, changed, removed',
'env-get-updated': 'env',
'env-purge-doc': 'env, docname',
'env-before-read-docs': 'env, docnames',
'source-read': 'docname, source text',
@ -60,7 +73,7 @@ events = {
'html-collect-pages': 'builder',
'html-page-context': 'pagename, context, doctree or None',
'build-finished': 'exception',
}
} # type: Dict[unicode, unicode]
builtin_extensions = (
'sphinx.builders.applehelp',
'sphinx.builders.changes',
@ -90,14 +103,23 @@ builtin_extensions = (
'sphinx.directives.other',
'sphinx.directives.patches',
'sphinx.roles',
)
# collectors should be loaded by specific order
'sphinx.environment.collectors.dependencies',
'sphinx.environment.collectors.asset',
'sphinx.environment.collectors.metadata',
'sphinx.environment.collectors.title',
'sphinx.environment.collectors.toctree',
'sphinx.environment.collectors.indexentries',
) # type: Tuple[unicode, ...]
CONFIG_FILENAME = 'conf.py'
ENV_PICKLE_FILENAME = 'environment.pickle'
# list of deprecated extensions. Keys are extension name.
# Values are Sphinx version that merge the extension.
EXTENSION_BLACKLIST = {"sphinxjp.themecore": "1.2"}
EXTENSION_BLACKLIST = {"sphinxjp.themecore": "1.2"} # type: Dict[unicode, unicode]
logger = logging.getLogger(__name__)
class Sphinx(object):
@ -106,19 +128,20 @@ class Sphinx(object):
confoverrides=None, status=sys.stdout, warning=sys.stderr,
freshenv=False, warningiserror=False, tags=None, verbosity=0,
parallel=0):
# type: (unicode, unicode, unicode, unicode, unicode, Dict, IO, IO, bool, bool, List[unicode], int, int) -> None # NOQA
self.verbosity = verbosity
self.next_listener_id = 0
self._extensions = {}
self._extension_metadata = {}
self._additional_source_parsers = {}
self._listeners = {}
self._setting_up_extension = ['?']
self.domains = {}
self._extensions = {} # type: Dict[unicode, Any]
self._extension_metadata = {} # type: Dict[unicode, Dict[unicode, Any]]
self._additional_source_parsers = {} # type: Dict[unicode, Parser]
self._listeners = {} # type: Dict[unicode, Dict[int, Callable]]
self._setting_up_extension = ['?'] # type: List[unicode]
self.domains = {} # type: Dict[unicode, Type[Domain]]
self.buildername = buildername
self.builderclasses = {}
self.builder = None
self.env = None
self.enumerable_nodes = {}
self.builderclasses = {} # type: Dict[unicode, Type[Builder]]
self.builder = None # type: Builder
self.env = None # type: BuildEnvironment
self.enumerable_nodes = {} # type: Dict[nodes.Node, Tuple[unicode, Callable]] # NOQA
self.srcdir = srcdir
self.confdir = confdir
@ -128,44 +151,45 @@ class Sphinx(object):
self.parallel = parallel
if status is None:
self._status = cStringIO()
self._status = cStringIO() # type: IO
self.quiet = True
else:
self._status = status
self.quiet = False
if warning is None:
self._warning = cStringIO()
self._warning = cStringIO() # type: IO
else:
self._warning = warning
self._warncount = 0
self.warningiserror = warningiserror
logging.setup(self, self._status, self._warning)
self._events = events.copy()
self._translators = {}
self._translators = {} # type: Dict[unicode, nodes.GenericNodeVisitor]
# keep last few messages for traceback
self.messagelog = deque(maxlen=10)
self.messagelog = deque(maxlen=10) # type: deque
# say hello to the world
self.info(bold('Running Sphinx v%s' % sphinx.__display_version__))
logger.info(bold('Running Sphinx v%s' % sphinx.__display_version__))
# status code for command-line application
self.statuscode = 0
if not path.isdir(outdir):
self.info('making output directory...')
logger.info('making output directory...')
os.makedirs(outdir)
# read config
self.tags = Tags(tags)
self.config = Config(confdir, CONFIG_FILENAME,
confoverrides or {}, self.tags)
self.config.check_unicode(self.warn)
self.config.check_unicode()
# defer checking types until i18n has been initialized
# initialize some limited config variables before loading extensions
self.config.pre_init_values(self.warn)
self.config.pre_init_values()
# check the Sphinx version if requested
if self.config.needs_sphinx and self.config.needs_sphinx > sphinx.__display_version__:
@ -173,12 +197,6 @@ class Sphinx(object):
'This project needs at least Sphinx v%s and therefore cannot '
'be built with this version.' % self.config.needs_sphinx)
# force preload html_translator_class
if self.config.html_translator_class:
translator_class = self.import_object(self.config.html_translator_class,
'html_translator_class setting')
self.set_translator('html', translator_class)
# set confdir to srcdir if -C given (!= no confdir); a few pieces
# of code expect a confdir to be set
if self.confdir is None:
@ -211,15 +229,15 @@ class Sphinx(object):
)
# now that we know all config values, collect them from conf.py
self.config.init_values(self.warn)
self.config.init_values()
# check extension versions if requested
if self.config.needs_extensions:
for extname, needs_ver in self.config.needs_extensions.items():
if extname not in self._extensions:
self.warn('needs_extensions config value specifies a '
'version requirement for extension %s, but it is '
'not loaded' % extname)
logger.warning('needs_extensions config value specifies a '
'version requirement for extension %s, but it is '
'not loaded', extname)
continue
has_ver = self._extension_metadata[extname]['version']
if has_ver == 'unknown version' or needs_ver > has_ver:
@ -230,12 +248,12 @@ class Sphinx(object):
# check primary_domain if requested
if self.config.primary_domain and self.config.primary_domain not in self.domains:
self.warn('primary_domain %r not found, ignored.' % self.config.primary_domain)
logger.warning('primary_domain %r not found, ignored.', self.config.primary_domain)
# set up translation infrastructure
self._init_i18n()
# check all configuration values for permissible types
self.config.check_types(self.warn)
self.config.check_types()
# set up source_parsers
self._init_source_parsers()
# set up the build environment
@ -246,19 +264,20 @@ class Sphinx(object):
self._init_enumerable_nodes()
def _init_i18n(self):
# type: () -> None
"""Load translated strings from the configured localedirs if enabled in
the configuration.
"""
if self.config.language is not None:
self.info(bold('loading translations [%s]... ' %
self.config.language), nonl=True)
logger.info(bold('loading translations [%s]... ' % self.config.language),
nonl=True)
user_locale_dirs = [
path.join(self.srcdir, x) for x in self.config.locale_dirs]
# compile mo files if sphinx.po file in user locale directories are updated
for catinfo in find_catalog_source_files(
user_locale_dirs, self.config.language, domains=['sphinx'],
charset=self.config.source_encoding):
catinfo.write_mo(self.config.language, self.warn)
catinfo.write_mo(self.config.language)
locale_dirs = [None, path.join(package_dir, 'locale')] + user_locale_dirs
else:
locale_dirs = []
@ -266,11 +285,12 @@ class Sphinx(object):
if self.config.language is not None:
if has_translation or self.config.language == 'en':
# "en" never needs to be translated
self.info('done')
logger.info('done')
else:
self.info('not available for built-in messages')
logger.info('not available for built-in messages')
def _init_source_parsers(self):
# type: () -> None
for suffix, parser in iteritems(self._additional_source_parsers):
if suffix not in self.config.source_suffix:
self.config.source_suffix.append(suffix)
@ -278,32 +298,31 @@ class Sphinx(object):
self.config.source_parsers[suffix] = parser
def _init_env(self, freshenv):
# type: (bool) -> None
if freshenv:
self.env = BuildEnvironment(self.srcdir, self.doctreedir, self.config)
self.env.set_warnfunc(self.warn)
self.env.find_files(self.config, self.buildername)
for domain in self.domains.keys():
self.env.domains[domain] = self.domains[domain](self.env)
else:
try:
self.info(bold('loading pickled environment... '), nonl=True)
logger.info(bold('loading pickled environment... '), nonl=True)
self.env = BuildEnvironment.frompickle(
self.srcdir, self.config, path.join(self.doctreedir, ENV_PICKLE_FILENAME))
self.env.set_warnfunc(self.warn)
self.env.init_managers()
self.env.domains = {}
for domain in self.domains.keys():
# this can raise if the data version doesn't fit
self.env.domains[domain] = self.domains[domain](self.env)
self.info('done')
logger.info('done')
except Exception as err:
if isinstance(err, IOError) and err.errno == ENOENT:
self.info('not yet created')
logger.info('not yet created')
else:
self.info('failed: %s' % err)
logger.info('failed: %s', err)
self._init_env(freshenv=True)
def _init_builder(self, buildername):
# type: (unicode) -> None
if buildername is None:
print('No builder selected, using default: html', file=self._status)
buildername = 'html'
@ -315,12 +334,14 @@ class Sphinx(object):
self.emit('builder-inited')
def _init_enumerable_nodes(self):
# type: () -> None
for node, settings in iteritems(self.enumerable_nodes):
self.env.domains['std'].enumerable_nodes[node] = settings
self.env.get_domain('std').enumerable_nodes[node] = settings # type: ignore
# ---- main "build" method -------------------------------------------------
def build(self, force_all=False, filenames=None):
# type: (bool, List[unicode]) -> None
try:
if force_all:
self.builder.compile_all_catalogs()
@ -335,11 +356,11 @@ class Sphinx(object):
status = (self.statuscode == 0 and
'succeeded' or 'finished with problems')
if self._warncount:
self.info(bold('build %s, %s warning%s.' %
(status, self._warncount,
self._warncount != 1 and 's' or '')))
logger.info(bold('build %s, %s warning%s.' %
(status, self._warncount,
self._warncount != 1 and 's' or '')))
else:
self.info(bold('build %s.' % status))
logger.info(bold('build %s.' % status))
except Exception as err:
# delete the saved env to force a fresh build next time
envfile = path.join(self.doctreedir, ENV_PICKLE_FILENAME)
@ -352,23 +373,9 @@ class Sphinx(object):
self.builder.cleanup()
# ---- logging handling ----------------------------------------------------
def _log(self, message, wfile, nonl=False):
try:
wfile.write(message)
except UnicodeEncodeError:
encoding = getattr(wfile, 'encoding', 'ascii') or 'ascii'
# wfile.write accept only str, not bytes.So, we encode and replace
# non-encodable characters, then decode them.
wfile.write(message.encode(encoding, 'replace').decode(encoding))
if not nonl:
wfile.write('\n')
if hasattr(wfile, 'flush'):
wfile.flush()
self.messagelog.append(message)
def warn(self, message, location=None, prefix='WARNING: ',
type=None, subtype=None, colorfunc=darkred):
def warn(self, message, location=None, prefix=None,
type=None, subtype=None, colorfunc=None):
# type: (unicode, unicode, unicode, unicode, unicode, Callable) -> None
"""Emit a warning.
If *location* is given, it should either be a tuple of (docname, lineno)
@ -384,139 +391,100 @@ class Sphinx(object):
:meth:`.BuildEnvironment.warn` since that will collect all
warnings during parsing for later output.
"""
if is_suppressed_warning(type, subtype, self.config.suppress_warnings):
return
if prefix:
warnings.warn('prefix option of warn() is now deprecated.',
RemovedInSphinx17Warning)
if colorfunc:
warnings.warn('colorfunc option of warn() is now deprecated.',
RemovedInSphinx17Warning)
if isinstance(location, tuple):
docname, lineno = location
if docname:
location = '%s:%s' % (self.env.doc2path(docname), lineno or '')
else:
location = None
warntext = location and '%s: %s%s\n' % (location, prefix, message) or \
'%s%s\n' % (prefix, message)
if self.warningiserror:
raise SphinxWarning(warntext)
self._warncount += 1
self._log(colorfunc(warntext), self._warning, True)
warnings.warn('app.warning() is now deprecated. Use sphinx.util.logging instead.',
RemovedInSphinx20Warning)
logger.warning(message, type=type, subtype=subtype, location=location)
def info(self, message='', nonl=False):
# type: (unicode, bool) -> None
"""Emit an informational message.
If *nonl* is true, don't emit a newline at the end (which implies that
more info output will follow soon.)
"""
self._log(message, self._status, nonl)
warnings.warn('app.info() is now deprecated. Use sphinx.util.logging instead.',
RemovedInSphinx20Warning)
logger.info(message, nonl=nonl)
def verbose(self, message, *args, **kwargs):
"""Emit a verbose informational message.
The message will only be emitted for verbosity levels >= 1 (i.e. at
least one ``-v`` option was given).
The message can contain %-style interpolation placeholders, which is
formatted with either the ``*args`` or ``**kwargs`` when output.
"""
if self.verbosity < 1:
return
if args or kwargs:
message = message % (args or kwargs)
self._log(message, self._status)
# type: (unicode, Any, Any) -> None
"""Emit a verbose informational message."""
warnings.warn('app.verbose() is now deprecated. Use sphinx.util.logging instead.',
RemovedInSphinx20Warning)
logger.verbose(message, *args, **kwargs)
def debug(self, message, *args, **kwargs):
"""Emit a debug-level informational message.
The message will only be emitted for verbosity levels >= 2 (i.e. at
least two ``-v`` options were given).
The message can contain %-style interpolation placeholders, which is
formatted with either the ``*args`` or ``**kwargs`` when output.
"""
if self.verbosity < 2:
return
if args or kwargs:
message = message % (args or kwargs)
self._log(darkgray(message), self._status)
# type: (unicode, Any, Any) -> None
"""Emit a debug-level informational message."""
warnings.warn('app.debug() is now deprecated. Use sphinx.util.logging instead.',
RemovedInSphinx20Warning)
logger.debug(message, *args, **kwargs)
def debug2(self, message, *args, **kwargs):
"""Emit a lowlevel debug-level informational message.
The message will only be emitted for verbosity level 3 (i.e. three
``-v`` options were given).
The message can contain %-style interpolation placeholders, which is
formatted with either the ``*args`` or ``**kwargs`` when output.
"""
if self.verbosity < 3:
return
if args or kwargs:
message = message % (args or kwargs)
self._log(lightgray(message), self._status)
# type: (unicode, Any, Any) -> None
"""Emit a lowlevel debug-level informational message."""
warnings.warn('app.debug2() is now deprecated. Use debug() instead.',
RemovedInSphinx20Warning)
logger.debug(message, *args, **kwargs)
def _display_chunk(chunk):
if isinstance(chunk, (list, tuple)):
if len(chunk) == 1:
return text_type(chunk[0])
return '%s .. %s' % (chunk[0], chunk[-1])
return text_type(chunk)
# type: (Any) -> unicode
warnings.warn('app._display_chunk() is now deprecated. '
'Use sphinx.util.display_chunk() instead.',
RemovedInSphinx17Warning)
return display_chunk(chunk)
def old_status_iterator(self, iterable, summary, colorfunc=darkgreen,
stringify_func=_display_chunk):
l = 0
for item in iterable:
if l == 0:
self.info(bold(summary), nonl=True)
l = 1
self.info(colorfunc(stringify_func(item)) + ' ', nonl=True)
stringify_func=display_chunk):
# type: (Iterable, unicode, Callable, Callable[[Any], unicode]) -> Iterator
warnings.warn('app.old_status_iterator() is now deprecated. '
'Use sphinx.util.status_iterator() instead.',
RemovedInSphinx17Warning)
for item in old_status_iterator(iterable, summary,
color="darkgreen", stringify_func=stringify_func):
yield item
if l == 1:
self.info()
# new version with progress info
def status_iterator(self, iterable, summary, colorfunc=darkgreen, length=0,
stringify_func=_display_chunk):
if length == 0:
for item in self.old_status_iterator(iterable, summary, colorfunc,
stringify_func):
yield item
return
l = 0
summary = bold(summary)
for item in iterable:
l += 1
s = '%s[%3d%%] %s' % (summary, 100 * l / length,
colorfunc(stringify_func(item)))
if self.verbosity:
s += '\n'
else:
s = term_width_line(s)
self.info(s, nonl=True)
# type: (Iterable, unicode, Callable, int, Callable[[Any], unicode]) -> Iterable
warnings.warn('app.status_iterator() is now deprecated. '
'Use sphinx.util.status_iterator() instead.',
RemovedInSphinx17Warning)
for item in status_iterator(iterable, summary, length=length, verbosity=self.verbosity,
color="darkgreen", stringify_func=stringify_func):
yield item
if l > 0:
self.info()
# ---- general extensibility interface -------------------------------------
def setup_extension(self, extension):
# type: (unicode) -> None
"""Import and setup a Sphinx extension module. No-op if called twice."""
self.debug('[app] setting up extension: %r', extension)
logger.debug('[app] setting up extension: %r', extension)
if extension in self._extensions:
return
if extension in EXTENSION_BLACKLIST:
self.warn('the extension %r was already merged with Sphinx since version %s; '
'this extension is ignored.' % (
extension, EXTENSION_BLACKLIST[extension]))
logger.warning('the extension %r was already merged with Sphinx since version %s; '
'this extension is ignored.',
extension, EXTENSION_BLACKLIST[extension])
return
self._setting_up_extension.append(extension)
try:
mod = __import__(extension, None, None, ['setup'])
except ImportError as err:
self.verbose('Original exception:\n' + traceback.format_exc())
logger.verbose('Original exception:\n' + traceback.format_exc())
raise ExtensionError('Could not import extension %s' % extension,
err)
if not hasattr(mod, 'setup'):
self.warn('extension %r has no setup() function; is it really '
'a Sphinx extension module?' % extension)
logger.warning('extension %r has no setup() function; is it really '
'a Sphinx extension module?', extension)
ext_meta = None
else:
try:
@ -536,30 +504,34 @@ class Sphinx(object):
if not ext_meta.get('version'):
ext_meta['version'] = 'unknown version'
except Exception:
self.warn('extension %r returned an unsupported object from '
'its setup() function; it should return None or a '
'metadata dictionary' % extension)
logger.warning('extension %r returned an unsupported object from '
'its setup() function; it should return None or a '
'metadata dictionary', extension)
ext_meta = {'version': 'unknown version'}
self._extensions[extension] = mod
self._extension_metadata[extension] = ext_meta
self._setting_up_extension.pop()
def require_sphinx(self, version):
# type: (unicode) -> None
# check the Sphinx version if requested
if version > sphinx.__display_version__[:3]:
raise VersionRequirementError(version)
def import_object(self, objname, source=None):
# type: (str, unicode) -> Any
"""Import an object from a 'module.name' string."""
return import_object(objname, source=None)
# event interface
def _validate_event(self, event):
# type: (unicode) -> None
if event not in self._events:
raise ExtensionError('Unknown event name: %s' % event)
def connect(self, event, callback):
# type: (unicode, Callable) -> int
self._validate_event(event)
listener_id = self.next_listener_id
if event not in self._listeners:
@ -567,18 +539,20 @@ class Sphinx(object):
else:
self._listeners[event][listener_id] = callback
self.next_listener_id += 1
self.debug('[app] connecting event %r: %r [id=%s]',
event, callback, listener_id)
logger.debug('[app] connecting event %r: %r [id=%s]',
event, callback, listener_id)
return listener_id
def disconnect(self, listener_id):
self.debug('[app] disconnecting event: [id=%s]', listener_id)
# type: (int) -> None
logger.debug('[app] disconnecting event: [id=%s]', listener_id)
for event in itervalues(self._listeners):
event.pop(listener_id, None)
def emit(self, event, *args):
# type: (unicode, Any) -> List
try:
self.debug2('[app] emitting event: %r%s', event, repr(args)[:100])
logger.debug('[app] emitting event: %r%s', event, repr(args)[:100])
except Exception:
# not every object likes to be repr()'d (think
# random stuff coming via autodoc)
@ -590,6 +564,7 @@ class Sphinx(object):
return results
def emit_firstresult(self, event, *args):
# type: (unicode, Any) -> Any
for result in self.emit(event, *args):
if result is not None:
return result
@ -598,7 +573,8 @@ class Sphinx(object):
# registering addon parts
def add_builder(self, builder):
self.debug('[app] adding builder: %r', builder)
# type: (Type[Builder]) -> None
logger.debug('[app] adding builder: %r', builder)
if not hasattr(builder, 'name'):
raise ExtensionError('Builder class %s has no "name" attribute'
% builder)
@ -609,32 +585,36 @@ class Sphinx(object):
self.builderclasses[builder.name] = builder
def add_config_value(self, name, default, rebuild, types=()):
self.debug('[app] adding config value: %r',
(name, default, rebuild) + ((types,) if types else ()))
if name in self.config.values:
# type: (unicode, Any, Union[bool, unicode], Any) -> None
logger.debug('[app] adding config value: %r',
(name, default, rebuild) + ((types,) if types else ())) # type: ignore
if name in self.config:
raise ExtensionError('Config value %r already present' % name)
if rebuild in (False, True):
rebuild = rebuild and 'env' or ''
self.config.values[name] = (default, rebuild, types)
self.config.add(name, default, rebuild, types)
def add_event(self, name):
self.debug('[app] adding event: %r', name)
# type: (unicode) -> None
logger.debug('[app] adding event: %r', name)
if name in self._events:
raise ExtensionError('Event %r already present' % name)
self._events[name] = ''
def set_translator(self, name, translator_class):
self.info(bold('A Translator for the %s builder is changed.' % name))
# type: (unicode, Any) -> None
logger.info(bold('A Translator for the %s builder is changed.' % name))
self._translators[name] = translator_class
def add_node(self, node, **kwds):
self.debug('[app] adding node: %r', (node, kwds))
# type: (nodes.Node, Any) -> None
logger.debug('[app] adding node: %r', (node, kwds))
if not kwds.pop('override', False) and \
hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__):
self.warn('while setting up extension %s: node class %r is '
'already registered, its visitors will be overridden' %
(self._setting_up_extension, node.__name__),
type='app', subtype='add_node')
logger.warning('while setting up extension %s: node class %r is '
'already registered, its visitors will be overridden',
self._setting_up_extension, node.__name__,
type='app', subtype='add_node')
nodes._add_node_class_names([node.__name__])
for key, val in iteritems(kwds):
try:
@ -646,17 +626,15 @@ class Sphinx(object):
if translator is not None:
pass
elif key == 'html':
from sphinx.writers.html import HTMLTranslator as translator
from sphinx.writers.html import HTMLTranslator as translator # type: ignore
elif key == 'latex':
from sphinx.writers.latex import LaTeXTranslator as translator
from sphinx.writers.latex import LaTeXTranslator as translator # type: ignore
elif key == 'text':
from sphinx.writers.text import TextTranslator as translator
from sphinx.writers.text import TextTranslator as translator # type: ignore
elif key == 'man':
from sphinx.writers.manpage import ManualPageTranslator \
as translator
from sphinx.writers.manpage import ManualPageTranslator as translator # type: ignore # NOQA
elif key == 'texinfo':
from sphinx.writers.texinfo import TexinfoTranslator \
as translator
from sphinx.writers.texinfo import TexinfoTranslator as translator # type: ignore # NOQA
else:
# ignore invalid keys for compatibility
continue
@ -665,14 +643,16 @@ class Sphinx(object):
setattr(translator, 'depart_' + node.__name__, depart)
def add_enumerable_node(self, node, figtype, title_getter=None, **kwds):
# type: (nodes.Node, unicode, Callable, Any) -> None
self.enumerable_nodes[node] = (figtype, title_getter)
self.add_node(node, **kwds)
def _directive_helper(self, obj, content=None, arguments=None, **options):
# type: (Any, unicode, Any, Any) -> Any
if isinstance(obj, (types.FunctionType, types.MethodType)):
obj.content = content
obj.arguments = arguments or (0, 0, False)
obj.options = options
obj.content = content # type: ignore
obj.arguments = arguments or (0, 0, False) # type: ignore
obj.options = options # type: ignore
return convert_directive_function(obj)
else:
if content or arguments or options:
@ -681,45 +661,50 @@ class Sphinx(object):
return obj
def add_directive(self, name, obj, content=None, arguments=None, **options):
self.debug('[app] adding directive: %r',
(name, obj, content, arguments, options))
# type: (unicode, Any, unicode, Any, Any) -> None
logger.debug('[app] adding directive: %r',
(name, obj, content, arguments, options))
if name in directives._directives:
self.warn('while setting up extension %s: directive %r is '
'already registered, it will be overridden' %
(self._setting_up_extension[-1], name),
type='app', subtype='add_directive')
logger.warning('while setting up extension %s: directive %r is '
'already registered, it will be overridden',
self._setting_up_extension[-1], name,
type='app', subtype='add_directive')
directives.register_directive(
name, self._directive_helper(obj, content, arguments, **options))
def add_role(self, name, role):
self.debug('[app] adding role: %r', (name, role))
# type: (unicode, Any) -> None
logger.debug('[app] adding role: %r', (name, role))
if name in roles._roles:
self.warn('while setting up extension %s: role %r is '
'already registered, it will be overridden' %
(self._setting_up_extension[-1], name),
type='app', subtype='add_role')
logger.warning('while setting up extension %s: role %r is '
'already registered, it will be overridden',
self._setting_up_extension[-1], name,
type='app', subtype='add_role')
roles.register_local_role(name, role)
def add_generic_role(self, name, nodeclass):
# type: (unicode, Any) -> None
# don't use roles.register_generic_role because it uses
# register_canonical_role
self.debug('[app] adding generic role: %r', (name, nodeclass))
logger.debug('[app] adding generic role: %r', (name, nodeclass))
if name in roles._roles:
self.warn('while setting up extension %s: role %r is '
'already registered, it will be overridden' %
(self._setting_up_extension[-1], name),
type='app', subtype='add_generic_role')
logger.warning('while setting up extension %s: role %r is '
'already registered, it will be overridden',
self._setting_up_extension[-1], name,
type='app', subtype='add_generic_role')
role = roles.GenericRole(name, nodeclass)
roles.register_local_role(name, role)
def add_domain(self, domain):
self.debug('[app] adding domain: %r', domain)
# type: (Type[Domain]) -> None
logger.debug('[app] adding domain: %r', domain)
if domain.name in self.domains:
raise ExtensionError('domain %s already registered' % domain.name)
self.domains[domain.name] = domain
def override_domain(self, domain):
self.debug('[app] overriding domain: %r', domain)
# type: (Type[Domain]) -> None
logger.debug('[app] overriding domain: %r', domain)
if domain.name not in self.domains:
raise ExtensionError('domain %s not yet registered' % domain.name)
if not issubclass(domain, self.domains[domain.name]):
@ -729,21 +714,24 @@ class Sphinx(object):
def add_directive_to_domain(self, domain, name, obj,
content=None, arguments=None, **options):
self.debug('[app] adding directive to domain: %r',
(domain, name, obj, content, arguments, options))
# type: (unicode, unicode, Any, unicode, Any, Any) -> None
logger.debug('[app] adding directive to domain: %r',
(domain, name, obj, content, arguments, options))
if domain not in self.domains:
raise ExtensionError('domain %s not yet registered' % domain)
self.domains[domain].directives[name] = \
self._directive_helper(obj, content, arguments, **options)
def add_role_to_domain(self, domain, name, role):
self.debug('[app] adding role to domain: %r', (domain, name, role))
# type: (unicode, unicode, Any) -> None
logger.debug('[app] adding role to domain: %r', (domain, name, role))
if domain not in self.domains:
raise ExtensionError('domain %s not yet registered' % domain)
self.domains[domain].roles[name] = role
def add_index_to_domain(self, domain, index):
self.debug('[app] adding index to domain: %r', (domain, index))
# type: (unicode, Type[Index]) -> None
logger.debug('[app] adding index to domain: %r', (domain, index))
if domain not in self.domains:
raise ExtensionError('domain %s not yet registered' % domain)
self.domains[domain].indices.append(index)
@ -751,15 +739,16 @@ class Sphinx(object):
def add_object_type(self, directivename, rolename, indextemplate='',
parse_node=None, ref_nodeclass=None, objname='',
doc_field_types=[]):
self.debug('[app] adding object type: %r',
(directivename, rolename, indextemplate, parse_node,
ref_nodeclass, objname, doc_field_types))
# type: (unicode, unicode, unicode, Callable, nodes.Node, unicode, List) -> None
logger.debug('[app] adding object type: %r',
(directivename, rolename, indextemplate, parse_node,
ref_nodeclass, objname, doc_field_types))
StandardDomain.object_types[directivename] = \
ObjType(objname or directivename, rolename)
# create a subclass of GenericObject as the new directive
new_directive = type(directivename, (GenericObject, object),
new_directive = type(directivename, (GenericObject, object), # type: ignore
{'indextemplate': indextemplate,
'parse_node': staticmethod(parse_node),
'parse_node': staticmethod(parse_node), # type: ignore
'doc_field_types': doc_field_types})
StandardDomain.directives[directivename] = new_directive
# XXX support more options?
@ -770,24 +759,27 @@ class Sphinx(object):
def add_crossref_type(self, directivename, rolename, indextemplate='',
ref_nodeclass=None, objname=''):
self.debug('[app] adding crossref type: %r',
(directivename, rolename, indextemplate, ref_nodeclass,
objname))
# type: (unicode, unicode, unicode, nodes.Node, unicode) -> None
logger.debug('[app] adding crossref type: %r',
(directivename, rolename, indextemplate, ref_nodeclass,
objname))
StandardDomain.object_types[directivename] = \
ObjType(objname or directivename, rolename)
# create a subclass of Target as the new directive
new_directive = type(directivename, (Target, object),
new_directive = type(directivename, (Target, object), # type: ignore
{'indextemplate': indextemplate})
StandardDomain.directives[directivename] = new_directive
# XXX support more options?
StandardDomain.roles[rolename] = XRefRole(innernodeclass=ref_nodeclass)
def add_transform(self, transform):
self.debug('[app] adding transform: %r', transform)
# type: (Transform) -> None
logger.debug('[app] adding transform: %r', transform)
SphinxStandaloneReader.transforms.append(transform)
def add_javascript(self, filename):
self.debug('[app] adding javascript: %r', filename)
# type: (unicode) -> None
logger.debug('[app] adding javascript: %r', filename)
from sphinx.builders.html import StandaloneHTMLBuilder
if '://' in filename:
StandaloneHTMLBuilder.script_files.append(filename)
@ -796,7 +788,8 @@ class Sphinx(object):
posixpath.join('_static', filename))
def add_stylesheet(self, filename):
self.debug('[app] adding stylesheet: %r', filename)
# type: (unicode) -> None
logger.debug('[app] adding stylesheet: %r', filename)
from sphinx.builders.html import StandaloneHTMLBuilder
if '://' in filename:
StandaloneHTMLBuilder.css_files.append(filename)
@ -805,43 +798,54 @@ class Sphinx(object):
posixpath.join('_static', filename))
def add_latex_package(self, packagename, options=None):
self.debug('[app] adding latex package: %r', packagename)
# type: (unicode, unicode) -> None
logger.debug('[app] adding latex package: %r', packagename)
if hasattr(self.builder, 'usepackages'): # only for LaTeX builder
self.builder.usepackages.append((packagename, options))
self.builder.usepackages.append((packagename, options)) # type: ignore
def add_lexer(self, alias, lexer):
self.debug('[app] adding lexer: %r', (alias, lexer))
# type: (unicode, Any) -> None
logger.debug('[app] adding lexer: %r', (alias, lexer))
from sphinx.highlighting import lexers
if lexers is None:
return
lexers[alias] = lexer
def add_autodocumenter(self, cls):
self.debug('[app] adding autodocumenter: %r', cls)
# type: (Any) -> None
logger.debug('[app] adding autodocumenter: %r', cls)
from sphinx.ext import autodoc
autodoc.add_documenter(cls)
self.add_directive('auto' + cls.objtype, autodoc.AutoDirective)
def add_autodoc_attrgetter(self, type, getter):
self.debug('[app] adding autodoc attrgetter: %r', (type, getter))
# type: (Any, Callable) -> None
logger.debug('[app] adding autodoc attrgetter: %r', (type, getter))
from sphinx.ext import autodoc
autodoc.AutoDirective._special_attrgetters[type] = getter
def add_search_language(self, cls):
self.debug('[app] adding search language: %r', cls)
# type: (Any) -> None
logger.debug('[app] adding search language: %r', cls)
from sphinx.search import languages, SearchLanguage
assert issubclass(cls, SearchLanguage)
languages[cls.lang] = cls
def add_source_parser(self, suffix, parser):
self.debug('[app] adding search source_parser: %r, %r', suffix, parser)
# type: (unicode, Parser) -> None
logger.debug('[app] adding search source_parser: %r, %r', suffix, parser)
if suffix in self._additional_source_parsers:
self.warn('while setting up extension %s: source_parser for %r is '
'already registered, it will be overridden' %
(self._setting_up_extension[-1], suffix),
type='app', subtype='add_source_parser')
logger.warning('while setting up extension %s: source_parser for %r is '
'already registered, it will be overridden',
self._setting_up_extension[-1], suffix,
type='app', subtype='add_source_parser')
self._additional_source_parsers[suffix] = parser
def add_env_collector(self, collector):
# type: (Type[EnvironmentCollector]) -> None
logger.debug('[app] adding environment collector: %r', collector)
collector().enable(self)
class TemplateBridge(object):
"""
@ -850,6 +854,7 @@ class TemplateBridge(object):
"""
def init(self, builder, theme=None, dirs=None):
# type: (Builder, unicode, List[unicode]) -> None
"""Called by the builder to initialize the template system.
*builder* is the builder object; you'll probably want to look at the
@ -861,6 +866,7 @@ class TemplateBridge(object):
raise NotImplementedError('must be implemented in subclasses')
def newest_template_mtime(self):
# type: () -> float
"""Called by the builder to determine if output files are outdated
because of template changes. Return the mtime of the newest template
file that was changed. The default implementation returns ``0``.
@ -868,12 +874,14 @@ class TemplateBridge(object):
return 0
def render(self, template, context):
# type: (unicode, Dict) -> None
"""Called by the builder to render a template given as a filename with
a specified context (a Python dictionary).
"""
raise NotImplementedError('must be implemented in subclasses')
def render_string(self, template, context):
# type: (unicode, Dict) -> unicode
"""Called by the builder to render a template given as a string with a
specified context (a Python dictionary).
"""

173
sphinx/builders/__init__.py
View File

@ -19,10 +19,10 @@ except ImportError:
from docutils import nodes
from sphinx.util import i18n, path_stabilize
from sphinx.util import i18n, path_stabilize, logging, status_iterator
from sphinx.util.osutil import SEP, relative_uri
from sphinx.util.i18n import find_catalog
from sphinx.util.console import bold, darkgreen
from sphinx.util.console import bold # type: ignore
from sphinx.util.parallel import ParallelTasks, SerialTasks, make_chunks, \
parallel_available
@ -30,6 +30,18 @@ from sphinx.util.parallel import ParallelTasks, SerialTasks, make_chunks, \
from sphinx import roles # noqa
from sphinx import directives # noqa
if False:
# For type annotation
from typing import Any, Callable, Iterable, Sequence, Tuple, Union # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.config import Config # NOQA
from sphinx.environment import BuildEnvironment # NOQA
from sphinx.util.i18n import CatalogInfo # NOQA
from sphinx.util.tags import Tags # NOQA
logger = logging.getLogger(__name__)
class Builder(object):
"""
@ -47,7 +59,8 @@ class Builder(object):
allow_parallel = False
def __init__(self, app):
self.env = app.env
# type: (Sphinx) -> None
self.env = app.env # type: BuildEnvironment
self.env.set_versioning_method(self.versioning_method,
self.versioning_compare)
self.srcdir = app.srcdir
@ -57,11 +70,11 @@ class Builder(object):
if not path.isdir(self.doctreedir):
os.makedirs(self.doctreedir)
self.app = app
self.warn = app.warn
self.info = app.info
self.config = app.config
self.tags = app.tags
self.app = app # type: Sphinx
self.warn = app.warn # type: Callable
self.info = app.info # type: Callable
self.config = app.config # type: Config
self.tags = app.tags # type: Tags
self.tags.add(self.format)
self.tags.add(self.name)
self.tags.add("format_%s" % self.format)
@ -71,7 +84,7 @@ class Builder(object):
self.old_status_iterator = app.old_status_iterator
# images that need to be copied over (source -> dest)
self.images = {}
self.images = {} # type: Dict[unicode, unicode]
# basename of images directory
self.imagedir = ""
# relative path to image directory from current docname (used at writing docs)
@ -79,7 +92,7 @@ class Builder(object):
# these get set later
self.parallel_ok = False
self.finish_tasks = None
self.finish_tasks = None # type: Any
# load default translator class
self.translator_class = app._translators.get(self.name)
@ -88,12 +101,14 @@ class Builder(object):
# helper methods
def init(self):
# type: () -> None
"""Load necessary templates and perform initialization. The default
implementation does nothing.
"""
pass
def create_template_bridge(self):
# type: () -> None
"""Return the template bridge configured."""
if self.config.template_bridge:
self.templates = self.app.import_object(
@ -103,6 +118,7 @@ class Builder(object):
self.templates = BuiltinTemplateLoader()
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
"""Return the target URI for a document name.
*typ* can be used to qualify the link characteristic for individual
@ -111,6 +127,7 @@ class Builder(object):
raise NotImplementedError
def get_relative_uri(self, from_, to, typ=None):
# type: (unicode, unicode, unicode) -> unicode
"""Return a relative URI between two source filenames.
May raise environment.NoUri if there's no way to return a sensible URI.
@ -119,6 +136,7 @@ class Builder(object):
self.get_target_uri(to, typ))
def get_outdated_docs(self):
# type: () -> Union[unicode, Iterable[unicode]]
"""Return an iterable of output files that are outdated, or a string
describing what an update build will build.
@ -128,9 +146,10 @@ class Builder(object):
"""
raise NotImplementedError
supported_image_types = []
supported_image_types = [] # type: List[unicode]
def post_process_images(self, doctree):
# type: (nodes.Node) -> None
"""Pick the best candidate for all image URIs."""
for node in doctree.traverse(nodes.image):
if '?' in node['candidates']:
@ -142,9 +161,8 @@ class Builder(object):
if candidate:
break
else:
self.warn(
'no matching candidate for image URI %r' % node['uri'],
'%s:%s' % (node.source, getattr(node, 'line', '')))
logger.warning('no matching candidate for image URI %r', node['uri'],
location=node)
continue
node['uri'] = candidate
else:
@ -157,19 +175,22 @@ class Builder(object):
# compile po methods
def compile_catalogs(self, catalogs, message):
# type: (Set[CatalogInfo], unicode) -> None
if not self.config.gettext_auto_build:
return
def cat2relpath(cat):
# type: (CatalogInfo) -> unicode
return path.relpath(cat.mo_path, self.env.srcdir).replace(path.sep, SEP)
self.info(bold('building [mo]: ') + message)
for catalog in self.app.status_iterator(
catalogs, 'writing output... ', darkgreen, len(catalogs),
cat2relpath):
catalog.write_mo(self.config.language, self.warn)
logger.info(bold('building [mo]: ') + message)
for catalog in status_iterator(catalogs, 'writing output... ', "darkgreen",
len(catalogs), self.app.verbosity,
stringify_func=cat2relpath):
catalog.write_mo(self.config.language)
def compile_all_catalogs(self):
# type: () -> None
catalogs = i18n.find_catalog_source_files(
[path.join(self.srcdir, x) for x in self.config.locale_dirs],
self.config.language,
@ -180,7 +201,9 @@ class Builder(object):
self.compile_catalogs(catalogs, message)
def compile_specific_catalogs(self, specified_files):
# type: (List[unicode]) -> None
def to_domain(fpath):
# type: (unicode) -> unicode
docname, _ = path.splitext(path_stabilize(fpath))
dom = find_catalog(docname, self.config.gettext_compact)
return dom
@ -196,6 +219,7 @@ class Builder(object):
self.compile_catalogs(catalogs, message)
def compile_update_catalogs(self):
# type: () -> None
catalogs = i18n.find_catalog_source_files(
[path.join(self.srcdir, x) for x in self.config.locale_dirs],
self.config.language,
@ -207,26 +231,29 @@ class Builder(object):
# build methods
def build_all(self):
# type: () -> None
"""Build all source files."""
self.build(None, summary='all source files', method='all')
def build_specific(self, filenames):
# type: (List[unicode]) -> None
"""Only rebuild as much as needed for changes in the *filenames*."""
# bring the filenames to the canonical format, that is,
# relative to the source directory and without source_suffix.
dirlen = len(self.srcdir) + 1
to_write = []
suffixes = tuple(self.config.source_suffix)
suffixes = None # type: Tuple[unicode]
suffixes = tuple(self.config.source_suffix) # type: ignore
for filename in filenames:
filename = path.normpath(path.abspath(filename))
if not filename.startswith(self.srcdir):
self.warn('file %r given on command line is not under the '
'source directory, ignoring' % filename)
logger.warning('file %r given on command line is not under the '
'source directory, ignoring', filename)
continue
if not (path.isfile(filename) or
any(path.isfile(filename + suffix) for suffix in suffixes)):
self.warn('file %r given on command line does not exist, '
'ignoring' % filename)
logger.warning('file %r given on command line does not exist, '
'ignoring', filename)
continue
filename = filename[dirlen:]
for suffix in suffixes:
@ -240,6 +267,7 @@ class Builder(object):
'line' % len(to_write))
def build_update(self):
# type: () -> None
"""Only rebuild what was changed or added since last build."""
to_build = self.get_outdated_docs()
if isinstance(to_build, str):
@ -251,46 +279,43 @@ class Builder(object):
'out of date' % len(to_build))
def build(self, docnames, summary=None, method='update'):
# type: (Iterable[unicode], unicode, unicode) -> None
"""Main build method.
First updates the environment, and then calls :meth:`write`.
"""
if summary:
self.info(bold('building [%s]' % self.name) + ': ' + summary)
logger.info(bold('building [%s]' % self.name) + ': ' + summary)
# while reading, collect all warnings from docutils
warnings = []
self.env.set_warnfunc(lambda *args, **kwargs: warnings.append((args, kwargs)))
updated_docnames = set(self.env.update(self.config, self.srcdir,
self.doctreedir, self.app))
self.env.set_warnfunc(self.warn)
for warning, kwargs in warnings:
self.warn(*warning, **kwargs)
with logging.pending_warnings():
updated_docnames = set(self.env.update(self.config, self.srcdir,
self.doctreedir, self.app))
doccount = len(updated_docnames)
self.info(bold('looking for now-outdated files... '), nonl=1)
for docname in self.env.check_dependents(updated_docnames):
logger.info(bold('looking for now-outdated files... '), nonl=1)
for docname in self.env.check_dependents(self.app, updated_docnames):
updated_docnames.add(docname)
outdated = len(updated_docnames) - doccount
if outdated:
self.info('%d found' % outdated)
logger.info('%d found', outdated)
else:
self.info('none found')
logger.info('none found')
if updated_docnames:
# save the environment
from sphinx.application import ENV_PICKLE_FILENAME
self.info(bold('pickling environment... '), nonl=True)
logger.info(bold('pickling environment... '), nonl=True)
self.env.topickle(path.join(self.doctreedir, ENV_PICKLE_FILENAME))
self.info('done')
logger.info('done')
# global actions
self.info(bold('checking consistency... '), nonl=True)
logger.info(bold('checking consistency... '), nonl=True)
self.env.check_consistency()
self.info('done')
logger.info('done')
else:
if method == 'update' and not docnames:
self.info(bold('no targets are out of date.'))
logger.info(bold('no targets are out of date.'))
return
# filter "docnames" (list of outdated files) by the updated
@ -306,8 +331,8 @@ class Builder(object):
for extname, md in self.app._extension_metadata.items():
par_ok = md.get('parallel_write_safe', True)
if not par_ok:
self.app.warn('the %s extension is not safe for parallel '
'writing, doing serial write' % extname)
logger.warning('the %s extension is not safe for parallel '
'writing, doing serial write', extname)
self.parallel_ok = False
break
@ -328,6 +353,7 @@ class Builder(object):
self.finish_tasks.join()
def write(self, build_docnames, updated_docnames, method='update'):
# type: (Iterable[unicode], Sequence[unicode], unicode) -> None
if build_docnames is None or build_docnames == ['__all__']:
# build_all
build_docnames = self.env.found_docs
@ -336,52 +362,42 @@ class Builder(object):
docnames = set(build_docnames) | set(updated_docnames)
else:
docnames = set(build_docnames)
self.app.debug('docnames to write: %s', ', '.join(sorted(docnames)))
logger.debug('docnames to write: %s', ', '.join(sorted(docnames)))
# add all toctree-containing files that may have changed
for docname in list(docnames):
for tocdocname in self.env.files_to_rebuild.get(docname, []):
for tocdocname in self.env.files_to_rebuild.get(docname, set()):
if tocdocname in self.env.found_docs:
docnames.add(tocdocname)
docnames.add(self.config.master_doc)
self.info(bold('preparing documents... '), nonl=True)
logger.info(bold('preparing documents... '), nonl=True)
self.prepare_writing(docnames)
self.info('done')
logger.info('done')
warnings = []
self.env.set_warnfunc(lambda *args, **kwargs: warnings.append((args, kwargs)))
if self.parallel_ok:
# number of subprocesses is parallel-1 because the main process
# is busy loading doctrees and doing write_doc_serialized()
self._write_parallel(sorted(docnames), warnings,
self._write_parallel(sorted(docnames),
nproc=self.app.parallel - 1)
else:
self._write_serial(sorted(docnames), warnings)
self.env.set_warnfunc(self.warn)
self._write_serial(sorted(docnames))
def _write_serial(self, docnames, warnings):
for docname in self.app.status_iterator(
docnames, 'writing output... ', darkgreen, len(docnames)):
doctree = self.env.get_and_resolve_doctree(docname, self)
self.write_doc_serialized(docname, doctree)
self.write_doc(docname, doctree)
for warning, kwargs in warnings:
self.warn(*warning, **kwargs)
def _write_serial(self, docnames):
# type: (Sequence[unicode]) -> None
with logging.pending_warnings():
for docname in status_iterator(docnames, 'writing output... ', "darkgreen",
len(docnames), self.app.verbosity):
doctree = self.env.get_and_resolve_doctree(docname, self)
self.write_doc_serialized(docname, doctree)
self.write_doc(docname, doctree)
def _write_parallel(self, docnames, warnings, nproc):
def _write_parallel(self, docnames, nproc):
# type: (Sequence[unicode], int) -> None
def write_process(docs):
local_warnings = []
def warnfunc(*args, **kwargs):
local_warnings.append((args, kwargs))
self.env.set_warnfunc(warnfunc)
# type: (List[Tuple[unicode, nodes.Node]]) -> None
for docname, doctree in docs:
self.write_doc(docname, doctree)
return local_warnings
def add_warnings(docs, wlist):
warnings.extend(wlist)
# warm up caches/compile templates using the first document
firstname, docnames = docnames[0], docnames[1:]
@ -392,37 +408,38 @@ class Builder(object):
tasks = ParallelTasks(nproc)
chunks = make_chunks(docnames, nproc)
for chunk in self.app.status_iterator(
chunks, 'writing output... ', darkgreen, len(chunks)):
for chunk in status_iterator(chunks, 'writing output... ', "darkgreen",
len(chunks), self.app.verbosity):
arg = []
for i, docname in enumerate(chunk):
doctree = self.env.get_and_resolve_doctree(docname, self)
self.write_doc_serialized(docname, doctree)
arg.append((docname, doctree))
tasks.add_task(write_process, arg, add_warnings)
tasks.add_task(write_process, arg)
# make sure all threads have finished
self.info(bold('waiting for workers...'))
logger.info(bold('waiting for workers...'))
tasks.join()
for warning, kwargs in warnings:
self.warn(*warning, **kwargs)
def prepare_writing(self, docnames):
# type: (Set[unicode]) -> None
"""A place where you can add logic before :meth:`write_doc` is run"""
raise NotImplementedError
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
"""Where you actually write something to the filesystem."""
raise NotImplementedError
def write_doc_serialized(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
"""Handle parts of write_doc that must be called in the main process
if parallel build is active.
"""
pass
def finish(self):
# type: () -> None
"""Finish the building process.
The default implementation does nothing.
@ -430,6 +447,7 @@ class Builder(object):
pass
def cleanup(self):
# type: () -> None
"""Cleanup any resources.
The default implementation does nothing.
@ -437,6 +455,7 @@ class Builder(object):
pass
def get_builder_config(self, option, default):
# type: (unicode, unicode) -> Any
"""Return a builder specific option.
This method allows customization of common builder settings by

59
sphinx/builders/applehelp.py
View File

@ -18,8 +18,9 @@ import shlex
from sphinx.builders.html import StandaloneHTMLBuilder
from sphinx.config import string_classes
from sphinx.util import logging
from sphinx.util.osutil import copyfile, ensuredir, make_filename
from sphinx.util.console import bold
from sphinx.util.console import bold # type: ignore
from sphinx.util.fileutil import copy_asset
from sphinx.util.pycompat import htmlescape
from sphinx.util.matching import Matcher
@ -28,10 +29,17 @@ from sphinx.errors import SphinxError
import plistlib
import subprocess
if False:
# For type annotation
from typing import Any # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
# Use plistlib.dump in 3.4 and above
try:
write_plist = plistlib.dump
write_plist = plistlib.dump # type: ignore
except AttributeError:
write_plist = plistlib.writePlist
@ -83,6 +91,7 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
search = False
def init(self):
# type: () -> None
super(AppleHelpBuilder, self).init()
# the output files for HTML help must be .html only
self.out_suffix = '.html'
@ -101,25 +110,28 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
self.config.applehelp_locale + '.lproj')
def handle_finish(self):
# type: () -> None
super(AppleHelpBuilder, self).handle_finish()
self.finish_tasks.add_task(self.copy_localized_files)
self.finish_tasks.add_task(self.build_helpbook)
def copy_localized_files(self):
# type: () -> None
source_dir = path.join(self.confdir, self.config.applehelp_locale + '.lproj')
target_dir = self.outdir
if path.isdir(source_dir):
self.info(bold('copying localized files... '), nonl=True)
logger.info(bold('copying localized files... '), nonl=True)
excluded = Matcher(self.config.exclude_patterns + ['**/.*'])
copy_asset(source_dir, target_dir, excluded,
context=self.globalcontext, renderer=self.templates)
self.info('done')
logger.info('done')
def build_helpbook(self):
# type: () -> None
contents_dir = path.join(self.bundle_path, 'Contents')
resources_dir = path.join(contents_dir, 'Resources')
language_dir = path.join(resources_dir,
@ -157,37 +169,36 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
if self.config.applehelp_remote_url is not None:
info_plist['HPDBookRemoteURL'] = self.config.applehelp_remote_url
self.info(bold('writing Info.plist... '), nonl=True)
logger.info(bold('writing Info.plist... '), nonl=True)
with open(path.join(contents_dir, 'Info.plist'), 'wb') as f:
write_plist(info_plist, f)
self.info('done')
logger.info('done')
# Copy the icon, if one is supplied
if self.config.applehelp_icon:
self.info(bold('copying icon... '), nonl=True)
logger.info(bold('copying icon... '), nonl=True)
try:
copyfile(path.join(self.srcdir, self.config.applehelp_icon),
path.join(resources_dir, info_plist['HPDBookIconPath']))
self.info('done')
logger.info('done')
except Exception as err:
self.warn('cannot copy icon file %r: %s' %
(path.join(self.srcdir, self.config.applehelp_icon),
err))
logger.warning('cannot copy icon file %r: %s',
path.join(self.srcdir, self.config.applehelp_icon), err)
del info_plist['HPDBookIconPath']
# Build the access page
self.info(bold('building access page...'), nonl=True)
logger.info(bold('building access page...'), nonl=True)
with codecs.open(path.join(language_dir, '_access.html'), 'w') as f:
f.write(access_page_template % {
'toc': htmlescape(toc, quote=True),
'title': htmlescape(self.config.applehelp_title)
})
self.info('done')
logger.info('done')
# Generate the help index
self.info(bold('generating help index... '), nonl=True)
logger.info(bold('generating help index... '), nonl=True)
args = [
self.config.applehelp_indexer_path,
@ -209,10 +220,10 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
args += ['-l', self.config.applehelp_locale]
if self.config.applehelp_disable_external_tools:
self.info('skipping')
logger.info('skipping')
self.warn('you will need to index this help book with:\n %s'
% (' '.join([pipes.quote(arg) for arg in args])))
logger.warning('you will need to index this help book with:\n %s',
' '.join([pipes.quote(arg) for arg in args]))
else:
try:
p = subprocess.Popen(args,
@ -224,13 +235,13 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
if p.returncode != 0:
raise AppleHelpIndexerFailed(output)
else:
self.info('done')
logger.info('done')
except OSError:
raise AppleHelpIndexerFailed('Command not found: %s' % args[0])
# If we've been asked to, sign the bundle
if self.config.applehelp_codesign_identity:
self.info(bold('signing help book... '), nonl=True)
logger.info(bold('signing help book... '), nonl=True)
args = [
self.config.applehelp_codesign_path,
@ -243,10 +254,9 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
args.append(self.bundle_path)
if self.config.applehelp_disable_external_tools:
self.info('skipping')
self.warn('you will need to sign this help book with:\n %s'
% (' '.join([pipes.quote(arg) for arg in args])))
logger.info('skipping')
logger.warning('you will need to sign this help book with:\n %s',
' '.join([pipes.quote(arg) for arg in args]))
else:
try:
p = subprocess.Popen(args,
@ -258,12 +268,13 @@ class AppleHelpBuilder(StandaloneHTMLBuilder):
if p.returncode != 0:
raise AppleHelpCodeSigningFailed(output)
else:
self.info('done')
logger.info('done')
except OSError:
raise AppleHelpCodeSigningFailed('Command not found: %s' % args[0])
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.setup_extension('sphinx.builders.html')
app.add_builder(AppleHelpBuilder)

45
sphinx/builders/changes.py
View File

@ -18,11 +18,20 @@ from sphinx import package_dir
from sphinx.locale import _
from sphinx.theming import Theme
from sphinx.builders import Builder
from sphinx.util import logging
from sphinx.util.osutil import ensuredir, os_path
from sphinx.util.console import bold
from sphinx.util.console import bold # type: ignore
from sphinx.util.fileutil import copy_asset_file
from sphinx.util.pycompat import htmlescape
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
class ChangesBuilder(Builder):
"""
@ -31,30 +40,32 @@ class ChangesBuilder(Builder):
name = 'changes'
def init(self):
# type: () -> None
self.create_template_bridge()
Theme.init_themes(self.confdir, self.config.html_theme_path,
warn=self.warn)
Theme.init_themes(self.confdir, self.config.html_theme_path)
self.theme = Theme('default')
self.templates.init(self, self.theme)
def get_outdated_docs(self):
# type: () -> unicode
return self.outdir
typemap = {
'versionadded': 'added',
'versionchanged': 'changed',
'deprecated': 'deprecated',
}
} # type: Dict[unicode, unicode]
def write(self, *ignored):
# type: (Any) -> None
version = self.config.version
libchanges = {}
apichanges = []
otherchanges = {}
libchanges = {} # type: Dict[unicode, List[Tuple[unicode, unicode, int]]]
apichanges = [] # type: List[Tuple[unicode, unicode, int]]
otherchanges = {} # type: Dict[Tuple[unicode, unicode], List[Tuple[unicode, unicode, int]]] # NOQA
if version not in self.env.versionchanges:
self.info(bold('no changes in version %s.' % version))
logger.info(bold('no changes in version %s.' % version))
return
self.info(bold('writing summary file...'))
logger.info(bold('writing summary file...'))
for type, docname, lineno, module, descname, content in \
self.env.versionchanges[version]:
if isinstance(descname, tuple):
@ -101,9 +112,9 @@ class ChangesBuilder(Builder):
'show_copyright': self.config.html_show_copyright,
'show_sphinx': self.config.html_show_sphinx,
}
with codecs.open(path.join(self.outdir, 'index.html'), 'w', 'utf8') as f:
with codecs.open(path.join(self.outdir, 'index.html'), 'w', 'utf8') as f: # type: ignore # NOQA
f.write(self.templates.render('changes/frameset.html', ctx))
with codecs.open(path.join(self.outdir, 'changes.html'), 'w', 'utf8') as f:
with codecs.open(path.join(self.outdir, 'changes.html'), 'w', 'utf8') as f: # type: ignore # NOQA
f.write(self.templates.render('changes/versionchanges.html', ctx))
hltext = ['.. versionadded:: %s' % version,
@ -111,6 +122,7 @@ class ChangesBuilder(Builder):
'.. deprecated:: %s' % version]
def hl(no, line):
# type: (int, unicode) -> unicode
line = '<a name="L%s"> </a>' % no + htmlescape(line)
for x in hltext:
if x in line:
@ -118,18 +130,18 @@ class ChangesBuilder(Builder):
break
return line
self.info(bold('copying source files...'))
logger.info(bold('copying source files...'))
for docname in self.env.all_docs:
with codecs.open(self.env.doc2path(docname), 'r',
with codecs.open(self.env.doc2path(docname), 'r', # type: ignore
self.env.config.source_encoding) as f:
try:
lines = f.readlines()
except UnicodeDecodeError:
self.warn('could not read %r for changelog creation' % docname)
logger.warning('could not read %r for changelog creation', docname)
continue
targetfn = path.join(self.outdir, 'rst', os_path(docname)) + '.html'
ensuredir(path.dirname(targetfn))
with codecs.open(targetfn, 'w', 'utf-8') as f:
with codecs.open(targetfn, 'w', 'utf-8') as f: # type: ignore
text = ''.join(hl(i + 1, line) for (i, line) in enumerate(lines))
ctx = {
'filename': self.env.doc2path(docname, None),
@ -144,6 +156,7 @@ class ChangesBuilder(Builder):
self.outdir)
def hl(self, text, version):
# type: (unicode, unicode) -> unicode
text = htmlescape(text)
for directive in ['versionchanged', 'versionadded', 'deprecated']:
text = text.replace('.. %s:: %s' % (directive, version),
@ -151,10 +164,12 @@ class ChangesBuilder(Builder):
return text
def finish(self):
# type: () -> None
pass
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(ChangesBuilder)
return {

25
sphinx/builders/devhelp.py
View File

@ -19,13 +19,23 @@ from os import path
from docutils import nodes
from sphinx import addnodes
from sphinx.util import logging
from sphinx.util.osutil import make_filename
from sphinx.builders.html import StandaloneHTMLBuilder
from sphinx.environment.adapters.indexentries import IndexEntries
try:
import xml.etree.ElementTree as etree
except ImportError:
import lxml.etree as etree
import lxml.etree as etree # type: ignore
if False:
# For type annotation
from typing import Any # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
class DevhelpBuilder(StandaloneHTMLBuilder):
@ -44,15 +54,18 @@ class DevhelpBuilder(StandaloneHTMLBuilder):
embedded = True
def init(self):
# type: () -> None
StandaloneHTMLBuilder.init(self)
self.out_suffix = '.html'
self.link_suffix = '.html'
def handle_finish(self):
# type: () -> None
self.build_devhelp(self.outdir, self.config.devhelp_basename)
def build_devhelp(self, outdir, outname):
self.info('dumping devhelp index...')
# type: (unicode, unicode) -> None
logger.info('dumping devhelp index...')
# Basic info
root = etree.Element('book',
@ -69,6 +82,7 @@ class DevhelpBuilder(StandaloneHTMLBuilder):
self.config.master_doc, self, prune_toctrees=False)
def write_toc(node, parent):
# type: (nodes.Node, nodes.Node) -> None
if isinstance(node, addnodes.compact_paragraph) or \
isinstance(node, nodes.bullet_list):
for subnode in node:
@ -82,6 +96,7 @@ class DevhelpBuilder(StandaloneHTMLBuilder):
parent.attrib['name'] = node.astext()
def istoctree(node):
# type: (nodes.Node) -> bool
return isinstance(node, addnodes.compact_paragraph) and \
'toctree' in node
@ -90,9 +105,10 @@ class DevhelpBuilder(StandaloneHTMLBuilder):
# Index
functions = etree.SubElement(root, 'functions')
index = self.env.create_index(self)
index = IndexEntries(self.env).create_index(self)
def write_index(title, refs, subitems):
# type: (unicode, List[Any], Any) -> None
if len(refs) == 0:
pass
elif len(refs) == 1:
@ -116,11 +132,12 @@ class DevhelpBuilder(StandaloneHTMLBuilder):
# Dump the XML file
xmlfile = path.join(outdir, outname + '.devhelp.gz')
with gzip.open(xmlfile, 'w') as f:
with gzip.open(xmlfile, 'w') as f: # type: ignore
tree.write(f, 'utf-8')
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.setup_extension('sphinx.builders.html')
app.add_builder(DevhelpBuilder)

13
sphinx/builders/dummy.py
View File

@ -12,31 +12,44 @@
from sphinx.builders import Builder
if False:
# For type annotation
from typing import Any # NOQA
from docutils import nodes # NOQA
from sphinx.application import Sphinx # NOQA
class DummyBuilder(Builder):
name = 'dummy'
allow_parallel = True
def init(self):
# type: () -> None
pass
def get_outdated_docs(self):
# type: () -> Set[unicode]
return self.env.found_docs
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
return ''
def prepare_writing(self, docnames):
# type: (Set[unicode]) -> None
pass
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
pass
def finish(self):
# type: () -> None
pass
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(DummyBuilder)
return {

115
sphinx/builders/epub.py
View File

@ -29,9 +29,18 @@ from docutils import nodes
from sphinx import addnodes
from sphinx.builders.html import StandaloneHTMLBuilder
from sphinx.util import logging
from sphinx.util import status_iterator
from sphinx.util.osutil import ensuredir, copyfile, make_filename, EEXIST
from sphinx.util.smartypants import sphinx_smarty_pants as ssp
from sphinx.util.console import brown
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
# (Fragment) templates from which the metainfo files content.opf, toc.ncx,
@ -159,7 +168,7 @@ MEDIA_TYPES = {
'.otf': 'application/x-font-otf',
'.ttf': 'application/x-font-ttf',
'.woff': 'application/font-woff',
}
} # type: Dict[unicode, unicode]
VECTOR_GRAPHICS_EXTENSIONS = ('.svg',)
@ -221,6 +230,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
refuri_re = REFURI_RE
def init(self):
# type: () -> None
StandaloneHTMLBuilder.init(self)
# the output files for epub must be .html only
self.out_suffix = '.xhtml'
@ -230,10 +240,12 @@ class EpubBuilder(StandaloneHTMLBuilder):
self.use_index = self.get_builder_config('use_index', 'epub')
def get_theme_config(self):
# type: () -> Tuple[unicode, Dict]
return self.config.epub_theme, self.config.epub_theme_options
# generic support functions
def make_id(self, name, id_cache={}):
# type: (unicode, Dict[unicode, unicode]) -> unicode
# id_cache is intentionally mutable
"""Return a unique id for name."""
id = id_cache.get(name)
@ -243,6 +255,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
return id
def esc(self, name):
# type: (unicode) -> unicode
"""Replace all characters not allowed in text an attribute values."""
# Like cgi.escape, but also replace apostrophe
name = name.replace('&', '&amp;')
@ -253,6 +266,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
return name
def get_refnodes(self, doctree, result):
# type: (nodes.Node, List[Dict[unicode, Any]]) -> List[Dict[unicode, Any]]
"""Collect section titles, their depth in the toc and the refuri."""
# XXX: is there a better way than checking the attribute
# toctree-l[1-8] on the parent node?
@ -276,6 +290,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
return result
def get_toc(self):
# type: () -> None
"""Get the total table of contents, containing the master_doc
and pre and post files not managed by sphinx.
"""
@ -291,6 +306,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
self.toc_add_files(self.refnodes)
def toc_add_files(self, refnodes):
# type: (List[nodes.Node]) -> None
"""Add the master_doc, pre and post files to a list of refnodes.
"""
refnodes.insert(0, {
@ -313,10 +329,12 @@ class EpubBuilder(StandaloneHTMLBuilder):
})
def fix_fragment(self, prefix, fragment):
# type: (unicode, unicode) -> unicode
"""Return a href/id attribute with colons replaced by hyphens."""
return prefix + fragment.replace(':', '-')
def fix_ids(self, tree):
# type: (nodes.Node) -> None
"""Replace colons with hyphens in href and id attributes.
Some readers crash because they interpret the part as a
@ -337,9 +355,11 @@ class EpubBuilder(StandaloneHTMLBuilder):
node.attributes['ids'] = newids
def add_visible_links(self, tree, show_urls='inline'):
# type: (nodes.Node, unicode) -> None
"""Add visible link targets for external links"""
def make_footnote_ref(doc, label):
# type: (nodes.Node, unicode) -> nodes.footnote_reference
"""Create a footnote_reference node with children"""
footnote_ref = nodes.footnote_reference('[#]_')
footnote_ref.append(nodes.Text(label))
@ -347,6 +367,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
return footnote_ref
def make_footnote(doc, label, uri):
# type: (nodes.Node, unicode, unicode) -> nodes.footnote
"""Create a footnote node with children"""
footnote = nodes.footnote(uri)
para = nodes.paragraph()
@ -357,6 +378,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
return footnote
def footnote_spot(tree):
# type: (nodes.Node) -> Tuple[nodes.Node, int]
"""Find or create a spot to place footnotes.
The function returns the tuple (parent, index)."""
@ -406,6 +428,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
fn_idx += 1
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
"""Write one document file.
This method is overwritten in order to fix fragment identifiers
@ -416,6 +439,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
StandaloneHTMLBuilder.write_doc(self, docname, doctree)
def fix_genindex(self, tree):
# type: (nodes.Node) -> None
"""Fix href attributes for genindex pages."""
# XXX: modifies tree inline
# Logic modeled from themes/basic/genindex.html
@ -434,31 +458,33 @@ class EpubBuilder(StandaloneHTMLBuilder):
self.fix_fragment(m.group(1), m.group(2)))
def is_vector_graphics(self, filename):
# type: (unicode) -> bool
"""Does the filename extension indicate a vector graphic format?"""
ext = path.splitext(filename)[-1]
return ext in VECTOR_GRAPHICS_EXTENSIONS
def copy_image_files_pil(self):
# type: () -> None
"""Copy images using the PIL.
The method tries to read and write the files with the PIL,
converting the format and resizing the image if necessary/possible.
"""
ensuredir(path.join(self.outdir, self.imagedir))
for src in self.app.status_iterator(self.images, 'copying images... ',
brown, len(self.images)):
for src in status_iterator(self.images, 'copying images... ', "brown",
len(self.images), self.app.verbosity):
dest = self.images[src]
try:
img = Image.open(path.join(self.srcdir, src))
except IOError:
if not self.is_vector_graphics(src):
self.warn('cannot read image file %r: copying it instead' %
(path.join(self.srcdir, src), ))
logger.warning('cannot read image file %r: copying it instead',
path.join(self.srcdir, src))
try:
copyfile(path.join(self.srcdir, src),
path.join(self.outdir, self.imagedir, dest))
except (IOError, OSError) as err:
self.warn('cannot copy image file %r: %s' %
(path.join(self.srcdir, src), err))
logger.warning('cannot copy image file %r: %s',
path.join(self.srcdir, src), err)
continue
if self.config.epub_fix_images:
if img.mode in ('P',):
@ -473,17 +499,18 @@ class EpubBuilder(StandaloneHTMLBuilder):
try:
img.save(path.join(self.outdir, self.imagedir, dest))
except (IOError, OSError) as err:
self.warn('cannot write image file %r: %s' %
(path.join(self.srcdir, src), err))
logger.warning('cannot write image file %r: %s',
path.join(self.srcdir, src), err)
def copy_image_files(self):
# type: () -> None
"""Copy image files to destination directory.
This overwritten method can use the PIL to convert image files.
"""
if self.images:
if self.config.epub_fix_images or self.config.epub_max_image_width:
if not Image:
self.warn('PIL not found - copying image files')
logger.warning('PIL not found - copying image files')
super(EpubBuilder, self).copy_image_files()
else:
self.copy_image_files_pil()
@ -491,10 +518,12 @@ class EpubBuilder(StandaloneHTMLBuilder):
super(EpubBuilder, self).copy_image_files()
def copy_download_files(self):
# type: () -> None
pass
def handle_page(self, pagename, addctx, templatename='page.html',
outfilename=None, event_arg=None):
# type: (unicode, Dict, unicode, unicode, Any) -> None
"""Create a rendered page.
This method is overwritten for genindex pages in order to fix href link
@ -510,6 +539,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
# Finish by building the epub file
def handle_finish(self):
# type: () -> None
"""Create the metainfo files and finally the epub."""
self.get_toc()
self.build_mimetype(self.outdir, 'mimetype')
@ -519,28 +549,31 @@ class EpubBuilder(StandaloneHTMLBuilder):
self.build_epub(self.outdir, self.config.epub_basename + '.epub')
def build_mimetype(self, outdir, outname):
# type: (unicode, unicode) -> None
"""Write the metainfo file mimetype."""
self.info('writing %s file...' % outname)
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f:
logger.info('writing %s file...', outname)
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f: # type: ignore
f.write(self.mimetype_template)
def build_container(self, outdir, outname):
# type: (unicode, unicode) -> None
"""Write the metainfo file META-INF/cointainer.xml."""
self.info('writing %s file...' % outname)
logger.info('writing %s file...', outname)
fn = path.join(outdir, outname)
try:
os.mkdir(path.dirname(fn))
except OSError as err:
if err.errno != EEXIST:
raise
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f:
f.write(self.container_template)
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f: # type: ignore
f.write(self.container_template) # type: ignore
def content_metadata(self, files, spine, guide):
# type: (List[unicode], List[unicode], List[unicode]) -> Dict[unicode, Any]
"""Create a dictionary with all metadata for the content.opf
file properly escaped.
"""
metadata = {}
metadata = {} # type: Dict[unicode, Any]
metadata['title'] = self.esc(self.config.epub_title)
metadata['author'] = self.esc(self.config.epub_author)
metadata['uid'] = self.esc(self.config.epub_uid)
@ -556,17 +589,18 @@ class EpubBuilder(StandaloneHTMLBuilder):
return metadata
def build_content(self, outdir, outname):
# type: (unicode, unicode) -> None
"""Write the metainfo file content.opf It contains bibliographic data,
a file list and the spine (the reading order).
"""
self.info('writing %s file...' % outname)
logger.info('writing %s file...', outname)
# files
if not outdir.endswith(os.sep):
outdir += os.sep
olen = len(outdir)
projectfiles = []
self.files = []
projectfiles = [] # type: List[unicode]
self.files = [] # type: List[unicode]
self.ignored_files = ['.buildinfo', 'mimetype', 'content.opf',
'toc.ncx', 'META-INF/container.xml',
'Thumbs.db', 'ehthumbs.db', '.DS_Store',
@ -584,8 +618,8 @@ class EpubBuilder(StandaloneHTMLBuilder):
# we always have JS and potentially OpenSearch files, don't
# always warn about them
if ext not in ('.js', '.xml'):
self.warn('unknown mimetype for %s, ignoring' % filename,
type='epub', subtype='unknown_project_files')
logger.warning('unknown mimetype for %s, ignoring', filename,
type='epub', subtype='unknown_project_files')
continue
filename = filename.replace(os.sep, '/')
projectfiles.append(self.file_template % {
@ -680,16 +714,17 @@ class EpubBuilder(StandaloneHTMLBuilder):
'title': self.guide_titles['toc'],
'uri': self.esc(self.refnodes[0]['refuri'])
})
projectfiles = '\n'.join(projectfiles)
spine = '\n'.join(spine)
guide = '\n'.join(guide)
projectfiles = '\n'.join(projectfiles) # type: ignore
spine = '\n'.join(spine) # type: ignore
guide = '\n'.join(guide) # type: ignore
# write the project file
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f:
f.write(content_tmpl %
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f: # type: ignore
f.write(content_tmpl % # type: ignore
self.content_metadata(projectfiles, spine, guide))
def new_navpoint(self, node, level, incr=True):
# type: (nodes.Node, int, bool) -> unicode
"""Create a new entry in the toc from the node at given level."""
# XXX Modifies the node
if incr:
@ -701,6 +736,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
return self.navpoint_template % node
def insert_subnav(self, node, subnav):
# type: (nodes.Node, unicode) -> unicode
"""Insert nested navpoints for given node.
The node and subnav are already rendered to text.
@ -710,6 +746,7 @@ class EpubBuilder(StandaloneHTMLBuilder):
return '\n'.join(nlist)
def build_navpoints(self, nodes):
# type: (nodes.Node) -> unicode
"""Create the toc navigation structure.
Subelements of a node are nested inside the navpoint. For nested nodes
@ -753,10 +790,11 @@ class EpubBuilder(StandaloneHTMLBuilder):
return '\n'.join(navlist)
def toc_metadata(self, level, navpoints):
# type: (int, List[unicode]) -> Dict[unicode, Any]
"""Create a dictionary with all metadata for the toc.ncx file
properly escaped.
"""
metadata = {}
metadata = {} # type: Dict[unicode, Any]
metadata['uid'] = self.config.epub_uid
metadata['title'] = self.config.epub_title
metadata['level'] = level
@ -764,8 +802,9 @@ class EpubBuilder(StandaloneHTMLBuilder):
return metadata
def build_toc(self, outdir, outname):
# type: (unicode, unicode) -> None
"""Write the metainfo file toc.ncx."""
self.info('writing %s file...' % outname)
logger.info('writing %s file...', outname)
if self.config.epub_tocscope == 'default':
doctree = self.env.get_and_resolve_doctree(self.config.master_doc,
@ -779,29 +818,31 @@ class EpubBuilder(StandaloneHTMLBuilder):
navpoints = self.build_navpoints(refnodes)
level = max(item['level'] for item in self.refnodes)
level = min(level, self.config.epub_tocdepth)
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f:
f.write(self.toc_template % self.toc_metadata(level, navpoints))
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f: # type: ignore
f.write(self.toc_template % self.toc_metadata(level, navpoints)) # type: ignore
def build_epub(self, outdir, outname):
# type: (unicode, unicode) -> None
"""Write the epub file.
It is a zip file with the mimetype file stored uncompressed as the first
entry.
"""
self.info('writing %s file...' % outname)
projectfiles = ['META-INF/container.xml', 'content.opf', 'toc.ncx'] \
+ self.files
epub = zipfile.ZipFile(path.join(outdir, outname), 'w',
logger.info('writing %s file...', outname)
projectfiles = ['META-INF/container.xml', 'content.opf', 'toc.ncx'] # type: List[unicode] # NOQA
projectfiles.extend(self.files)
epub = zipfile.ZipFile(path.join(outdir, outname), 'w', # type: ignore
zipfile.ZIP_DEFLATED)
epub.write(path.join(outdir, 'mimetype'), 'mimetype',
epub.write(path.join(outdir, 'mimetype'), 'mimetype', # type: ignore
zipfile.ZIP_STORED)
for file in projectfiles:
fp = path.join(outdir, file)
epub.write(fp, file, zipfile.ZIP_DEFLATED)
epub.write(fp, file, zipfile.ZIP_DEFLATED) # type: ignore
epub.close()
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.setup_extension('sphinx.builders.html')
app.add_builder(EpubBuilder)

50
sphinx/builders/epub3.py
View File

@ -16,6 +16,15 @@ from datetime import datetime
from sphinx.config import string_classes
from sphinx.builders.epub import EpubBuilder
from sphinx.util import logging
if False:
# For type annotation
from typing import Any, Iterable # NOQA
from docutils import nodes # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
# (Fragment) templates from which the metainfo files content.opf, toc.ncx,
@ -113,6 +122,7 @@ class Epub3Builder(EpubBuilder):
# Finish by building the epub file
def handle_finish(self):
# type: () -> None
"""Create the metainfo files and finally the epub."""
self.get_toc()
self.build_mimetype(self.outdir, 'mimetype')
@ -123,6 +133,7 @@ class Epub3Builder(EpubBuilder):
self.build_epub(self.outdir, self.config.epub_basename + '.epub')
def content_metadata(self, files, spine, guide):
# type: (List[unicode], List[unicode], List[unicode]) -> Dict
"""Create a dictionary with all metadata for the content.opf
file properly escaped.
"""
@ -137,6 +148,7 @@ class Epub3Builder(EpubBuilder):
return metadata
def _page_progression_direction(self):
# type: () -> unicode
if self.config.epub_writing_mode == 'horizontal':
page_progression_direction = 'ltr'
elif self.config.epub_writing_mode == 'vertical':
@ -146,6 +158,7 @@ class Epub3Builder(EpubBuilder):
return page_progression_direction
def _ibook_scroll_axis(self):
# type: () -> unicode
if self.config.epub_writing_mode == 'horizontal':
scroll_axis = 'vertical'
elif self.config.epub_writing_mode == 'vertical':
@ -155,6 +168,7 @@ class Epub3Builder(EpubBuilder):
return scroll_axis
def _css_writing_mode(self):
# type: () -> unicode
if self.config.epub_writing_mode == 'vertical':
editing_mode = 'vertical-rl'
else:
@ -162,10 +176,12 @@ class Epub3Builder(EpubBuilder):
return editing_mode
def prepare_writing(self, docnames):
# type: (Iterable[unicode]) -> None
super(Epub3Builder, self).prepare_writing(docnames)
self.globalcontext['theme_writing_mode'] = self._css_writing_mode()
def new_navlist(self, node, level, has_child):
# type: (nodes.Node, int, bool) -> unicode
"""Create a new entry in the toc from the node at given level."""
# XXX Modifies the node
self.tocid += 1
@ -176,14 +192,17 @@ class Epub3Builder(EpubBuilder):
return self.navlist_template % node
def begin_navlist_block(self, level):
# type: (int) -> unicode
return self.navlist_template_begin_block % {
"indent": self.navlist_indent * level
}
def end_navlist_block(self, level):
# type: (int) -> unicode
return self.navlist_template_end_block % {"indent": self.navlist_indent * level}
def build_navlist(self, nodes):
def build_navlist(self, navnodes):
# type: (List[nodes.Node]) -> unicode
"""Create the toc navigation structure.
This method is almost same as build_navpoints method in epub.py.
@ -196,7 +215,7 @@ class Epub3Builder(EpubBuilder):
navlist = []
level = 1
usenodes = []
for node in nodes:
for node in navnodes:
if not node['text']:
continue
file = node['refuri'].split('#')[0]
@ -224,6 +243,7 @@ class Epub3Builder(EpubBuilder):
return '\n'.join(navlist)
def navigation_doc_metadata(self, navlist):
# type: (unicode) -> Dict
"""Create a dictionary with all metadata for the nav.xhtml file
properly escaped.
"""
@ -234,8 +254,9 @@ class Epub3Builder(EpubBuilder):
return metadata
def build_navigation_doc(self, outdir, outname):
# type: (unicode, unicode) -> None
"""Write the metainfo file nav.xhtml."""
self.info('writing %s file...' % outname)
logger.info('writing %s file...', outname)
if self.config.epub_tocscope == 'default':
doctree = self.env.get_and_resolve_doctree(
@ -247,8 +268,8 @@ class Epub3Builder(EpubBuilder):
# 'includehidden'
refnodes = self.refnodes
navlist = self.build_navlist(refnodes)
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f:
f.write(self.navigation_doc_template %
with codecs.open(path.join(outdir, outname), 'w', 'utf-8') as f: # type: ignore
f.write(self.navigation_doc_template % # type: ignore
self.navigation_doc_metadata(navlist))
# Add nav.xhtml to epub file
@ -256,31 +277,14 @@ class Epub3Builder(EpubBuilder):
self.files.append(outname)
def validate_config_values(app):
if app.config.epub3_description is not None:
app.warn('epub3_description is deprecated. Use epub_description instead.')
app.config.epub_description = app.config.epub3_description
if app.config.epub3_contributor is not None:
app.warn('epub3_contributor is deprecated. Use epub_contributor instead.')
app.config.epub_contributor = app.config.epub3_contributor
if app.config.epub3_page_progression_direction is not None:
app.warn('epub3_page_progression_direction option is deprecated'
' from 1.5. Use epub_writing_mode instead.')
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.setup_extension('sphinx.builders.epub')
app.add_builder(Epub3Builder)
app.connect('builder-inited', validate_config_values)
app.add_config_value('epub_description', '', 'epub3', string_classes)
app.add_config_value('epub_contributor', 'unknown', 'epub3', string_classes)
app.add_config_value('epub_writing_mode', 'horizontal', 'epub3', string_classes)
app.add_config_value('epub3_description', None, 'epub3', string_classes)
app.add_config_value('epub3_contributor', None, 'epub3', string_classes)
app.add_config_value('epub3_page_progression_direction', None, 'epub3', string_classes)
return {
'version': 'builtin',

75
sphinx/builders/gettext.py
View File

@ -21,14 +21,24 @@ from uuid import uuid4
from six import iteritems
from sphinx.builders import Builder
from sphinx.util import split_index_msg
from sphinx.util import split_index_msg, logging, status_iterator
from sphinx.util.tags import Tags
from sphinx.util.nodes import extract_messages, traverse_translatable_index
from sphinx.util.osutil import safe_relpath, ensuredir, canon_path
from sphinx.util.i18n import find_catalog
from sphinx.util.console import darkgreen, purple, bold
from sphinx.util.console import bold # type: ignore
from sphinx.locale import pairindextypes
if False:
# For type annotation
from typing import Any, Iterable, Tuple # NOQA
from docutils import nodes # NOQA
from sphinx.util.i18n import CatalogInfo # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
POHEADER = r"""
# SOME DESCRIPTIVE TITLE.
# Copyright (C) %(copyright)s
@ -55,10 +65,14 @@ class Catalog(object):
"""Catalog of translatable messages."""
def __init__(self):
self.messages = [] # retain insertion order, a la OrderedDict
self.metadata = {} # msgid -> file, line, uid
# type: () -> None
self.messages = [] # type: List[unicode]
# retain insertion order, a la OrderedDict
self.metadata = {} # type: Dict[unicode, List[Tuple[unicode, int, unicode]]]
# msgid -> file, line, uid
def add(self, msg, origin):
# type: (unicode, MsgOrigin) -> None
if not hasattr(origin, 'uid'):
# Nodes that are replicated like todo don't have a uid,
# however i18n is also unnecessary.
@ -75,6 +89,7 @@ class MsgOrigin(object):
"""
def __init__(self, source, line):
# type: (unicode, int) -> None
self.source = source
self.line = line
self.uid = uuid4().hex
@ -87,6 +102,7 @@ class I18nTags(Tags):
always returns True value even if no tags are defined.
"""
def eval_condition(self, condition):
# type: (Any) -> bool
return True
@ -99,27 +115,34 @@ class I18nBuilder(Builder):
versioning_compare = None # be set by `gettext_uuid`
def __init__(self, app):
# type: (Sphinx) -> None
self.versioning_compare = app.env.config.gettext_uuid
super(I18nBuilder, self).__init__(app)
def init(self):
# type: () -> None
Builder.init(self)
self.tags = I18nTags()
self.catalogs = defaultdict(Catalog)
self.catalogs = defaultdict(Catalog) # type: defaultdict[unicode, Catalog]
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
return ''
def get_outdated_docs(self):
# type: () -> Set[unicode]
return self.env.found_docs
def prepare_writing(self, docnames):
# type: (Set[unicode]) -> None
return
def compile_catalogs(self, catalogs, message):
# type: (Set[CatalogInfo], unicode) -> None
return
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
catalog = self.catalogs[find_catalog(docname,
self.config.gettext_compact)]
@ -153,13 +176,16 @@ if source_date_epoch is not None:
class LocalTimeZone(tzinfo):
def __init__(self, *args, **kw):
super(LocalTimeZone, self).__init__(*args, **kw)
# type: (Any, Any) -> None
super(LocalTimeZone, self).__init__(*args, **kw) # type: ignore
self.tzdelta = tzdelta
def utcoffset(self, dt):
# type: (datetime) -> timedelta
return self.tzdelta
def dst(self, dt):
# type: (datetime) -> timedelta
return timedelta(0)
@ -173,11 +199,13 @@ class MessageCatalogBuilder(I18nBuilder):
name = 'gettext'
def init(self):
# type: () -> None
I18nBuilder.init(self)
self.create_template_bridge()
self.templates.init(self)
def _collect_templates(self):
# type: () -> Set[unicode]
template_files = set()
for template_path in self.config.templates_path:
tmpl_abs_path = path.join(self.app.srcdir, template_path)
@ -189,66 +217,71 @@ class MessageCatalogBuilder(I18nBuilder):
return template_files
def _extract_from_template(self):
# type: () -> None
files = self._collect_templates()
self.info(bold('building [%s]: ' % self.name), nonl=1)
self.info('targets for %d template files' % len(files))
logger.info(bold('building [%s]: ' % self.name), nonl=1)
logger.info('targets for %d template files', len(files))
extract_translations = self.templates.environment.extract_translations
for template in self.app.status_iterator(
files, 'reading templates... ', purple, len(files)):
with open(template, 'r', encoding='utf-8') as f:
for template in status_iterator(files, 'reading templates... ', "purple", # type: ignore # NOQA
len(files), self.app.verbosity):
with open(template, 'r', encoding='utf-8') as f: # type: ignore
context = f.read()
for line, meth, msg in extract_translations(context):
origin = MsgOrigin(template, line)
self.catalogs['sphinx'].add(msg, origin)
def build(self, docnames, summary=None, method='update'):
# type: (Iterable[unicode], unicode, unicode) -> None
self._extract_from_template()
I18nBuilder.build(self, docnames, summary, method)
def finish(self):
# type: () -> None
I18nBuilder.finish(self)
data = dict(
version = self.config.version,
copyright = self.config.copyright,
project = self.config.project,
ctime = datetime.fromtimestamp(
ctime = datetime.fromtimestamp( # type: ignore
timestamp, ltz).strftime('%Y-%m-%d %H:%M%z'),
)
for textdomain, catalog in self.app.status_iterator(
iteritems(self.catalogs), "writing message catalogs... ",
darkgreen, len(self.catalogs),
lambda textdomain__: textdomain__[0]):
for textdomain, catalog in status_iterator(iteritems(self.catalogs), # type: ignore
"writing message catalogs... ",
"darkgreen", len(self.catalogs),
self.app.verbosity,
lambda textdomain__: textdomain__[0]):
# noop if config.gettext_compact is set
ensuredir(path.join(self.outdir, path.dirname(textdomain)))
pofn = path.join(self.outdir, textdomain + '.pot')
with open(pofn, 'w', encoding='utf-8') as pofile:
pofile.write(POHEADER % data)
with open(pofn, 'w', encoding='utf-8') as pofile: # type: ignore
pofile.write(POHEADER % data) # type: ignore
for message in catalog.messages:
positions = catalog.metadata[message]
if self.config.gettext_location:
# generate "#: file1:line1\n#: file2:line2 ..."
pofile.write("#: %s\n" % "\n#: ".join(
pofile.write("#: %s\n" % "\n#: ".join( # type: ignore
"%s:%s" % (canon_path(
safe_relpath(source, self.outdir)), line)
for source, line, _ in positions))
if self.config.gettext_uuid:
# generate "# uuid1\n# uuid2\n ..."
pofile.write("# %s\n" % "\n# ".join(
pofile.write("# %s\n" % "\n# ".join( # type: ignore
uid for _, _, uid in positions))
# message contains *one* line of text ready for translation
message = message.replace('\\', r'\\'). \
replace('"', r'\"'). \
replace('\n', '\\n"\n"')
pofile.write('msgid "%s"\nmsgstr ""\n\n' % message)
pofile.write('msgid "%s"\nmsgstr ""\n\n' % message) # type: ignore
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(MessageCatalogBuilder)
app.add_config_value('gettext_compact', True, 'gettext')

292
sphinx/builders/html.py
View File

@ -19,6 +19,7 @@ from hashlib import md5
from six import iteritems, text_type, string_types
from six.moves import cPickle as pickle
from docutils import nodes
from docutils.io import DocTreeInput, StringOutput
from docutils.core import Publisher
@ -27,7 +28,7 @@ from docutils.frontend import OptionParser
from docutils.readers.doctree import Reader as DoctreeReader
from sphinx import package_dir, __display_version__
from sphinx.util import jsonimpl
from sphinx.util import jsonimpl, logging, status_iterator
from sphinx.util.i18n import format_date
from sphinx.util.osutil import SEP, os_path, relative_uri, ensuredir, \
movefile, copyfile
@ -41,17 +42,28 @@ from sphinx.theming import Theme
from sphinx.builders import Builder
from sphinx.application import ENV_PICKLE_FILENAME
from sphinx.highlighting import PygmentsBridge
from sphinx.util.console import bold, darkgreen, brown
from sphinx.util.console import bold, darkgreen # type: ignore
from sphinx.writers.html import HTMLWriter, HTMLTranslator, \
SmartyPantsHTMLTranslator
from sphinx.environment.adapters.toctree import TocTree
from sphinx.environment.adapters.indexentries import IndexEntries
if False:
# For type annotation
from typing import Any, Iterable, Iterator, Type, Tuple, Union # NOQA
from sphinx.domains import Domain, Index # NOQA
from sphinx.application import Sphinx # NOQA
#: the filename for the inventory of objects
INVENTORY_FILENAME = 'objects.inv'
#: the filename for the "last build" file (for serializing builders)
LAST_BUILD_FILENAME = 'last_build'
logger = logging.getLogger(__name__)
def get_stable_hash(obj):
# type: (Any) -> unicode
"""
Return a stable hash for a Python data structure. We can't just use
the md5 of str(obj) since for example dictionary items are enumerated
@ -85,13 +97,17 @@ class StandaloneHTMLBuilder(Builder):
allow_sharp_as_current_path = True
embedded = False # for things like HTML help or Qt help: suppresses sidebar
search = True # for things like HTML help and Apple help: suppress search
use_index = False
download_support = True # enable download role
# This is a class attribute because it is mutated by Sphinx.add_javascript.
script_files = ['_static/jquery.js', '_static/underscore.js',
'_static/doctools.js']
'_static/doctools.js'] # type: List[unicode]
# Dito for this one.
css_files = []
css_files = [] # type: List[unicode]
imgpath = None # type: unicode
domain_indices = [] # type: List[Tuple[unicode, Type[Index], List[Tuple[unicode, List[List[Union[unicode, int]]]]], bool]] # NOQA
default_sidebars = ['localtoc.html', 'relations.html',
'sourcelink.html', 'searchbox.html']
@ -100,15 +116,16 @@ class StandaloneHTMLBuilder(Builder):
_publisher = None
def init(self):
# type: () -> None
# a hash of all config values that, if changed, cause a full rebuild
self.config_hash = ''
self.tags_hash = ''
self.config_hash = '' # type: unicode
self.tags_hash = '' # type: unicode
# basename of images directory
self.imagedir = '_images'
# section numbers for headings in the currently visited document
self.secnumbers = {}
self.secnumbers = {} # type: Dict[unicode, Tuple[int, ...]]
# currently written docname
self.current_docname = None
self.current_docname = None # type: unicode
self.init_templates()
self.init_highlighter()
@ -127,31 +144,35 @@ class StandaloneHTMLBuilder(Builder):
self.use_index = self.get_builder_config('use_index', 'html')
def _get_translations_js(self):
candidates = [path.join(package_dir, 'locale', self.config.language,
# type: () -> unicode
candidates = [path.join(dir, self.config.language,
'LC_MESSAGES', 'sphinx.js')
for dir in self.config.locale_dirs] + \
[path.join(package_dir, 'locale', self.config.language,
'LC_MESSAGES', 'sphinx.js'),
path.join(sys.prefix, 'share/sphinx/locale',
self.config.language, 'sphinx.js')] + \
[path.join(dir, self.config.language,
'LC_MESSAGES', 'sphinx.js')
for dir in self.config.locale_dirs]
self.config.language, 'sphinx.js')]
for jsfile in candidates:
if path.isfile(jsfile):
return jsfile
return None
def get_theme_config(self):
# type: () -> Tuple[unicode, Dict]
return self.config.html_theme, self.config.html_theme_options
def init_templates(self):
Theme.init_themes(self.confdir, self.config.html_theme_path,
warn=self.warn)
# type: () -> None
Theme.init_themes(self.confdir, self.config.html_theme_path)
themename, themeoptions = self.get_theme_config()
self.theme = Theme(themename, warn=self.warn)
self.theme = Theme(themename)
self.theme_options = themeoptions.copy()
self.create_template_bridge()
self.templates.init(self, self.theme)
def init_highlighter(self):
# type: () -> None
# determine Pygments style and create the highlighter
if self.config.pygments_style is not None:
style = self.config.pygments_style
@ -163,6 +184,7 @@ class StandaloneHTMLBuilder(Builder):
self.config.trim_doctest_flags)
def init_translator_class(self):
# type: () -> None
if self.translator_class is None:
if self.config.html_use_smartypants:
self.translator_class = SmartyPantsHTMLTranslator
@ -170,11 +192,10 @@ class StandaloneHTMLBuilder(Builder):
self.translator_class = HTMLTranslator
def get_outdated_docs(self):
cfgdict = dict((name, self.config[name])
for (name, desc) in iteritems(self.config.values)
if desc[1] == 'html')
# type: () -> Iterator[unicode]
cfgdict = dict((confval.name, confval.value) for confval in self.config.filter('html'))
self.config_hash = get_stable_hash(cfgdict)
self.tags_hash = get_stable_hash(sorted(self.tags))
self.tags_hash = get_stable_hash(sorted(self.tags)) # type: ignore
old_config_hash = old_tags_hash = ''
try:
with open(path.join(self.outdir, '.buildinfo')) as fp:
@ -189,8 +210,8 @@ class StandaloneHTMLBuilder(Builder):
if tag != 'tags':
raise ValueError
except ValueError:
self.warn('unsupported build info format in %r, building all' %
path.join(self.outdir, '.buildinfo'))
logger.warning('unsupported build info format in %r, building all',
path.join(self.outdir, '.buildinfo'))
except Exception:
pass
if old_config_hash != self.config_hash or \
@ -222,6 +243,7 @@ class StandaloneHTMLBuilder(Builder):
pass
def render_partial(self, node):
# type: (nodes.Nodes) -> Dict[unicode, unicode]
"""Utility: Render a lone doctree node."""
if node is None:
return {'fragment': ''}
@ -247,6 +269,7 @@ class StandaloneHTMLBuilder(Builder):
return pub.writer.parts
def prepare_writing(self, docnames):
# type: (Iterable[unicode]) -> nodes.Node
# create the search indexer
self.indexer = None
if self.search:
@ -272,16 +295,13 @@ class StandaloneHTMLBuilder(Builder):
indices_config = self.config.html_domain_indices
if indices_config:
for domain_name in sorted(self.env.domains):
domain = None # type: Domain
domain = self.env.domains[domain_name]
for indexcls in domain.indices:
indexname = '%s-%s' % (domain.name, indexcls.name)
indexname = '%s-%s' % (domain.name, indexcls.name) # type: unicode
if isinstance(indices_config, list):
if indexname not in indices_config:
continue
# deprecated config value
if indexname == 'py-modindex' and \
not self.config.html_use_modindex:
continue
content, collapse = indexcls(domain).generate()
if content:
self.domain_indices.append(
@ -292,8 +312,7 @@ class StandaloneHTMLBuilder(Builder):
lufmt = self.config.html_last_updated_fmt
if lufmt is not None:
self.last_updated = format_date(lufmt or _('%b %d, %Y'),
language=self.config.language,
warn=self.warn)
language=self.config.language)
else:
self.last_updated = None
@ -303,14 +322,14 @@ class StandaloneHTMLBuilder(Builder):
favicon = self.config.html_favicon and \
path.basename(self.config.html_favicon) or ''
if favicon and os.path.splitext(favicon)[1] != '.ico':
self.warn('html_favicon is not an .ico file')
logger.warning('html_favicon is not an .ico file')
if not isinstance(self.config.html_use_opensearch, string_types):
self.warn('html_use_opensearch config value must now be a string')
logger.warning('html_use_opensearch config value must now be a string')
self.relations = self.env.collect_relations()
rellinks = []
rellinks = [] # type: List[Tuple[unicode, unicode, unicode, unicode]]
if self.use_index:
rellinks.append(('genindex', _('General Index'), 'I', _('index')))
for indexname, indexcls, content, collapse in self.domain_indices:
@ -353,7 +372,7 @@ class StandaloneHTMLBuilder(Builder):
parents = [],
logo = logo,
favicon = favicon,
)
) # type: Dict[unicode, Any]
if self.theme:
self.globalcontext.update(
('theme_' + key, val) for (key, val) in
@ -361,6 +380,7 @@ class StandaloneHTMLBuilder(Builder):
self.globalcontext.update(self.config.html_context)
def get_doc_context(self, docname, body, metatags):
# type: (unicode, unicode, Dict) -> Dict[unicode, Any]
"""Collect items for the template context of a page."""
# find out relations
prev = next = None
@ -421,7 +441,7 @@ class StandaloneHTMLBuilder(Builder):
meta = self.env.metadata.get(docname)
# local TOC and global TOC tree
self_toc = self.env.get_toc_for(docname, self)
self_toc = TocTree(self.env).get_toc_for(docname, self)
toc = self.render_partial(self_toc)['fragment']
return dict(
@ -441,6 +461,7 @@ class StandaloneHTMLBuilder(Builder):
)
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
destination = StringOutput(encoding='utf-8')
doctree.settings = self.docsettings
@ -458,6 +479,7 @@ class StandaloneHTMLBuilder(Builder):
self.handle_page(docname, ctx, event_arg=doctree)
def write_doc_serialized(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
self.imgpath = relative_uri(self.get_target_uri(docname), self.imagedir)
self.post_process_images(doctree)
title = self.env.longtitles.get(docname)
@ -465,6 +487,7 @@ class StandaloneHTMLBuilder(Builder):
self.index_page(docname, doctree, title)
def finish(self):
# type: () -> None
self.finish_tasks.add_task(self.gen_indices)
self.finish_tasks.add_task(self.gen_additional_pages)
self.finish_tasks.add_task(self.copy_image_files)
@ -477,7 +500,8 @@ class StandaloneHTMLBuilder(Builder):
self.handle_finish()
def gen_indices(self):
self.info(bold('generating indices...'), nonl=1)
# type: () -> None
logger.info(bold('generating indices...'), nonl=1)
# the global general index
if self.use_index:
@ -486,15 +510,16 @@ class StandaloneHTMLBuilder(Builder):
# the global domain-specific indices
self.write_domain_indices()
self.info()
logger.info('')
def gen_additional_pages(self):
# type: () -> None
# pages from extensions
for pagelist in self.app.emit('html-collect-pages'):
for pagename, context, template in pagelist:
self.handle_page(pagename, context, template)
self.info(bold('writing additional pages...'), nonl=1)
logger.info(bold('writing additional pages...'), nonl=1)
# additional pages from conf.py
for pagename, template in self.config.html_additional_pages.items():
@ -503,21 +528,22 @@ class StandaloneHTMLBuilder(Builder):
# the search page
if self.search:
self.info(' search', nonl=1)
logger.info(' search', nonl=1)
self.handle_page('search', {}, 'search.html')
# the opensearch xml file
if self.config.html_use_opensearch and self.search:
self.info(' opensearch', nonl=1)
logger.info(' opensearch', nonl=1)
fn = path.join(self.outdir, '_static', 'opensearch.xml')
self.handle_page('opensearch', {}, 'opensearch.xml', outfilename=fn)
self.info()
logger.info('')
def write_genindex(self):
# type: () -> None
# the total count of lines for each index letter, used to distribute
# the entries into two columns
genindex = self.env.create_index(self)
genindex = IndexEntries(self.env).create_index(self)
indexcounts = []
for _k, entries in genindex:
indexcounts.append(sum(1 + len(subitems)
@ -528,7 +554,7 @@ class StandaloneHTMLBuilder(Builder):
genindexcounts = indexcounts,
split_index = self.config.html_split_index,
)
self.info(' genindex', nonl=1)
logger.info(' genindex', nonl=1)
if self.config.html_split_index:
self.handle_page('genindex', genindexcontext,
@ -544,54 +570,58 @@ class StandaloneHTMLBuilder(Builder):
self.handle_page('genindex', genindexcontext, 'genindex.html')
def write_domain_indices(self):
# type: () -> None
for indexname, indexcls, content, collapse in self.domain_indices:
indexcontext = dict(
indextitle = indexcls.localname,
content = content,
collapse_index = collapse,
)
self.info(' ' + indexname, nonl=1)
logger.info(' ' + indexname, nonl=1)
self.handle_page(indexname, indexcontext, 'domainindex.html')
def copy_image_files(self):
# type: () -> None
# copy image files
if self.images:
ensuredir(path.join(self.outdir, self.imagedir))
for src in self.app.status_iterator(self.images, 'copying images... ',
brown, len(self.images)):
for src in status_iterator(self.images, 'copying images... ', "brown",
len(self.images), self.app.verbosity):
dest = self.images[src]
try:
copyfile(path.join(self.srcdir, src),
path.join(self.outdir, self.imagedir, dest))
except Exception as err:
self.warn('cannot copy image file %r: %s' %
(path.join(self.srcdir, src), err))
logger.warning('cannot copy image file %r: %s',
path.join(self.srcdir, src), err)
def copy_download_files(self):
# type: () -> None
def to_relpath(f):
# type: (unicode) -> unicode
return relative_path(self.srcdir, f)
# copy downloadable files
if self.env.dlfiles:
ensuredir(path.join(self.outdir, '_downloads'))
for src in self.app.status_iterator(self.env.dlfiles,
'copying downloadable files... ',
brown, len(self.env.dlfiles),
stringify_func=to_relpath):
for src in status_iterator(self.env.dlfiles, 'copying downloadable files... ',
"brown", len(self.env.dlfiles), self.app.verbosity,
stringify_func=to_relpath):
dest = self.env.dlfiles[src][1]
try:
copyfile(path.join(self.srcdir, src),
path.join(self.outdir, '_downloads', dest))
except Exception as err:
self.warn('cannot copy downloadable file %r: %s' %
(path.join(self.srcdir, src), err))
logger.warning('cannot copy downloadable file %r: %s',
path.join(self.srcdir, src), err)
def copy_static_files(self):
# type: () -> None
# copy static files
self.info(bold('copying static files... '), nonl=True)
logger.info(bold('copying static files... '), nonl=True)
ensuredir(path.join(self.outdir, '_static'))
# first, create pygments style file
with open(path.join(self.outdir, '_static', 'pygments.css'), 'w') as f:
f.write(self.highlighter.get_stylesheet())
f.write(self.highlighter.get_stylesheet()) # type: ignore
# then, copy translations JavaScript file
if self.config.language is not None:
jsfile = self._get_translations_js()
@ -622,7 +652,7 @@ class StandaloneHTMLBuilder(Builder):
for static_path in self.config.html_static_path:
entry = path.join(self.confdir, static_path)
if not path.exists(entry):
self.warn('html_static_path entry %r does not exist' % entry)
logger.warning('html_static_path entry %r does not exist', entry)
continue
copy_asset(entry, path.join(self.outdir, '_static'), excluded,
context=ctx, renderer=self.templates)
@ -631,7 +661,7 @@ class StandaloneHTMLBuilder(Builder):
logobase = path.basename(self.config.html_logo)
logotarget = path.join(self.outdir, '_static', logobase)
if not path.isfile(path.join(self.confdir, self.config.html_logo)):
self.warn('logo file %r does not exist' % self.config.html_logo)
logger.warning('logo file %r does not exist', self.config.html_logo)
elif not path.isfile(logotarget):
copyfile(path.join(self.confdir, self.config.html_logo),
logotarget)
@ -639,27 +669,29 @@ class StandaloneHTMLBuilder(Builder):
iconbase = path.basename(self.config.html_favicon)
icontarget = path.join(self.outdir, '_static', iconbase)
if not path.isfile(path.join(self.confdir, self.config.html_favicon)):
self.warn('favicon file %r does not exist' % self.config.html_favicon)
logger.warning('favicon file %r does not exist', self.config.html_favicon)
elif not path.isfile(icontarget):
copyfile(path.join(self.confdir, self.config.html_favicon),
icontarget)
self.info('done')
logger.info('done')
def copy_extra_files(self):
# type: () -> None
# copy html_extra_path files
self.info(bold('copying extra files... '), nonl=True)
logger.info(bold('copying extra files... '), nonl=True)
excluded = Matcher(self.config.exclude_patterns)
for extra_path in self.config.html_extra_path:
entry = path.join(self.confdir, extra_path)
if not path.exists(entry):
self.warn('html_extra_path entry %r does not exist' % entry)
logger.warning('html_extra_path entry %r does not exist', entry)
continue
copy_asset(entry, self.outdir, excluded)
self.info('done')
logger.info('done')
def write_buildinfo(self):
# type: () -> None
# write build info file
with open(path.join(self.outdir, '.buildinfo'), 'w') as fp:
fp.write('# Sphinx build info version 1\n'
@ -669,11 +701,13 @@ class StandaloneHTMLBuilder(Builder):
(self.config_hash, self.tags_hash))
def cleanup(self):
# type: () -> None
# clean up theme stuff
if self.theme:
self.theme.cleanup()
def post_process_images(self, doctree):
# type: (nodes.Node) -> None
"""Pick the best candidate for an image and link down-scaled images to
their high res version.
"""
@ -699,24 +733,26 @@ class StandaloneHTMLBuilder(Builder):
reference.append(node)
def load_indexer(self, docnames):
# type: (Iterable[unicode]) -> None
keep = set(self.env.all_docs) - set(docnames)
try:
searchindexfn = path.join(self.outdir, self.searchindex_filename)
if self.indexer_dumps_unicode:
f = codecs.open(searchindexfn, 'r', encoding='utf-8')
f = codecs.open(searchindexfn, 'r', encoding='utf-8') # type: ignore
else:
f = open(searchindexfn, 'rb')
f = open(searchindexfn, 'rb') # type: ignore
with f:
self.indexer.load(f, self.indexer_format)
self.indexer.load(f, self.indexer_format) # type: ignore
except (IOError, OSError, ValueError):
if keep:
self.warn('search index couldn\'t be loaded, but not all '
'documents will be built: the index will be '
'incomplete.')
logger.warning('search index couldn\'t be loaded, but not all '
'documents will be built: the index will be '
'incomplete.')
# delete all entries for files that will be rebuilt
self.indexer.prune(keep)
def index_page(self, pagename, doctree, title):
# type: (unicode, nodes.Node, unicode) -> None
# only index pages with title
if self.indexer is not None and title:
filename = self.env.doc2path(pagename, base=None)
@ -724,19 +760,23 @@ class StandaloneHTMLBuilder(Builder):
self.indexer.feed(pagename, filename, title, doctree)
except TypeError:
# fallback for old search-adapters
self.indexer.feed(pagename, title, doctree)
self.indexer.feed(pagename, title, doctree) # type: ignore
def _get_local_toctree(self, docname, collapse=True, **kwds):
# type: (unicode, bool, Any) -> unicode
if 'includehidden' not in kwds:
kwds['includehidden'] = False
return self.render_partial(self.env.get_toctree_for(
return self.render_partial(TocTree(self.env).get_toctree_for(
docname, self, collapse, **kwds))['fragment']
def get_outfilename(self, pagename):
# type: (unicode) -> unicode
return path.join(self.outdir, os_path(pagename) + self.out_suffix)
def add_sidebars(self, pagename, ctx):
# type: (unicode, Dict) -> None
def has_wildcard(pattern):
# type: (unicode) -> bool
return any(char in pattern for char in '*?[')
sidebars = None
matched = None
@ -747,9 +787,9 @@ class StandaloneHTMLBuilder(Builder):
if has_wildcard(pattern):
# warn if both patterns contain wildcards
if has_wildcard(matched):
self.warn('page %s matches two patterns in '
'html_sidebars: %r and %r' %
(pagename, matched, pattern))
logger.warning('page %s matches two patterns in '
'html_sidebars: %r and %r',
pagename, matched, pattern)
# else the already matched pattern is more specific
# than the present one, because it contains no wildcard
continue
@ -768,20 +808,24 @@ class StandaloneHTMLBuilder(Builder):
# --------- these are overwritten by the serialization builder
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
return docname + self.link_suffix
def handle_page(self, pagename, addctx, templatename='page.html',
outfilename=None, event_arg=None):
# type: (unicode, Dict, unicode, unicode, Any) -> None
ctx = self.globalcontext.copy()
ctx['warn'] = self.warn
# current_page_name is backwards compatibility
ctx['pagename'] = ctx['current_page_name'] = pagename
ctx['encoding'] = self.config.html_output_encoding
default_baseuri = self.get_target_uri(pagename)
# in the singlehtml builder, default_baseuri still contains an #anchor
# part, which relative_uri doesn't really like...
default_baseuri = default_baseuri.rsplit('#', 1)[0]
def pathto(otheruri, resource=False, baseuri=default_baseuri):
# type: (unicode, bool, unicode) -> unicode
if resource and '://' in otheruri:
# allow non-local resources given by scheme
return otheruri
@ -794,6 +838,7 @@ class StandaloneHTMLBuilder(Builder):
ctx['pathto'] = pathto
def hasdoc(name):
# type: (unicode) -> bool
if name in self.env.all_docs:
return True
elif name == 'search' and self.search:
@ -803,14 +848,11 @@ class StandaloneHTMLBuilder(Builder):
return False
ctx['hasdoc'] = hasdoc
if self.name != 'htmlhelp':
ctx['encoding'] = encoding = self.config.html_output_encoding
else:
ctx['encoding'] = encoding = self.encoding
ctx['toctree'] = lambda **kw: self._get_local_toctree(pagename, **kw)
self.add_sidebars(pagename, ctx)
ctx.update(addctx)
self.update_page_context(pagename, templatename, ctx, event_arg)
newtmpl = self.app.emit_firstresult('html-page-context', pagename,
templatename, ctx, event_arg)
if newtmpl:
@ -819,9 +861,9 @@ class StandaloneHTMLBuilder(Builder):
try:
output = self.templates.render(templatename, ctx)
except UnicodeError:
self.warn("a Unicode error occurred when rendering the page %s. "
"Please make sure all config values that contain "
"non-ASCII content are Unicode strings." % pagename)
logger.warning("a Unicode error occurred when rendering the page %s. "
"Please make sure all config values that contain "
"non-ASCII content are Unicode strings.", pagename)
return
if not outfilename:
@ -829,10 +871,10 @@ class StandaloneHTMLBuilder(Builder):
# outfilename's path is in general different from self.outdir
ensuredir(path.dirname(outfilename))
try:
with codecs.open(outfilename, 'w', encoding, 'xmlcharrefreplace') as f:
with codecs.open(outfilename, 'w', ctx['encoding'], 'xmlcharrefreplace') as f: # type: ignore # NOQA
f.write(output)
except (IOError, OSError) as err:
self.warn("error writing file %s: %s" % (outfilename, err))
logger.warning("error writing file %s: %s", outfilename, err)
if self.copysource and ctx.get('sourcename'):
# copy the source file for the "show source" link
source_name = path.join(self.outdir, '_sources',
@ -840,13 +882,19 @@ class StandaloneHTMLBuilder(Builder):
ensuredir(path.dirname(source_name))
copyfile(self.env.doc2path(pagename), source_name)
def update_page_context(self, pagename, templatename, ctx, event_arg):
# type: (unicode, unicode, Dict, Any) -> None
pass
def handle_finish(self):
# type: () -> None
if self.indexer:
self.finish_tasks.add_task(self.dump_search_index)
self.finish_tasks.add_task(self.dump_inventory)
def dump_inventory(self):
self.info(bold('dumping object inventory... '), nonl=True)
# type: () -> None
logger.info(bold('dumping object inventory... '), nonl=True)
with open(path.join(self.outdir, INVENTORY_FILENAME), 'wb') as f:
f.write((u'# Sphinx inventory version 2\n'
u'# Project: %s\n'
@ -869,10 +917,11 @@ class StandaloneHTMLBuilder(Builder):
(u'%s %s:%s %s %s %s\n' % (name, domainname, type,
prio, uri, dispname)).encode('utf-8')))
f.write(compressor.flush())
self.info('done')
logger.info('done')
def dump_search_index(self):
self.info(
# type: () -> None
logger.info(
bold('dumping search index in %s ... ' % self.indexer.label()),
nonl=True)
self.indexer.prune(self.env.all_docs)
@ -880,13 +929,13 @@ class StandaloneHTMLBuilder(Builder):
# first write to a temporary file, so that if dumping fails,
# the existing index won't be overwritten
if self.indexer_dumps_unicode:
f = codecs.open(searchindexfn + '.tmp', 'w', encoding='utf-8')
f = codecs.open(searchindexfn + '.tmp', 'w', encoding='utf-8') # type: ignore
else:
f = open(searchindexfn + '.tmp', 'wb')
f = open(searchindexfn + '.tmp', 'wb') # type: ignore
with f:
self.indexer.dump(f, self.indexer_format)
self.indexer.dump(f, self.indexer_format) # type: ignore
movefile(searchindexfn + '.tmp', searchindexfn)
self.info('done')
logger.info('done')
class DirectoryHTMLBuilder(StandaloneHTMLBuilder):
@ -898,6 +947,7 @@ class DirectoryHTMLBuilder(StandaloneHTMLBuilder):
name = 'dirhtml'
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
if docname == 'index':
return ''
if docname.endswith(SEP + 'index'):
@ -905,6 +955,7 @@ class DirectoryHTMLBuilder(StandaloneHTMLBuilder):
return docname + SEP
def get_outfilename(self, pagename):
# type: (unicode) -> unicode
if pagename == 'index' or pagename.endswith(SEP + 'index'):
outfilename = path.join(self.outdir, os_path(pagename) +
self.out_suffix)
@ -915,6 +966,7 @@ class DirectoryHTMLBuilder(StandaloneHTMLBuilder):
return outfilename
def prepare_writing(self, docnames):
# type: (Iterable[unicode]) -> None
StandaloneHTMLBuilder.prepare_writing(self, docnames)
self.globalcontext['no_search_suffix'] = True
@ -927,10 +979,12 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
name = 'singlehtml'
copysource = False
def get_outdated_docs(self):
def get_outdated_docs(self): # type: ignore
# type: () -> Union[unicode, List[unicode]]
return 'all documents'
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
if docname in self.env.all_docs:
# all references are on the same page...
return self.config.master_doc + self.out_suffix + \
@ -940,10 +994,12 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
return docname + self.out_suffix
def get_relative_uri(self, from_, to, typ=None):
# type: (unicode, unicode, unicode) -> unicode
# ignore source
return self.get_target_uri(to, typ)
def fix_refuris(self, tree):
# type: (nodes.Node) -> None
# fix refuris with double anchor
fname = self.config.master_doc + self.out_suffix
for refnode in tree.traverse(nodes.reference):
@ -958,13 +1014,15 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
refnode['refuri'] = fname + refuri[hashindex:]
def _get_local_toctree(self, docname, collapse=True, **kwds):
# type: (unicode, bool, Any) -> unicode
if 'includehidden' not in kwds:
kwds['includehidden'] = False
toctree = self.env.get_toctree_for(docname, self, collapse, **kwds)
toctree = TocTree(self.env).get_toctree_for(docname, self, collapse, **kwds)
self.fix_refuris(toctree)
return self.render_partial(toctree)['fragment']
def assemble_doctree(self):
# type: () -> nodes.Node
master = self.config.master_doc
tree = self.env.get_doctree(master)
tree = inline_all_toctrees(self, set(), master, tree, darkgreen, [master])
@ -974,6 +1032,7 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
return tree
def assemble_toc_secnumbers(self):
# type: () -> Dict[unicode, Dict[Tuple[unicode, unicode], Tuple[int, ...]]]
# Assemble toc_secnumbers to resolve section numbers on SingleHTML.
# Merge all secnumbers to single secnumber.
#
@ -991,6 +1050,7 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
return {self.config.master_doc: new_secnumbers}
def assemble_toc_fignumbers(self):
# type: () -> Dict[unicode, Dict[Tuple[unicode, unicode], Dict[unicode, Tuple[int, ...]]]] # NOQA
# Assemble toc_fignumbers to resolve figure numbers on SingleHTML.
# Merge all fignumbers to single fignumber.
#
@ -1000,7 +1060,7 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
#
# There are related codes in inline_all_toctres() and
# HTMLTranslter#add_fignumber().
new_fignumbers = {}
new_fignumbers = {} # type: Dict[Tuple[unicode, unicode], Dict[unicode, Tuple[int, ...]]] # NOQA
# {u'foo': {'figure': {'id2': (2,), 'id1': (1,)}}, u'bar': {'figure': {'id1': (3,)}}}
for docname, fignumlist in iteritems(self.env.toc_fignumbers):
for figtype, fignums in iteritems(fignumlist):
@ -1011,8 +1071,10 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
return {self.config.master_doc: new_fignumbers}
def get_doc_context(self, docname, body, metatags):
# type: (unicode, unicode, Dict) -> Dict
# no relation links...
toc = self.env.get_toctree_for(self.config.master_doc, self, False)
toc = TocTree(self.env).get_toctree_for(self.config.master_doc,
self, False)
# if there is no toctree, toc is None
if toc:
self.fix_refuris(toc)
@ -1037,25 +1099,27 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
)
def write(self, *ignored):
# type: (Any) -> None
docnames = self.env.all_docs
self.info(bold('preparing documents... '), nonl=True)
logger.info(bold('preparing documents... '), nonl=True)
self.prepare_writing(docnames)
self.info('done')
logger.info('done')
self.info(bold('assembling single document... '), nonl=True)
logger.info(bold('assembling single document... '), nonl=True)
doctree = self.assemble_doctree()
self.env.toc_secnumbers = self.assemble_toc_secnumbers()
self.env.toc_fignumbers = self.assemble_toc_fignumbers()
self.info()
self.info(bold('writing... '), nonl=True)
logger.info('')
logger.info(bold('writing... '), nonl=True)
self.write_doc_serialized(self.config.master_doc, doctree)
self.write_doc(self.config.master_doc, doctree)
self.info('done')
logger.info('done')
def finish(self):
# type: () -> None
# no indices or search pages are supported
self.info(bold('writing additional files...'), nonl=1)
logger.info(bold('writing additional files...'), nonl=1)
# additional pages from conf.py
for pagename, template in self.config.html_additional_pages.items():
@ -1063,11 +1127,11 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder):
self.handle_page(pagename, {}, template)
if self.config.html_use_opensearch:
self.info(' opensearch', nonl=1)
logger.info(' opensearch', nonl=1)
fn = path.join(self.outdir, '_static', 'opensearch.xml')
self.handle_page('opensearch', {}, 'opensearch.xml', outfilename=fn)
self.info()
logger.info('')
self.copy_image_files()
self.copy_download_files()
@ -1084,18 +1148,19 @@ class SerializingHTMLBuilder(StandaloneHTMLBuilder):
#: the serializing implementation to use. Set this to a module that
#: implements a `dump`, `load`, `dumps` and `loads` functions
#: (pickle, simplejson etc.)
implementation = None
implementation = None # type: Any
implementation_dumps_unicode = False
#: additional arguments for dump()
additional_dump_args = ()
#: the filename for the global context file
globalcontext_filename = None
globalcontext_filename = None # type: unicode
supported_image_types = ['image/svg+xml', 'image/png',
'image/gif', 'image/jpeg']
def init(self):
# type: () -> None
self.config_hash = ''
self.tags_hash = ''
self.imagedir = '_images'
@ -1108,6 +1173,7 @@ class SerializingHTMLBuilder(StandaloneHTMLBuilder):
self.use_index = self.get_builder_config('use_index', 'html')
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
if docname == 'index':
return ''
if docname.endswith(SEP + 'index'):
@ -1115,15 +1181,17 @@ class SerializingHTMLBuilder(StandaloneHTMLBuilder):
return docname + SEP
def dump_context(self, context, filename):
# type: (Dict, unicode) -> None
if self.implementation_dumps_unicode:
f = codecs.open(filename, 'w', encoding='utf-8')
f = codecs.open(filename, 'w', encoding='utf-8') # type: ignore
else:
f = open(filename, 'wb')
f = open(filename, 'wb') # type: ignore
with f:
self.implementation.dump(context, f, *self.additional_dump_args)
def handle_page(self, pagename, ctx, templatename='page.html',
outfilename=None, event_arg=None):
# type: (unicode, Dict, unicode, unicode, Any) -> None
ctx['current_page_name'] = pagename
self.add_sidebars(pagename, ctx)
@ -1147,6 +1215,7 @@ class SerializingHTMLBuilder(StandaloneHTMLBuilder):
copyfile(self.env.doc2path(pagename), source_name)
def handle_finish(self):
# type: () -> None
# dump the global context
outfilename = path.join(self.outdir, self.globalcontext_filename)
self.dump_context(self.globalcontext, outfilename)
@ -1197,16 +1266,12 @@ class JSONHTMLBuilder(SerializingHTMLBuilder):
searchindex_filename = 'searchindex.json'
def init(self):
# type: () -> None
SerializingHTMLBuilder.init(self)
def validate_config_values(app):
if app.config.html_translator_class:
app.warn('html_translator_class is deprecated. '
'Use Sphinx.set_translator() API instead.')
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
# builders
app.add_builder(StandaloneHTMLBuilder)
app.add_builder(DirectoryHTMLBuilder)
@ -1214,8 +1279,6 @@ def setup(app):
app.add_builder(PickleHTMLBuilder)
app.add_builder(JSONHTMLBuilder)
app.connect('builder-inited', validate_config_values)
# config values
app.add_config_value('html_theme', 'alabaster', 'html')
app.add_config_value('html_theme_path', [], 'html')
@ -1233,7 +1296,6 @@ def setup(app):
app.add_config_value('html_use_smartypants', True, 'html')
app.add_config_value('html_sidebars', {}, 'html')
app.add_config_value('html_additional_pages', {}, 'html')
app.add_config_value('html_use_modindex', True, 'html') # deprecated
app.add_config_value('html_domain_indices', True, 'html', [list])
app.add_config_value('html_add_permalinks', u'\u00B6', 'html')
app.add_config_value('html_use_index', True, 'html')

38
sphinx/builders/htmlhelp.py
View File

@ -18,10 +18,20 @@ from os import path
from docutils import nodes
from sphinx import addnodes
from sphinx.util.osutil import make_filename
from sphinx.builders.html import StandaloneHTMLBuilder
from sphinx.environment.adapters.indexentries import IndexEntries
from sphinx.util import logging
from sphinx.util.osutil import make_filename
from sphinx.util.pycompat import htmlescape
if False:
# For type annotation
from typing import Any, IO, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
# Project file (*.hhp) template. 'outname' is the file basename (like
# the pythlp in pythlp.hhp); 'version' is the doc version number (like
@ -181,6 +191,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
encoding = 'cp1252'
def init(self):
# type: () -> None
StandaloneHTMLBuilder.init(self)
# the output files for HTML help must be .html only
self.out_suffix = '.html'
@ -191,14 +202,21 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
self.lcid, self.encoding = locale
def open_file(self, outdir, basename, mode='w'):
# type: (unicode, unicode, unicode) -> IO
# open a file with the correct encoding for the selected language
return codecs.open(path.join(outdir, basename), mode,
return codecs.open(path.join(outdir, basename), mode, # type: ignore
self.encoding, 'xmlcharrefreplace')
def update_page_context(self, pagename, templatename, ctx, event_arg):
# type: (unicode, unicode, Dict, unicode) -> None
ctx['encoding'] = self.encoding
def handle_finish(self):
# type: () -> None
self.build_hhx(self.outdir, self.config.htmlhelp_basename)
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
for node in doctree.traverse(nodes.reference):
# add ``target=_blank`` attributes to external links
if node.get('internal') is None and 'refuri' in node:
@ -207,12 +225,13 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
StandaloneHTMLBuilder.write_doc(self, docname, doctree)
def build_hhx(self, outdir, outname):
self.info('dumping stopword list...')
# type: (unicode, unicode) -> None
logger.info('dumping stopword list...')
with self.open_file(outdir, outname + '.stp') as f:
for word in sorted(stopwords):
print(word, file=f)
self.info('writing project file...')
logger.info('writing project file...')
with self.open_file(outdir, outname + '.hhp') as f:
f.write(project_template % {
'outname': outname,
@ -233,7 +252,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
print(path.join(root, fn)[olen:].replace(os.sep, '\\'),
file=f)
self.info('writing TOC file...')
logger.info('writing TOC file...')
with self.open_file(outdir, outname + '.hhc') as f:
f.write(contents_header)
# special books
@ -247,6 +266,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
self.config.master_doc, self, prune_toctrees=False)
def write_toc(node, ullevel=0):
# type: (nodes.Node, int) -> None
if isinstance(node, nodes.list_item):
f.write('<LI> ')
for subnode in node:
@ -267,19 +287,22 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
write_toc(subnode, ullevel)
def istoctree(node):
# type: (nodes.Node) -> bool
return isinstance(node, addnodes.compact_paragraph) and \
'toctree' in node
for node in tocdoc.traverse(istoctree):
write_toc(node)
f.write(contents_footer)
self.info('writing index file...')
index = self.env.create_index(self)
logger.info('writing index file...')
index = IndexEntries(self.env).create_index(self)
with self.open_file(outdir, outname + '.hhk') as f:
f.write('<UL>\n')
def write_index(title, refs, subitems):
# type: (unicode, List[Tuple[unicode, unicode]], List[Tuple[unicode, List[Tuple[unicode, unicode]]]]) -> None # NOQA
def write_param(name, value):
# type: (unicode, unicode) -> None
item = ' <param name="%s" value="%s">\n' % \
(name, value)
f.write(item)
@ -308,6 +331,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.setup_extension('sphinx.builders.html')
app.add_builder(HTMLHelpBuilder)

135
sphinx/builders/latex.py
View File

@ -10,16 +10,19 @@
"""
import os
import warnings
from os import path
from six import iteritems
from docutils import nodes
from docutils.io import FileOutput
from docutils.utils import new_document
from docutils.frontend import OptionParser
from sphinx import package_dir, addnodes, highlighting
from sphinx.util import texescape
from sphinx.deprecation import RemovedInSphinx17Warning
from sphinx.util import texescape, logging
from sphinx.config import string_classes, ENUM
from sphinx.errors import SphinxError
from sphinx.locale import _
@ -28,9 +31,18 @@ from sphinx.environment import NoUri
from sphinx.util.nodes import inline_all_toctrees
from sphinx.util.fileutil import copy_asset_file
from sphinx.util.osutil import SEP, make_filename
from sphinx.util.console import bold, darkgreen
from sphinx.util.console import bold, darkgreen # type: ignore
from sphinx.writers.latex import LaTeXWriter
if False:
# For type annotation
from typing import Any, Iterable, Tuple, Union # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.config import Config # NOQA
logger = logging.getLogger(__name__)
class LaTeXBuilder(Builder):
"""
@ -41,44 +53,50 @@ class LaTeXBuilder(Builder):
supported_image_types = ['application/pdf', 'image/png', 'image/jpeg']
def init(self):
self.docnames = []
self.document_data = []
self.usepackages = []
# type: () -> None
self.docnames = [] # type: Iterable[unicode]
self.document_data = [] # type: List[Tuple[unicode, unicode, unicode, unicode, unicode, bool]] # NOQA
self.usepackages = [] # type: List[unicode]
texescape.init()
def get_outdated_docs(self):
# type: () -> Union[unicode, List[unicode]]
return 'all documents' # for now
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
if docname not in self.docnames:
raise NoUri
else:
return '%' + docname
def get_relative_uri(self, from_, to, typ=None):
# type: (unicode, unicode, unicode) -> unicode
# ignore source path
return self.get_target_uri(to, typ)
def init_document_data(self):
# type: () -> None
preliminary_document_data = [list(x) for x in self.config.latex_documents]
if not preliminary_document_data:
self.warn('no "latex_documents" config value found; no documents '
'will be written')
logger.warning('no "latex_documents" config value found; no documents '
'will be written')
return
# assign subdirs to titles
self.titles = []
self.titles = [] # type: List[Tuple[unicode, unicode]]
for entry in preliminary_document_data:
docname = entry[0]
if docname not in self.env.all_docs:
self.warn('"latex_documents" config value references unknown '
'document %s' % docname)
logger.warning('"latex_documents" config value references unknown '
'document %s', docname)
continue
self.document_data.append(entry)
self.document_data.append(entry) # type: ignore
if docname.endswith(SEP + 'index'):
docname = docname[:-5]
self.titles.append((docname, entry[2]))
def write_stylesheet(self):
# type: () -> None
highlighter = highlighting.PygmentsBridge(
'latex', self.config.pygments_style, self.config.trim_doctest_flags)
stylesheet = path.join(self.outdir, 'sphinxhighlight.sty')
@ -86,9 +104,10 @@ class LaTeXBuilder(Builder):
f.write('\\NeedsTeXFormat{LaTeX2e}[1995/12/01]\n')
f.write('\\ProvidesPackage{sphinxhighlight}'
'[2016/05/29 stylesheet for highlighting with pygments]\n\n')
f.write(highlighter.get_stylesheet())
f.write(highlighter.get_stylesheet()) # type: ignore
def write(self, *ignored):
# type: (Any) -> None
docwriter = LaTeXWriter(self)
docsettings = OptionParser(
defaults=self.env.settings,
@ -106,7 +125,7 @@ class LaTeXBuilder(Builder):
destination = FileOutput(
destination_path=path.join(self.outdir, targetname),
encoding='utf-8')
self.info("processing " + targetname + "... ", nonl=1)
logger.info("processing %s...", targetname, nonl=1)
toctrees = self.env.get_doctree(docname).traverse(addnodes.toctree)
if toctrees:
if toctrees[0].get('maxdepth') > 0:
@ -120,7 +139,7 @@ class LaTeXBuilder(Builder):
appendices=((docclass != 'howto') and self.config.latex_appendices or []))
doctree['tocdepth'] = tocdepth
self.post_process_images(doctree)
self.info("writing... ", nonl=1)
logger.info("writing... ", nonl=1)
doctree.settings = docsettings
doctree.settings.author = author
doctree.settings.title = title
@ -128,9 +147,10 @@ class LaTeXBuilder(Builder):
doctree.settings.docname = docname
doctree.settings.docclass = docclass
docwriter.write(doctree, destination)
self.info("done")
logger.info("done")
def get_contentsname(self, indexfile):
# type: (unicode) -> unicode
tree = self.env.get_doctree(indexfile)
contentsname = None
for toctree in tree.traverse(addnodes.toctree):
@ -141,8 +161,9 @@ class LaTeXBuilder(Builder):
return contentsname
def assemble_doctree(self, indexfile, toctree_only, appendices):
# type: (unicode, bool, List[unicode]) -> nodes.Node
self.docnames = set([indexfile] + appendices)
self.info(darkgreen(indexfile) + " ", nonl=1)
logger.info(darkgreen(indexfile) + " ", nonl=1)
tree = self.env.get_doctree(indexfile)
tree['docname'] = indexfile
if toctree_only:
@ -163,8 +184,8 @@ class LaTeXBuilder(Builder):
appendix = self.env.get_doctree(docname)
appendix['docname'] = docname
largetree.append(appendix)
self.info()
self.info("resolving references...")
logger.info('')
logger.info("resolving references...")
self.env.resolve_references(largetree, indexfile, self)
# resolve :ref:s to distant tex files -- we can't add a cross-reference,
# but append the document name
@ -184,18 +205,19 @@ class LaTeXBuilder(Builder):
return largetree
def finish(self):
# type: () -> None
# copy image files
if self.images:
self.info(bold('copying images...'), nonl=1)
logger.info(bold('copying images...'), nonl=1)
for src, dest in iteritems(self.images):
self.info(' ' + src, nonl=1)
logger.info(' ' + src, nonl=1)
copy_asset_file(path.join(self.srcdir, src),
path.join(self.outdir, dest))
self.info()
logger.info('')
# copy TeX support files from texinputs
context = {'latex_engine': self.config.latex_engine}
self.info(bold('copying TeX support files...'))
logger.info(bold('copying TeX support files...'))
staticdirname = path.join(package_dir, 'texinputs')
for filename in os.listdir(staticdirname):
if not filename.startswith('.'):
@ -204,11 +226,11 @@ class LaTeXBuilder(Builder):
# copy additional files
if self.config.latex_additional_files:
self.info(bold('copying additional files...'), nonl=1)
logger.info(bold('copying additional files...'), nonl=1)
for filename in self.config.latex_additional_files:
self.info(' ' + filename, nonl=1)
logger.info(' ' + filename, nonl=1)
copy_asset_file(path.join(self.confdir, filename), self.outdir)
self.info()
logger.info('')
# the logo is handled differently
if self.config.latex_logo:
@ -216,59 +238,29 @@ class LaTeXBuilder(Builder):
raise SphinxError('logo file %r does not exist' % self.config.latex_logo)
else:
copy_asset_file(path.join(self.confdir, self.config.latex_logo), self.outdir)
self.info('done')
logger.info('done')
def validate_config_values(app):
# type: (Sphinx) -> None
if app.config.latex_toplevel_sectioning not in (None, 'part', 'chapter', 'section'):
app.warn('invalid latex_toplevel_sectioning, ignored: %s' %
app.config.latex_toplevel_sectioning)
app.config.latex_toplevel_sectioning = None
if app.config.latex_use_parts:
if app.config.latex_toplevel_sectioning:
app.warn('latex_use_parts conflicts with latex_toplevel_sectioning, ignored.')
else:
app.warn('latex_use_parts is deprecated. Use latex_toplevel_sectioning instead.')
app.config.latex_toplevel_sectioning = 'part'
if app.config.latex_use_modindex is not True: # changed by user
app.warn('latex_use_modindex is deprecated. Use latex_domain_indices instead.')
if app.config.latex_preamble:
if app.config.latex_elements.get('preamble'):
app.warn("latex_preamble conflicts with latex_elements['preamble'], ignored.")
else:
app.warn("latex_preamble is deprecated. Use latex_elements['preamble'] instead.")
app.config.latex_elements['preamble'] = app.config.latex_preamble
if app.config.latex_paper_size != 'letter':
if app.config.latex_elements.get('papersize'):
app.warn("latex_paper_size conflicts with latex_elements['papersize'], ignored.")
else:
app.warn("latex_paper_size is deprecated. "
"Use latex_elements['papersize'] instead.")
if app.config.latex_paper_size:
app.config.latex_elements['papersize'] = app.config.latex_paper_size + 'paper'
if app.config.latex_font_size != '10pt':
if app.config.latex_elements.get('pointsize'):
app.warn("latex_font_size conflicts with latex_elements['pointsize'], ignored.")
else:
app.warn("latex_font_size is deprecated. Use latex_elements['pointsize'] instead.")
app.config.latex_elements['pointsize'] = app.config.latex_font_size
logger.warning('invalid latex_toplevel_sectioning, ignored: %s',
app.config.latex_toplevel_sectioning)
app.config.latex_toplevel_sectioning = None # type: ignore
if 'footer' in app.config.latex_elements:
if 'postamble' in app.config.latex_elements:
app.warn("latex_elements['footer'] conflicts with "
"latex_elements['postamble'], ignored.")
logger.warning("latex_elements['footer'] conflicts with "
"latex_elements['postamble'], ignored.")
else:
app.warn("latex_elements['footer'] is deprecated. "
"Use latex_elements['preamble'] instead.")
warnings.warn("latex_elements['footer'] is deprecated. "
"Use latex_elements['preamble'] instead.",
RemovedInSphinx17Warning)
app.config.latex_elements['postamble'] = app.config.latex_elements['footer']
def default_latex_engine(config):
# type: (Config) -> unicode
""" Better default latex_engine settings for specific languages. """
if config.language == 'ja':
return 'platex'
@ -277,6 +269,7 @@ def default_latex_engine(config):
def default_latex_docclass(config):
# type: (Config) -> Dict[unicode, unicode]
""" Better default latex_docclass settings for specific languages. """
if config.language == 'ja':
return {'manual': 'jsbook',
@ -286,6 +279,7 @@ def default_latex_docclass(config):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(LaTeXBuilder)
app.connect('builder-inited', validate_config_values)
@ -298,23 +292,14 @@ def setup(app):
app.add_config_value('latex_logo', None, None, string_classes)
app.add_config_value('latex_appendices', [], None)
app.add_config_value('latex_keep_old_macro_names', True, None)
# now deprecated - use latex_toplevel_sectioning
app.add_config_value('latex_use_parts', False, None)
app.add_config_value('latex_toplevel_sectioning', None, None, [str])
app.add_config_value('latex_use_modindex', True, None) # deprecated
app.add_config_value('latex_domain_indices', True, None, [list])
app.add_config_value('latex_show_urls', 'no', None)
app.add_config_value('latex_show_pagerefs', False, None)
# paper_size and font_size are still separate values
# so that you can give them easily on the command line
app.add_config_value('latex_paper_size', 'letter', None)
app.add_config_value('latex_font_size', '10pt', None)
app.add_config_value('latex_elements', {}, None)
app.add_config_value('latex_additional_files', [], None)
app.add_config_value('latex_docclass', default_latex_docclass, None)
# now deprecated - use latex_elements
app.add_config_value('latex_preamble', '', None)
return {
'version': 'builtin',

71
sphinx/builders/linkcheck.py
View File

@ -16,7 +16,7 @@ import threading
from os import path
from requests.exceptions import HTTPError
from six.moves import queue, html_parser
from six.moves import queue, html_parser # type: ignore
from six.moves.urllib.parse import unquote
from docutils import nodes
@ -25,22 +25,33 @@ from docutils import nodes
# going to just remove it. If it doesn't exist, define an exception that will
# never be caught but leaves the code in check_anchor() intact.
try:
from six.moves.html_parser import HTMLParseError
from six.moves.html_parser import HTMLParseError # type: ignore
except ImportError:
class HTMLParseError(Exception):
class HTMLParseError(Exception): # type: ignore
pass
from sphinx.builders import Builder
from sphinx.util import encode_uri, requests
from sphinx.util.console import purple, red, darkgreen, darkgray, \
darkred, turquoise
from sphinx.util import encode_uri, requests, logging
from sphinx.util.console import ( # type: ignore
purple, red, darkgreen, darkgray, darkred, turquoise
)
from sphinx.util.requests import is_ssl_error
if False:
# For type annotation
from typing import Any, Tuple, Union # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.util.requests.requests import Response # NOQA
logger = logging.getLogger(__name__)
class AnchorCheckParser(html_parser.HTMLParser):
"""Specialized HTML parser that looks for a specific anchor."""
def __init__(self, search_anchor):
# type: (unicode) -> None
html_parser.HTMLParser.__init__(self)
self.search_anchor = search_anchor
@ -54,6 +65,7 @@ class AnchorCheckParser(html_parser.HTMLParser):
def check_anchor(response, anchor):
# type: (Response, unicode) -> bool
"""Reads HTML data from a response object `response` searching for `anchor`.
Returns True if anchor was found, False otherwise.
"""
@ -80,12 +92,13 @@ class CheckExternalLinksBuilder(Builder):
name = 'linkcheck'
def init(self):
# type: () -> None
self.to_ignore = [re.compile(x) for x in self.app.config.linkcheck_ignore]
self.anchors_ignore = [re.compile(x)
for x in self.app.config.linkcheck_anchors_ignore]
self.good = set()
self.broken = {}
self.redirected = {}
self.good = set() # type: Set[unicode]
self.broken = {} # type: Dict[unicode, unicode]
self.redirected = {} # type: Dict[unicode, Tuple[unicode, int]]
# set a timeout for non-responding servers
socket.setdefaulttimeout(5.0)
# create output file
@ -94,7 +107,7 @@ class CheckExternalLinksBuilder(Builder):
# create queues and worker threads
self.wqueue = queue.Queue()
self.rqueue = queue.Queue()
self.workers = []
self.workers = [] # type: List[threading.Thread]
for i in range(self.app.config.linkcheck_workers):
thread = threading.Thread(target=self.check_thread)
thread.setDaemon(True)
@ -102,6 +115,7 @@ class CheckExternalLinksBuilder(Builder):
self.workers.append(thread)
def check_thread(self):
# type: () -> None
kwargs = {}
if self.app.config.linkcheck_timeout:
kwargs['timeout'] = self.app.config.linkcheck_timeout
@ -109,6 +123,7 @@ class CheckExternalLinksBuilder(Builder):
kwargs['allow_redirects'] = True
def check_uri():
# type: () -> Tuple[unicode, unicode, int]
# split off anchor
if '#' in uri:
req_url, anchor = uri.split('#', 1)
@ -170,6 +185,7 @@ class CheckExternalLinksBuilder(Builder):
return 'redirected', new_url, code
def check():
# type: () -> Tuple[unicode, unicode, int]
# check for various conditions without bothering the network
if len(uri) == 0 or uri.startswith(('#', 'mailto:', 'ftp:')):
return 'unchecked', '', 0
@ -208,30 +224,31 @@ class CheckExternalLinksBuilder(Builder):
self.rqueue.put((uri, docname, lineno, status, info, code))
def process_result(self, result):
# type: (Tuple[unicode, unicode, int, unicode, unicode, int]) -> None
uri, docname, lineno, status, info, code = result
if status == 'unchecked':
return
if status == 'working' and info == 'old':
return
if lineno:
self.info('(line %4d) ' % lineno, nonl=1)
logger.info('(line %4d) ', lineno, nonl=1)
if status == 'ignored':
if info:
self.info(darkgray('-ignored- ') + uri + ': ' + info)
logger.info(darkgray('-ignored- ') + uri + ': ' + info)
else:
self.info(darkgray('-ignored- ') + uri)
logger.info(darkgray('-ignored- ') + uri)
elif status == 'local':
self.info(darkgray('-local- ') + uri)
logger.info(darkgray('-local- ') + uri)
self.write_entry('local', docname, lineno, uri)
elif status == 'working':
self.info(darkgreen('ok ') + uri + info)
logger.info(darkgreen('ok ') + uri + info)
elif status == 'broken':
self.write_entry('broken', docname, lineno, uri + ': ' + info)
if self.app.quiet or self.app.warningiserror:
self.warn('broken link: %s (%s)' % (uri, info),
'%s:%s' % (self.env.doc2path(docname), lineno))
logger.warning('broken link: %s (%s)', uri, info,
location=(self.env.doc2path(docname), lineno))
else:
self.info(red('broken ') + uri + red(' - ' + info))
logger.info(red('broken ') + uri + red(' - ' + info))
elif status == 'redirected':
text, color = {
301: ('permanently', darkred),
@ -242,19 +259,23 @@ class CheckExternalLinksBuilder(Builder):
}[code]
self.write_entry('redirected ' + text, docname, lineno,
uri + ' to ' + info)
self.info(color('redirect ') + uri + color(' - ' + text + ' to ' + info))
logger.info(color('redirect ') + uri + color(' - ' + text + ' to ' + info))
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
return ''
def get_outdated_docs(self):
# type: () -> Set[unicode]
return self.env.found_docs
def prepare_writing(self, docnames):
# type: (nodes.Node) -> None
return
def write_doc(self, docname, doctree):
self.info()
# type: (unicode, nodes.Node) -> None
logger.info('')
n = 0
for node in doctree.traverse(nodes.reference):
if 'refuri' not in node:
@ -277,17 +298,19 @@ class CheckExternalLinksBuilder(Builder):
self.app.statuscode = 1
def write_entry(self, what, docname, line, uri):
output = codecs.open(path.join(self.outdir, 'output.txt'), 'a', 'utf-8')
output.write("%s:%s: [%s] %s\n" % (self.env.doc2path(docname, None),
line, what, uri))
output.close()
# type: (unicode, unicode, int, unicode) -> None
with codecs.open(path.join(self.outdir, 'output.txt'), 'a', 'utf-8') as output: # type: ignore # NOQA
output.write("%s:%s: [%s] %s\n" % (self.env.doc2path(docname, None),
line, what, uri))
def finish(self):
# type: () -> None
for worker in self.workers:
self.wqueue.put((None, None, None), False)
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(CheckExternalLinksBuilder)
app.add_config_value('linkcheck_ignore', [], None)

34
sphinx/builders/manpage.py
View File

@ -12,17 +12,27 @@
from os import path
from six import string_types
from docutils.io import FileOutput
from docutils.frontend import OptionParser
from sphinx import addnodes
from sphinx.builders import Builder
from sphinx.environment import NoUri
from sphinx.util import logging
from sphinx.util.nodes import inline_all_toctrees
from sphinx.util.osutil import make_filename
from sphinx.util.console import bold, darkgreen
from sphinx.util.console import bold, darkgreen # type: ignore
from sphinx.writers.manpage import ManualPageWriter
if False:
# For type annotation
from typing import Any, Union # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
class ManualPageBuilder(Builder):
"""
@ -30,29 +40,33 @@ class ManualPageBuilder(Builder):
"""
name = 'man'
format = 'man'
supported_image_types = []
supported_image_types = [] # type: List[unicode]
def init(self):
# type: () -> None
if not self.config.man_pages:
self.warn('no "man_pages" config value found; no manual pages '
'will be written')
logger.warning('no "man_pages" config value found; no manual pages '
'will be written')
def get_outdated_docs(self):
# type: () -> Union[unicode, List[unicode]]
return 'all manpages' # for now
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
if typ == 'token':
return ''
raise NoUri
def write(self, *ignored):
# type: (Any) -> None
docwriter = ManualPageWriter(self)
docsettings = OptionParser(
defaults=self.env.settings,
components=(docwriter,),
read_config_files=True).get_default_values()
self.info(bold('writing... '), nonl=True)
logger.info(bold('writing... '), nonl=True)
for info in self.config.man_pages:
docname, name, description, authors, section = info
@ -63,16 +77,16 @@ class ManualPageBuilder(Builder):
authors = []
targetname = '%s.%s' % (name, section)
self.info(darkgreen(targetname) + ' { ', nonl=True)
logger.info(darkgreen(targetname) + ' { ', nonl=True)
destination = FileOutput(
destination_path=path.join(self.outdir, targetname),
encoding='utf-8')
tree = self.env.get_doctree(docname)
docnames = set()
docnames = set() # type: Set[unicode]
largetree = inline_all_toctrees(self, docnames, docname, tree,
darkgreen, [docname])
self.info('} ', nonl=True)
logger.info('} ', nonl=True)
self.env.resolve_references(largetree, docname, self)
# remove pending_xref nodes
for pendingnode in largetree.traverse(addnodes.pending_xref):
@ -85,13 +99,15 @@ class ManualPageBuilder(Builder):
largetree.settings.section = section
docwriter.write(largetree, destination)
self.info()
logger.info('')
def finish(self):
# type: () -> None
pass
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(ManualPageBuilder)
app.add_config_value('man_pages',

50
sphinx/builders/qthelp.py
View File

@ -16,14 +16,24 @@ import posixpath
from os import path
from six import text_type
from docutils import nodes
from sphinx import addnodes
from sphinx.builders.html import StandaloneHTMLBuilder
from sphinx.util import force_decode
from sphinx.environment.adapters.indexentries import IndexEntries
from sphinx.util import force_decode, logging
from sphinx.util.osutil import make_filename
from sphinx.util.pycompat import htmlescape
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
_idpattern = re.compile(
r'(?P<title>.+) (\((class in )?(?P<id>[\w\.]+)( (?P<descr>\w+))?\))$')
@ -115,6 +125,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
search = False
def init(self):
# type: () -> None
StandaloneHTMLBuilder.init(self)
# the output files for HTML help must be .html only
self.out_suffix = '.html'
@ -122,19 +133,23 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
# self.config.html_style = 'traditional.css'
def get_theme_config(self):
# type: () -> Tuple[unicode, Dict]
return self.config.qthelp_theme, self.config.qthelp_theme_options
def handle_finish(self):
# type: () -> None
self.build_qhp(self.outdir, self.config.qthelp_basename)
def build_qhp(self, outdir, outname):
self.info('writing project file...')
# type: (unicode, unicode) -> None
logger.info('writing project file...')
# sections
tocdoc = self.env.get_and_resolve_doctree(self.config.master_doc, self,
prune_toctrees=False)
def istoctree(node):
# type: (nodes.Node) -> bool
return isinstance(node, addnodes.compact_paragraph) and \
'toctree' in node
sections = []
@ -153,15 +168,15 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
new_sections.append(force_decode(section, None))
else:
new_sections.append(section)
sections = u'\n'.join(new_sections)
sections = u'\n'.join(new_sections) # type: ignore
# keywords
keywords = []
index = self.env.create_index(self, group_entries=False)
index = IndexEntries(self.env).create_index(self, group_entries=False)
for (key, group) in index:
for title, (refs, subitems, key_) in group:
keywords.extend(self.build_keywords(title, refs, subitems))
keywords = u'\n'.join(keywords)
keywords = u'\n'.join(keywords) # type: ignore
# files
if not outdir.endswith(os.sep):
@ -179,7 +194,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
filename = path.join(root, fn)[olen:]
projectfiles.append(file_template %
{'filename': htmlescape(filename)})
projectfiles = '\n'.join(projectfiles)
projectfiles = '\n'.join(projectfiles) # type: ignore
# it seems that the "namespace" may not contain non-alphanumeric
# characters, and more than one successive dot, or leading/trailing
@ -190,8 +205,8 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
nspace = nspace.lower()
# write the project file
with codecs.open(path.join(outdir, outname + '.qhp'), 'w', 'utf-8') as f:
f.write(project_template % {
with codecs.open(path.join(outdir, outname + '.qhp'), 'w', 'utf-8') as f: # type: ignore # NOQA
f.write(project_template % { # type: ignore
'outname': htmlescape(outname),
'title': htmlescape(self.config.html_title),
'version': htmlescape(self.config.version),
@ -206,15 +221,16 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
nspace, 'doc', self.get_target_uri(self.config.master_doc))
startpage = 'qthelp://' + posixpath.join(nspace, 'doc', 'index.html')
self.info('writing collection project file...')
with codecs.open(path.join(outdir, outname + '.qhcp'), 'w', 'utf-8') as f:
f.write(collection_template % {
logger.info('writing collection project file...')
with codecs.open(path.join(outdir, outname + '.qhcp'), 'w', 'utf-8') as f: # type: ignore # NOQA
f.write(collection_template % { # type: ignore
'outname': htmlescape(outname),
'title': htmlescape(self.config.html_short_title),
'homepage': htmlescape(homepage),
'startpage': htmlescape(startpage)})
def isdocnode(self, node):
# type: (nodes.Node) -> bool
if not isinstance(node, nodes.list_item):
return False
if len(node.children) != 2:
@ -228,8 +244,9 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
return True
def write_toc(self, node, indentlevel=4):
# type: (nodes.Node, int) -> List[unicode]
# XXX this should return a Unicode string, not a bytestring
parts = []
parts = [] # type: List[unicode]
if self.isdocnode(node):
refnode = node.children[0][0]
link = refnode['refuri']
@ -247,7 +264,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
link = node['refuri']
title = htmlescape(node.astext()).replace('"', '&quot;')
item = section_template % {'title': title, 'ref': link}
item = u' ' * 4 * indentlevel + item
item = u' ' * 4 * indentlevel + item # type: ignore
parts.append(item.encode('ascii', 'xmlcharrefreplace'))
elif isinstance(node, nodes.bullet_list):
for subnode in node:
@ -259,7 +276,8 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
return parts
def keyword_item(self, name, ref):
matchobj = _idpattern.match(name)
# type: (unicode, Any) -> unicode
matchobj = _idpattern.match(name) # type: ignore
if matchobj:
groupdict = matchobj.groupdict()
shortname = groupdict['title']
@ -280,7 +298,8 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
return item
def build_keywords(self, title, refs, subitems):
keywords = []
# type: (unicode, List[Any], Any) -> List[unicode]
keywords = [] # type: List[unicode]
title = htmlescape(title)
# if len(refs) == 0: # XXX
@ -304,6 +323,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.setup_extension('sphinx.builders.html')
app.add_builder(QtHelpBuilder)

64
sphinx/builders/texinfo.py
View File

@ -12,6 +12,7 @@
from os import path
from six import iteritems
from docutils import nodes
from docutils.io import FileOutput
from docutils.utils import new_document
@ -21,11 +22,19 @@ from sphinx import addnodes
from sphinx.locale import _
from sphinx.builders import Builder
from sphinx.environment import NoUri
from sphinx.util import logging
from sphinx.util.nodes import inline_all_toctrees
from sphinx.util.osutil import SEP, copyfile, make_filename
from sphinx.util.console import bold, darkgreen
from sphinx.util.console import bold, darkgreen # type: ignore
from sphinx.writers.texinfo import TexinfoWriter
if False:
# For type annotation
from sphinx.application import Sphinx # NOQA
from typing import Any, Iterable, Tuple, Union # NOQA
logger = logging.getLogger(__name__)
TEXINFO_MAKEFILE = '''\
# Makefile for Sphinx Texinfo output
@ -91,47 +100,53 @@ class TexinfoBuilder(Builder):
'image/gif']
def init(self):
self.docnames = []
self.document_data = []
# type: () -> None
self.docnames = [] # type: Iterable[unicode]
self.document_data = [] # type: List[Tuple[unicode, unicode, unicode, unicode, unicode, unicode, unicode, bool]] # NOQA
def get_outdated_docs(self):
# type: () -> Union[unicode, List[unicode]]
return 'all documents' # for now
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
if docname not in self.docnames:
raise NoUri
else:
return '%' + docname
def get_relative_uri(self, from_, to, typ=None):
# type: (unicode, unicode, unicode) -> unicode
# ignore source path
return self.get_target_uri(to, typ)
def init_document_data(self):
# type: () -> None
preliminary_document_data = [list(x) for x in self.config.texinfo_documents]
if not preliminary_document_data:
self.warn('no "texinfo_documents" config value found; no documents '
'will be written')
logger.warning('no "texinfo_documents" config value found; no documents '
'will be written')
return
# assign subdirs to titles
self.titles = []
self.titles = [] # type: List[Tuple[unicode, unicode]]
for entry in preliminary_document_data:
docname = entry[0]
if docname not in self.env.all_docs:
self.warn('"texinfo_documents" config value references unknown '
'document %s' % docname)
logger.warning('"texinfo_documents" config value references unknown '
'document %s', docname)
continue
self.document_data.append(entry)
self.document_data.append(entry) # type: ignore
if docname.endswith(SEP + 'index'):
docname = docname[:-5]
self.titles.append((docname, entry[2]))
def write(self, *ignored):
# type: (Any) -> None
self.init_document_data()
for entry in self.document_data:
docname, targetname, title, author = entry[:4]
targetname += '.texi'
direntry = description = category = ''
direntry = description = category = '' # type: unicode
if len(entry) > 6:
direntry, description, category = entry[4:7]
toctree_only = False
@ -140,11 +155,11 @@ class TexinfoBuilder(Builder):
destination = FileOutput(
destination_path=path.join(self.outdir, targetname),
encoding='utf-8')
self.info("processing " + targetname + "... ", nonl=1)
logger.info("processing " + targetname + "... ", nonl=1)
doctree = self.assemble_doctree(
docname, toctree_only,
appendices=(self.config.texinfo_appendices or []))
self.info("writing... ", nonl=1)
logger.info("writing... ", nonl=1)
self.post_process_images(doctree)
docwriter = TexinfoWriter(self)
settings = OptionParser(
@ -161,11 +176,12 @@ class TexinfoBuilder(Builder):
settings.docname = docname
doctree.settings = settings
docwriter.write(doctree, destination)
self.info("done")
logger.info("done")
def assemble_doctree(self, indexfile, toctree_only, appendices):
# type: (unicode, bool, List[unicode]) -> nodes.Node
self.docnames = set([indexfile] + appendices)
self.info(darkgreen(indexfile) + " ", nonl=1)
logger.info(darkgreen(indexfile) + " ", nonl=1)
tree = self.env.get_doctree(indexfile)
tree['docname'] = indexfile
if toctree_only:
@ -186,8 +202,8 @@ class TexinfoBuilder(Builder):
appendix = self.env.get_doctree(docname)
appendix['docname'] = docname
largetree.append(appendix)
self.info()
self.info("resolving references...")
logger.info('')
logger.info("resolving references...")
self.env.resolve_references(largetree, indexfile, self)
# TODO: add support for external :ref:s
for pendingnode in largetree.traverse(addnodes.pending_xref):
@ -206,28 +222,30 @@ class TexinfoBuilder(Builder):
return largetree
def finish(self):
# type: () -> None
# copy image files
if self.images:
self.info(bold('copying images...'), nonl=1)
logger.info(bold('copying images...'), nonl=1)
for src, dest in iteritems(self.images):
self.info(' ' + src, nonl=1)
logger.info(' ' + src, nonl=1)
copyfile(path.join(self.srcdir, src),
path.join(self.outdir, dest))
self.info()
logger.info('')
self.info(bold('copying Texinfo support files... '), nonl=True)
logger.info(bold('copying Texinfo support files... '), nonl=True)
# copy Makefile
fn = path.join(self.outdir, 'Makefile')
self.info(fn, nonl=1)
logger.info(fn, nonl=1)
try:
with open(fn, 'w') as mkfile:
mkfile.write(TEXINFO_MAKEFILE)
except (IOError, OSError) as err:
self.warn("error writing file %s: %s" % (fn, err))
self.info(' done')
logger.warning("error writing file %s: %s", fn, err)
logger.info(' done')
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(TexinfoBuilder)
app.add_config_value('texinfo_documents',

22
sphinx/builders/text.py
View File

@ -15,9 +15,18 @@ from os import path
from docutils.io import StringOutput
from sphinx.builders import Builder
from sphinx.util import logging
from sphinx.util.osutil import ensuredir, os_path
from sphinx.writers.text import TextWriter
if False:
# For type annotation
from typing import Any, Iterator # NOQA
from docutils import nodes # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
class TextBuilder(Builder):
name = 'text'
@ -25,10 +34,14 @@ class TextBuilder(Builder):
out_suffix = '.txt'
allow_parallel = True
current_docname = None # type: unicode
def init(self):
# type: () -> None
pass
def get_outdated_docs(self):
# type: () -> Iterator[unicode]
for docname in self.env.found_docs:
if docname not in self.env.all_docs:
yield docname
@ -48,28 +61,33 @@ class TextBuilder(Builder):
pass
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
return ''
def prepare_writing(self, docnames):
# type: (Set[unicode]) -> None
self.writer = TextWriter(self)
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
self.current_docname = docname
destination = StringOutput(encoding='utf-8')
self.writer.write(doctree, destination)
outfilename = path.join(self.outdir, os_path(docname) + self.out_suffix)
ensuredir(path.dirname(outfilename))
try:
with codecs.open(outfilename, 'w', 'utf-8') as f:
with codecs.open(outfilename, 'w', 'utf-8') as f: # type: ignore
f.write(self.writer.output)
except (IOError, OSError) as err:
self.warn("error writing file %s: %s" % (outfilename, err))
logger.warning("error writing file %s: %s", outfilename, err)
def finish(self):
# type: () -> None
pass
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(TextBuilder)
app.add_config_value('text_sectionchars', '*=-~"+`', 'env')

25
sphinx/builders/websupport.py
View File

@ -20,6 +20,12 @@ from sphinx.util.osutil import os_path, relative_uri, ensuredir, copyfile
from sphinx.builders.html import PickleHTMLBuilder
from sphinx.writers.websupport import WebSupportTranslator
if False:
# For type annotation
from typing import Any, Iterable, Tuple # NOQA
from docutils import nodes # NOQA
from sphinx.application import Sphinx # NOQA
class WebSupportBuilder(PickleHTMLBuilder):
"""
@ -30,6 +36,7 @@ class WebSupportBuilder(PickleHTMLBuilder):
versioning_compare = True # for commentable node's uuid stability.
def init(self):
# type: () -> None
PickleHTMLBuilder.init(self)
# templates are needed for this builder, but the serializing
# builder does not initialize them
@ -41,20 +48,24 @@ class WebSupportBuilder(PickleHTMLBuilder):
self.script_files.append('_static/websupport.js')
def set_webinfo(self, staticdir, virtual_staticdir, search, storage):
# type: (unicode, unicode, Any, unicode) -> None
self.staticdir = staticdir
self.virtual_staticdir = virtual_staticdir
self.search = search
self.storage = storage
def init_translator_class(self):
# type: () -> None
if self.translator_class is None:
self.translator_class = WebSupportTranslator
def prepare_writing(self, docnames):
# type: (Iterable[unicode]) -> None
PickleHTMLBuilder.prepare_writing(self, docnames)
self.globalcontext['no_search_suffix'] = True
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
destination = StringOutput(encoding='utf-8')
doctree.settings = self.docsettings
@ -72,6 +83,7 @@ class WebSupportBuilder(PickleHTMLBuilder):
self.handle_page(docname, ctx, event_arg=doctree)
def write_doc_serialized(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
self.imgpath = '/' + posixpath.join(self.virtual_staticdir, self.imagedir)
self.post_process_images(doctree)
title = self.env.longtitles.get(docname)
@ -79,10 +91,12 @@ class WebSupportBuilder(PickleHTMLBuilder):
self.index_page(docname, doctree, title)
def load_indexer(self, docnames):
self.indexer = self.search
self.indexer.init_indexing(changed=docnames)
# type: (Iterable[unicode]) -> None
self.indexer = self.search # type: ignore
self.indexer.init_indexing(changed=docnames) # type: ignore
def _render_page(self, pagename, addctx, templatename, event_arg=None):
# type: (unicode, Dict, unicode, unicode) -> Tuple[Dict, Dict]
# This is mostly copied from StandaloneHTMLBuilder. However, instead
# of rendering the template and saving the html, create a context
# dict and pickle it.
@ -91,6 +105,7 @@ class WebSupportBuilder(PickleHTMLBuilder):
def pathto(otheruri, resource=False,
baseuri=self.get_target_uri(pagename)):
# type: (unicode, bool, unicode) -> unicode
if resource and '://' in otheruri:
return otheruri
elif not resource:
@ -128,6 +143,7 @@ class WebSupportBuilder(PickleHTMLBuilder):
def handle_page(self, pagename, addctx, templatename='page.html',
outfilename=None, event_arg=None):
# type: (unicode, Dict, unicode, unicode, unicode) -> None
ctx, doc_ctx = self._render_page(pagename, addctx,
templatename, event_arg)
@ -146,6 +162,7 @@ class WebSupportBuilder(PickleHTMLBuilder):
copyfile(self.env.doc2path(pagename), source_name)
def handle_finish(self):
# type: () -> None
# get global values for css and script files
_, doc_ctx = self._render_page('tmp', {}, 'page.html')
self.globalcontext['css'] = doc_ctx['css']
@ -164,10 +181,12 @@ class WebSupportBuilder(PickleHTMLBuilder):
shutil.move(src, dst)
def dump_search_index(self):
self.indexer.finish_indexing()
# type: () -> None
self.indexer.finish_indexing() # type: ignore
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(WebSupportBuilder)
return {

19
sphinx/builders/xml.py
View File

@ -16,9 +16,17 @@ from docutils import nodes
from docutils.io import StringOutput
from sphinx.builders import Builder
from sphinx.util import logging
from sphinx.util.osutil import ensuredir, os_path
from sphinx.writers.xml import XMLWriter, PseudoXMLWriter
if False:
# For type annotation
from typing import Any, Iterator # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
class XMLBuilder(Builder):
"""
@ -32,9 +40,11 @@ class XMLBuilder(Builder):
_writer_class = XMLWriter
def init(self):
# type: () -> None
pass
def get_outdated_docs(self):
# type: () -> Iterator[unicode]
for docname in self.env.found_docs:
if docname not in self.env.all_docs:
yield docname
@ -54,12 +64,15 @@ class XMLBuilder(Builder):
pass
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
return docname
def prepare_writing(self, docnames):
# type: (Set[unicode]) -> None
self.writer = self._writer_class(self)
def write_doc(self, docname, doctree):
# type: (unicode, nodes.Node) -> None
# work around multiple string % tuple issues in docutils;
# replace tuples in attribute values with lists
doctree = doctree.deepcopy()
@ -77,12 +90,13 @@ class XMLBuilder(Builder):
outfilename = path.join(self.outdir, os_path(docname) + self.out_suffix)
ensuredir(path.dirname(outfilename))
try:
with codecs.open(outfilename, 'w', 'utf-8') as f:
with codecs.open(outfilename, 'w', 'utf-8') as f: # type: ignore
f.write(self.writer.output)
except (IOError, OSError) as err:
self.warn("error writing file %s: %s" % (outfilename, err))
logger.warning("error writing file %s: %s", outfilename, err)
def finish(self):
# type: () -> None
pass
@ -98,6 +112,7 @@ class PseudoXMLBuilder(XMLBuilder):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(XMLBuilder)
app.add_builder(PseudoXMLBuilder)

38
sphinx/cmdline.py
View File

@ -16,17 +16,22 @@ import traceback
from os import path
from six import text_type, binary_type
from docutils.utils import SystemMessage
from sphinx import __display_version__
from sphinx.errors import SphinxError
from sphinx.application import Sphinx
from sphinx.util import Tee, format_exception_cut_frames, save_traceback
from sphinx.util.console import red, nocolor, color_terminal
from sphinx.util.console import red, nocolor, color_terminal # type: ignore
from sphinx.util.docutils import docutils_namespace
from sphinx.util.osutil import abspath, fs_encoding
from sphinx.util.pycompat import terminal_safe
if False:
# For type annotation
from typing import Any, IO, Union # NOQA
USAGE = """\
Sphinx v%s
@ -45,18 +50,21 @@ For more information, visit <http://sphinx-doc.org/>.
class MyFormatter(optparse.IndentedHelpFormatter):
def format_usage(self, usage):
# type: (Any) -> Any
return usage
def format_help(self, formatter):
result = []
if self.description:
# type: (Any) -> unicode
result = [] # type: List[unicode]
if self.description: # type: ignore
result.append(self.format_description(formatter))
if self.option_list:
result.append(self.format_option_help(formatter))
if self.option_list: # type: ignore
result.append(self.format_option_help(formatter)) # type: ignore
return "\n".join(result)
def handle_exception(app, opts, exception, stderr=sys.stderr):
# type: (Sphinx, Any, Union[Exception, KeyboardInterrupt], IO) -> None
if opts.pdb:
import pdb
print(red('Exception occurred while building, starting debugger:'),
@ -107,9 +115,7 @@ def handle_exception(app, opts, exception, stderr=sys.stderr):
def main(argv):
if not color_terminal():
nocolor()
# type: (List[unicode]) -> int
parser = optparse.OptionParser(USAGE, epilog=EPILOG, formatter=MyFormatter())
parser.add_option('--version', action='store_true', dest='version',
help='show version information and exit')
@ -158,8 +164,12 @@ def main(argv):
help='no output on stdout, just warnings on stderr')
group.add_option('-Q', action='store_true', dest='really_quiet',
help='no output at all, not even warnings')
group.add_option('-N', action='store_true', dest='nocolor',
help='do not emit colored output')
group.add_option('--color', dest='color',
action='store_const', const='yes', default='auto',
help='Do emit colored output (default: auto-detect)')
group.add_option('-N', '--no-color', dest='color',
action='store_const', const='no',
help='Do not emit colored output (default: auot-detect)')
group.add_option('-w', metavar='FILE', dest='warnfile',
help='write warnings (and errors) to given file')
group.add_option('-W', action='store_true', dest='warningiserror',
@ -210,11 +220,11 @@ def main(argv):
# handle remaining filename arguments
filenames = args[2:]
err = 0
err = 0 # type: ignore
for filename in filenames:
if not path.isfile(filename):
print('Error: Cannot find file %r.' % filename, file=sys.stderr)
err = 1
err = 1 # type: ignore
if err:
return 1
@ -229,7 +239,7 @@ def main(argv):
print('Error: Cannot combine -a option and filenames.', file=sys.stderr)
return 1
if opts.nocolor:
if opts.color == 'no' or (opts.color == 'auto' and not color_terminal()):
nocolor()
doctreedir = abspath(opts.doctreedir or path.join(outdir, '.doctrees'))
@ -249,7 +259,7 @@ def main(argv):
print('Error: Cannot open warning file %r: %s' %
(opts.warnfile, exc), file=sys.stderr)
sys.exit(1)
warning = Tee(warning, warnfp)
warning = Tee(warning, warnfp) # type: ignore
error = warning
confoverrides = {}

96
sphinx/config.py
View File

@ -13,12 +13,21 @@ import re
from os import path, getenv
from six import PY2, PY3, iteritems, string_types, binary_type, text_type, integer_types
from typing import Any, NamedTuple, Union
from sphinx.errors import ConfigError
from sphinx.locale import l_
from sphinx.util import logging
from sphinx.util.i18n import format_date
from sphinx.util.osutil import cd
from sphinx.util.pycompat import execfile_, NoneType
from sphinx.util.i18n import format_date
if False:
# For type annotation
from typing import Any, Callable, Iterable, Iterator, Tuple # NOQA
from sphinx.util.tags import Tags # NOQA
logger = logging.getLogger(__name__)
nonascii_re = re.compile(br'[\x80-\xff]')
copyright_year_re = re.compile(r'^((\d{4}-)?)(\d{4})(?=[ ,])')
@ -35,6 +44,13 @@ CONFIG_PERMITTED_TYPE_WARNING = "The config value `{name}' has type `{current.__
CONFIG_TYPE_WARNING = "The config value `{name}' has type `{current.__name__}', " \
"defaults to `{default.__name__}'."
if PY3:
unicode = str # special alias for static typing...
ConfigValue = NamedTuple('ConfigValue', [('name', str),
('value', Any),
('rebuild', Union[bool, unicode])])
class ENUM(object):
"""represents the config value should be a one of candidates.
@ -43,13 +59,15 @@ class ENUM(object):
app.add_config_value('latex_show_urls', 'no', ENUM('no', 'footnote', 'inline'))
"""
def __init__(self, *candidates):
# type: (unicode) -> None
self.candidates = candidates
def match(self, value):
# type: (unicode) -> bool
return value in self.candidates
string_classes = [text_type]
string_classes = [text_type] # type: List
if PY2:
string_classes.append(binary_type) # => [str, unicode]
@ -114,15 +132,13 @@ class Config(object):
tls_verify = (True, 'env'),
tls_cacerts = (None, 'env'),
# pre-initialized confval for HTML builder
html_translator_class = (None, 'html', string_classes),
)
) # type: Dict[unicode, Tuple]
def __init__(self, dirname, filename, overrides, tags):
# type: (unicode, unicode, Dict, Tags) -> None
self.overrides = overrides
self.values = Config.config_values.copy()
config = {}
config = {} # type: Dict[unicode, Any]
if dirname is not None:
config_file = path.join(dirname, filename)
config['__file__'] = config_file
@ -140,14 +156,14 @@ class Config(object):
self._raw_config = config
# these two must be preinitialized because extensions can add their
# own config values
self.setup = config.get('setup', None)
self.setup = config.get('setup', None) # type: Callable
if 'extensions' in overrides:
if isinstance(overrides['extensions'], string_types):
config['extensions'] = overrides.pop('extensions').split(',')
else:
config['extensions'] = overrides.pop('extensions')
self.extensions = config.get('extensions', [])
self.extensions = config.get('extensions', []) # type: List[unicode]
# correct values of copyright year that are not coherent with
# the SOURCE_DATE_EPOCH environment variable (if set)
@ -155,10 +171,10 @@ class Config(object):
if getenv('SOURCE_DATE_EPOCH') is not None:
for k in ('copyright', 'epub_copyright'):
if k in config:
config[k] = copyright_year_re.sub('\g<1>%s' % format_date('%Y'),
config[k])
config[k] = copyright_year_re.sub('\g<1>%s' % format_date('%Y'), config[k])
def check_types(self, warn):
def check_types(self):
# type: () -> None
# check all values for deviation from the default value's type, since
# that can result in TypeErrors all over the place
# NB. since config values might use l_() we have to wait with calling
@ -177,7 +193,7 @@ class Config(object):
current = self[name]
if isinstance(permitted, ENUM):
if not permitted.match(current):
warn(CONFIG_ENUM_WARNING.format(
logger.warning(CONFIG_ENUM_WARNING.format(
name=name, current=current, candidates=permitted.candidates))
else:
if type(current) is type(default):
@ -192,23 +208,25 @@ class Config(object):
continue # at least we share a non-trivial base class
if permitted:
warn(CONFIG_PERMITTED_TYPE_WARNING.format(
logger.warning(CONFIG_PERMITTED_TYPE_WARNING.format(
name=name, current=type(current),
permitted=str([cls.__name__ for cls in permitted])))
else:
warn(CONFIG_TYPE_WARNING.format(
logger.warning(CONFIG_TYPE_WARNING.format(
name=name, current=type(current), default=type(default)))
def check_unicode(self, warn):
def check_unicode(self):
# type: () -> None
# check all string values for non-ASCII characters in bytestrings,
# since that can result in UnicodeErrors all over the place
for name, value in iteritems(self._raw_config):
if isinstance(value, binary_type) and nonascii_re.search(value):
warn('the config value %r is set to a string with non-ASCII '
'characters; this can lead to Unicode errors occurring. '
'Please use Unicode strings, e.g. %r.' % (name, u'Content'))
logger.warning('the config value %r is set to a string with non-ASCII '
'characters; this can lead to Unicode errors occurring. '
'Please use Unicode strings, e.g. %r.', name, u'Content')
def convert_overrides(self, name, value):
# type: (unicode, Any) -> Any
if not isinstance(value, string_types):
return value
else:
@ -218,10 +236,10 @@ class Config(object):
'ignoring (use %r to set individual elements)' %
(name, name + '.key=value'))
elif isinstance(defvalue, list):
return value.split(',')
return value.split(',') # type: ignore
elif isinstance(defvalue, integer_types):
try:
return int(value)
return int(value) # type: ignore
except ValueError:
raise ValueError('invalid number %r for config value %r, ignoring' %
(value, name))
@ -233,9 +251,10 @@ class Config(object):
else:
return value
def pre_init_values(self, warn):
def pre_init_values(self):
# type: () -> None
"""Initialize some limited config variables before loading extensions"""
variables = ['needs_sphinx', 'suppress_warnings', 'html_translator_class']
variables = ['needs_sphinx', 'suppress_warnings']
for name in variables:
try:
if name in self.overrides:
@ -243,9 +262,10 @@ class Config(object):
elif name in self._raw_config:
self.__dict__[name] = self._raw_config[name]
except ValueError as exc:
warn(exc)
logger.warning("%s", exc)
def init_values(self, warn):
def init_values(self):
# type: () -> None
config = self._raw_config
for valname, value in iteritems(self.overrides):
try:
@ -254,21 +274,22 @@ class Config(object):
config.setdefault(realvalname, {})[key] = value
continue
elif valname not in self.values:
warn('unknown config value %r in override, ignoring' % valname)
logger.warning('unknown config value %r in override, ignoring', valname)
continue
if isinstance(value, string_types):
config[valname] = self.convert_overrides(valname, value)
else:
config[valname] = value
except ValueError as exc:
warn(exc)
logger.warning("%s", exc)
for name in config:
if name in self.values:
self.__dict__[name] = config[name]
if isinstance(self.source_suffix, string_types):
self.source_suffix = [self.source_suffix]
if isinstance(self.source_suffix, string_types): # type: ignore
self.source_suffix = [self.source_suffix] # type: ignore
def __getattr__(self, name):
# type: (unicode) -> Any
if name.startswith('_'):
raise AttributeError(name)
if name not in self.values:
@ -279,13 +300,30 @@ class Config(object):
return default
def __getitem__(self, name):
# type: (unicode) -> unicode
return getattr(self, name)
def __setitem__(self, name, value):
# type: (unicode, Any) -> None
setattr(self, name, value)
def __delitem__(self, name):
# type: (unicode) -> None
delattr(self, name)
def __contains__(self, name):
# type: (unicode) -> bool
return name in self.values
def __iter__(self):
# type: () -> Iterable[ConfigValue]
for name, value in iteritems(self.values):
yield ConfigValue(name, getattr(self, name), value[1]) # type: ignore
def add(self, name, default, rebuild, types):
# type: (unicode, Any, Union[bool, unicode], Any) -> None
self.values[name] = (default, rebuild, types)
def filter(self, rebuild):
# type: (str) -> Iterator[ConfigValue]
return (value for value in self if value.rebuild == rebuild) # type: ignore

10
sphinx/deprecation.py
View File

@ -10,12 +10,16 @@
"""
class RemovedInSphinx16Warning(DeprecationWarning):
class RemovedInSphinx17Warning(DeprecationWarning):
pass
class RemovedInSphinx17Warning(PendingDeprecationWarning):
class RemovedInSphinx18Warning(PendingDeprecationWarning):
pass
RemovedInNextVersionWarning = RemovedInSphinx16Warning
class RemovedInSphinx20Warning(PendingDeprecationWarning):
pass
RemovedInNextVersionWarning = RemovedInSphinx17Warning

26
sphinx/directives/__init__.py
View File

@ -29,6 +29,12 @@ from sphinx.directives.patches import ( # noqa
Figure, Meta
)
if False:
# For type annotation
from typing import Any # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
# RE to strip backslash escapes
nl_escape_re = re.compile(r'\\\n')
@ -51,9 +57,13 @@ class ObjectDescription(Directive):
}
# types of doc fields that this directive handles, see sphinx.util.docfields
doc_field_types = []
doc_field_types = [] # type: List[Any]
domain = None # type: unicode
objtype = None # type: unicode
indexnode = None # type: addnodes.index
def get_signatures(self):
# type: () -> List[unicode]
"""
Retrieve the signatures to document from the directive arguments. By
default, signatures are given as arguments, one per line.
@ -65,6 +75,7 @@ class ObjectDescription(Directive):
return [strip_backslash_re.sub(r'\1', line.strip()) for line in lines]
def handle_signature(self, sig, signode):
# type: (unicode, addnodes.desc_signature) -> Any
"""
Parse the signature *sig* into individual nodes and append them to
*signode*. If ValueError is raised, parsing is aborted and the whole
@ -77,6 +88,7 @@ class ObjectDescription(Directive):
raise ValueError
def add_target_and_index(self, name, sig, signode):
# type: (Any, unicode, addnodes.desc_signature) -> None
"""
Add cross-reference IDs and entries to self.indexnode, if applicable.
@ -85,6 +97,7 @@ class ObjectDescription(Directive):
return # do nothing by default
def before_content(self):
# type: () -> None
"""
Called before parsing content. Used to set information about the current
directive context on the build environment.
@ -92,6 +105,7 @@ class ObjectDescription(Directive):
pass
def after_content(self):
# type: () -> None
"""
Called after parsing content. Used to reset information about the
current directive context on the build environment.
@ -99,6 +113,7 @@ class ObjectDescription(Directive):
pass
def run(self):
# type: () -> List[nodes.Node]
"""
Main directive entry function, called by docutils upon encountering the
directive.
@ -120,7 +135,7 @@ class ObjectDescription(Directive):
self.domain, self.objtype = self.name.split(':', 1)
else:
self.domain, self.objtype = '', self.name
self.env = self.state.document.settings.env
self.env = self.state.document.settings.env # type: BuildEnvironment
self.indexnode = addnodes.index(entries=[])
node = addnodes.desc()
@ -130,7 +145,7 @@ class ObjectDescription(Directive):
node['objtype'] = node['desctype'] = self.objtype
node['noindex'] = noindex = ('noindex' in self.options)
self.names = []
self.names = [] # type: List[unicode]
signatures = self.get_signatures()
for i, sig in enumerate(signatures):
# add a signature node for each signature in the current unit
@ -181,6 +196,7 @@ class DefaultRole(Directive):
final_argument_whitespace = False
def run(self):
# type: () -> List[nodes.Node]
if not self.arguments:
if '' in roles._roles:
# restore the "default" default role
@ -209,9 +225,10 @@ class DefaultDomain(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = False
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
domain_name = self.arguments[0].lower()
# if domain_name not in env.domains:
@ -225,6 +242,7 @@ class DefaultDomain(Directive):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
directives.register_directive('default-role', DefaultRole)
directives.register_directive('default-domain', DefaultDomain)
directives.register_directive('describe', ObjectDescription)

22
sphinx/directives/code.py
View File

@ -11,17 +11,22 @@ import sys
import codecs
from difflib import unified_diff
from six import string_types
from docutils import nodes
from docutils.parsers.rst import Directive, directives
from docutils.statemachine import ViewList
from six import string_types
from sphinx import addnodes
from sphinx.locale import _
from sphinx.util import parselinenos
from sphinx.util.nodes import set_source_info
if False:
# For type annotation
from typing import Any # NOQA
from sphinx.application import Sphinx # NOQA
class Highlight(Directive):
"""
@ -38,6 +43,7 @@ class Highlight(Directive):
}
def run(self):
# type: () -> List[nodes.Node]
if 'linenothreshold' in self.options:
try:
linenothreshold = int(self.options['linenothreshold'])
@ -50,6 +56,7 @@ class Highlight(Directive):
def dedent_lines(lines, dedent):
# type: (List[unicode], int) -> List[unicode]
if not dedent:
return lines
@ -64,6 +71,7 @@ def dedent_lines(lines, dedent):
def container_wrapper(directive, literal_node, caption):
# type: (Directive, nodes.Node, unicode) -> nodes.container
container_node = nodes.container('', literal_block=True,
classes=['literal-block-wrapper'])
parsed = nodes.Element()
@ -101,6 +109,7 @@ class CodeBlock(Directive):
}
def run(self):
# type: () -> List[nodes.Node]
code = u'\n'.join(self.content)
linespec = self.options.get('emphasize-lines')
@ -137,7 +146,7 @@ class CodeBlock(Directive):
literal = container_wrapper(self, literal, caption)
except ValueError as exc:
document = self.state.document
errmsg = _('Invalid caption: %s' % exc[0][0].astext())
errmsg = _('Invalid caption: %s' % exc[0][0].astext()) # type: ignore
return [document.reporter.warning(errmsg, line=self.lineno)]
# literal will be note_implicit_target that is linked from caption and numref.
@ -182,11 +191,12 @@ class LiteralInclude(Directive):
}
def read_with_encoding(self, filename, document, codec_info, encoding):
# type: (unicode, nodes.Node, Any, unicode) -> List
try:
with codecs.StreamReaderWriter(open(filename, 'rb'), codec_info[2],
codec_info[3], 'strict') as f:
lines = f.readlines()
lines = dedent_lines(lines, self.options.get('dedent'))
lines = dedent_lines(lines, self.options.get('dedent')) # type: ignore
return lines
except (IOError, OSError):
return [document.reporter.warning(
@ -199,6 +209,7 @@ class LiteralInclude(Directive):
(encoding, filename))]
def run(self):
# type: () -> List[nodes.Node]
document = self.state.document
if not document.settings.file_insertion_enabled:
return [document.reporter.warning('File insertion disabled',
@ -371,7 +382,7 @@ class LiteralInclude(Directive):
retnode = container_wrapper(self, retnode, caption)
except ValueError as exc:
document = self.state.document
errmsg = _('Invalid caption: %s' % exc[0][0].astext())
errmsg = _('Invalid caption: %s' % exc[0][0].astext()) # type: ignore
return [document.reporter.warning(errmsg, line=self.lineno)]
# retnode will be note_implicit_target that is linked from caption and numref.
@ -382,6 +393,7 @@ class LiteralInclude(Directive):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
directives.register_directive('highlight', Highlight)
directives.register_directive('highlightlang', Highlight) # old
directives.register_directive('code-block', CodeBlock)

38
sphinx/directives/other.py
View File

@ -8,6 +8,7 @@
"""
from six.moves import range
from docutils import nodes
from docutils.parsers.rst import Directive, directives
from docutils.parsers.rst.directives.admonitions import BaseAdmonition
@ -21,8 +22,14 @@ from sphinx.util.nodes import explicit_title_re, set_source_info, \
process_index_entry
from sphinx.util.matching import patfilter
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
def int_or_nothing(argument):
# type: (unicode) -> int
if not argument:
return 999
return int(argument)
@ -50,6 +57,7 @@ class TocTree(Directive):
}
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
suffixes = env.config.source_suffix
glob = 'glob' in self.options
@ -57,7 +65,7 @@ class TocTree(Directive):
ret = []
# (title, ref) pairs, where ref may be a document, or an external link,
# and title may be None if the document's title is to be used
entries = []
entries = [] # type: List[Tuple[unicode, unicode]]
includefiles = []
all_docnames = env.found_docs.copy()
# don't add the currently visited file in catch-all patterns
@ -136,9 +144,10 @@ class Author(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
if not env.config.show_authors:
return []
@ -168,20 +177,21 @@ class Index(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
arguments = self.arguments[0].split('\n')
env = self.state.document.settings.env
targetid = 'index-%s' % env.new_serialno('index')
targetnode = nodes.target('', '', ids=[targetid])
self.state.document.note_explicit_target(targetnode)
indexnode = addnodes.index()
indexnode['entries'] = ne = []
indexnode['entries'] = []
indexnode['inline'] = False
set_source_info(self, indexnode)
for entry in arguments:
ne.extend(process_index_entry(entry, targetid))
indexnode['entries'].extend(process_index_entry(entry, targetid))
return [indexnode, targetnode]
@ -193,9 +203,10 @@ class VersionChange(Directive):
required_arguments = 1
optional_arguments = 1
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
node = addnodes.versionmodified()
node.document = self.state.document
set_source_info(self, node)
@ -248,9 +259,10 @@ class TabularColumns(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
node = addnodes.tabular_col_spec()
node['spec'] = self.arguments[0]
set_source_info(self, node)
@ -265,9 +277,10 @@ class Centered(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
if not self.arguments:
return []
subnode = addnodes.centered()
@ -285,9 +298,10 @@ class Acks(Directive):
required_arguments = 0
optional_arguments = 0
final_argument_whitespace = False
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
node = addnodes.acks()
node.document = self.state.document
self.state.nested_parse(self.content, self.content_offset, node)
@ -311,6 +325,7 @@ class HList(Directive):
}
def run(self):
# type: () -> List[nodes.Node]
ncolumns = self.options.get('columns', 2)
node = nodes.paragraph()
node.document = self.state.document
@ -342,9 +357,10 @@ class Only(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
node = addnodes.only()
node.document = self.state.document
set_source_info(self, node)
@ -398,6 +414,7 @@ class Include(BaseInclude):
"""
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
if self.arguments[0].startswith('<') and \
self.arguments[0].endswith('>'):
@ -410,6 +427,7 @@ class Include(BaseInclude):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
directives.register_directive('toctree', TocTree)
directives.register_directive('sectionauthor', Author)
directives.register_directive('moduleauthor', Author)

7
sphinx/directives/patches.py
View File

@ -13,6 +13,10 @@ from docutils.parsers.rst.directives import images, html
from sphinx import addnodes
if False:
# For type annotation
from sphinx.application import Sphinx # NOQA
class Figure(images.Figure):
"""The figure directive which applies `:name:` option to the figure node
@ -20,6 +24,7 @@ class Figure(images.Figure):
"""
def run(self):
# type: () -> List[nodes.Node]
name = self.options.pop('name', None)
result = images.Figure.run(self)
if len(result) == 2 or isinstance(result[0], nodes.system_message):
@ -39,6 +44,7 @@ class Figure(images.Figure):
class Meta(html.Meta):
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
result = html.Meta.run(self)
for node in result:
@ -56,6 +62,7 @@ class Meta(html.Meta):
def setup(app):
# type: (Sphinx) -> Dict
directives.register_directive('figure', Figure)
directives.register_directive('meta', Meta)

70
sphinx/domains/__init__.py
View File

@ -17,6 +17,14 @@ from six import iteritems
from sphinx.errors import SphinxError
from sphinx.locale import _
if False:
# For type annotation
from typing import Any, Callable, Iterable, Tuple, Type, Union # NOQA
from docutils import nodes # NOQA
from docutils.parsers.rst.states import Inliner # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
class ObjType(object):
"""
@ -38,9 +46,10 @@ class ObjType(object):
}
def __init__(self, lname, *roles, **attrs):
self.lname = lname
self.roles = roles
self.attrs = self.known_attrs.copy()
# type: (unicode, Any, Any) -> None
self.lname = lname # type: unicode
self.roles = roles # type: Tuple
self.attrs = self.known_attrs.copy() # type: Dict
self.attrs.update(attrs)
@ -59,17 +68,19 @@ class Index(object):
domains using :meth:`~sphinx.application.Sphinx.add_index_to_domain()`.
"""
name = None
localname = None
shortname = None
name = None # type: unicode
localname = None # type: unicode
shortname = None # type: unicode
def __init__(self, domain):
# type: (Domain) -> None
if self.name is None or self.localname is None:
raise SphinxError('Index subclass %s has no valid name or localname'
% self.__class__.__name__)
self.domain = domain
def generate(self, docnames=None):
# type: (Iterable[unicode]) -> Tuple[List[Tuple[unicode, List[List[Union[unicode, int]]]]], bool] # NOQA
"""Return entries for the index given by *name*. If *docnames* is
given, restrict to entries referring to these docnames.
@ -97,7 +108,7 @@ class Index(object):
Qualifier and description are not rendered e.g. in LaTeX output.
"""
return []
raise NotImplementedError
class Domain(object):
@ -128,23 +139,26 @@ class Domain(object):
#: domain label: longer, more descriptive (used in messages)
label = ''
#: type (usually directive) name -> ObjType instance
object_types = {}
object_types = {} # type: Dict[unicode, Any]
#: directive name -> directive class
directives = {}
directives = {} # type: Dict[unicode, Any]
#: role name -> role callable
roles = {}
roles = {} # type: Dict[unicode, Callable]
#: a list of Index subclasses
indices = []
indices = [] # type: List[Type[Index]]
#: role name -> a warning message if reference is missing
dangling_warnings = {}
dangling_warnings = {} # type: Dict[unicode, unicode]
#: data value for a fresh environment
initial_data = {}
initial_data = {} # type: Dict
#: data value
data = None # type: Dict
#: data version, bump this when the format of `self.data` changes
data_version = 0
def __init__(self, env):
self.env = env
# type: (BuildEnvironment) -> None
self.env = env # type: BuildEnvironment
if self.name not in env.domaindata:
assert isinstance(self.initial_data, dict)
new_data = copy.deepcopy(self.initial_data)
@ -154,18 +168,19 @@ class Domain(object):
self.data = env.domaindata[self.name]
if self.data['version'] != self.data_version:
raise IOError('data of %r domain out of date' % self.label)
self._role_cache = {}
self._directive_cache = {}
self._role2type = {}
self._type2role = {}
self._role_cache = {} # type: Dict[unicode, Callable]
self._directive_cache = {} # type: Dict[unicode, Callable]
self._role2type = {} # type: Dict[unicode, List[unicode]]
self._type2role = {} # type: Dict[unicode, unicode]
for name, obj in iteritems(self.object_types):
for rolename in obj.roles:
self._role2type.setdefault(rolename, []).append(name)
self._type2role[name] = obj.roles[0] if obj.roles else ''
self.objtypes_for_role = self._role2type.get
self.role_for_objtype = self._type2role.get
self.objtypes_for_role = self._role2type.get # type: Callable[[unicode], List[unicode]] # NOQA
self.role_for_objtype = self._type2role.get # type: Callable[[unicode], unicode]
def role(self, name):
# type: (unicode) -> Callable
"""Return a role adapter function that always gives the registered
role its full name ('domain:name') as the first argument.
"""
@ -175,14 +190,15 @@ class Domain(object):
return None
fullname = '%s:%s' % (self.name, name)
def role_adapter(typ, rawtext, text, lineno, inliner,
options={}, content=[]):
def role_adapter(typ, rawtext, text, lineno, inliner, options={}, content=[]):
# type: (unicode, unicode, unicode, int, Inliner, Dict, List[unicode]) -> nodes.Node # NOQA
return self.roles[name](fullname, rawtext, text, lineno,
inliner, options, content)
self._role_cache[name] = role_adapter
return role_adapter
def directive(self, name):
# type: (unicode) -> Callable
"""Return a directive adapter class that always gives the registered
directive its full name ('domain:name') as ``self.name``.
"""
@ -193,8 +209,9 @@ class Domain(object):
fullname = '%s:%s' % (self.name, name)
BaseDirective = self.directives[name]
class DirectiveAdapter(BaseDirective):
class DirectiveAdapter(BaseDirective): # type: ignore
def run(self):
# type: () -> List[nodes.Node]
self.name = fullname
return BaseDirective.run(self)
self._directive_cache[name] = DirectiveAdapter
@ -203,10 +220,12 @@ class Domain(object):
# methods that should be overwritten
def clear_doc(self, docname):
# type: (unicode) -> None
"""Remove traces of a document in the domain-specific inventories."""
pass
def merge_domaindata(self, docnames, otherdata):
# type: (List[unicode], Dict) -> None
"""Merge in data regarding *docnames* from a different domaindata
inventory (coming from a subprocess in parallel builds).
"""
@ -215,11 +234,13 @@ class Domain(object):
self.__class__)
def process_doc(self, env, docname, document):
# type: (BuildEnvironment, unicode, nodes.Node) -> None
"""Process a document after it is read by the environment."""
pass
def resolve_xref(self, env, fromdocname, builder,
typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
"""Resolve the pending_xref *node* with the given *typ* and *target*.
This method should return a new node, to replace the xref node,
@ -236,6 +257,7 @@ class Domain(object):
pass
def resolve_any_xref(self, env, fromdocname, builder, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, nodes.Node, nodes.Node) -> List[Tuple[unicode, nodes.Node]] # NOQA
"""Resolve the pending_xref *node* with the given *target*.
The reference comes from an "any" or similar role, which means that we
@ -252,6 +274,7 @@ class Domain(object):
raise NotImplementedError
def get_objects(self):
# type: () -> Iterable[Tuple[unicode, unicode, unicode, unicode, unicode, int]]
"""Return an iterable of "object descriptions", which are tuples with
five items:
@ -271,6 +294,7 @@ class Domain(object):
return []
def get_type_name(self, type, primary=False):
# type: (ObjType, bool) -> unicode
"""Return full name for given ObjType."""
if primary:
return type.lname

35
sphinx/domains/c.py
View File

@ -22,6 +22,13 @@ from sphinx.directives import ObjectDescription
from sphinx.util.nodes import make_refnode
from sphinx.util.docfields import Field, TypedField
if False:
# For type annotation
from typing import Any, Iterator, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
# RE to split at word boundaries
wsplit_re = re.compile(r'(\W+)')
@ -74,8 +81,9 @@ class CObject(ObjectDescription):
))
def _parse_type(self, node, ctype):
# type: (nodes.Node, unicode) -> None
# add cross-ref nodes for all words
for part in [_f for _f in wsplit_re.split(ctype) if _f]:
for part in [_f for _f in wsplit_re.split(ctype) if _f]: # type: ignore
tnode = nodes.Text(part, part)
if part[0] in string.ascii_letters + '_' and \
part not in self.stopwords:
@ -88,11 +96,12 @@ class CObject(ObjectDescription):
node += tnode
def _parse_arglist(self, arglist):
# type: (unicode) -> Iterator[unicode]
while True:
m = c_funcptr_arg_sig_re.match(arglist)
m = c_funcptr_arg_sig_re.match(arglist) # type: ignore
if m:
yield m.group()
arglist = c_funcptr_arg_sig_re.sub('', arglist)
arglist = c_funcptr_arg_sig_re.sub('', arglist) # type: ignore
if ',' in arglist:
_, arglist = arglist.split(',', 1)
else:
@ -106,11 +115,12 @@ class CObject(ObjectDescription):
break
def handle_signature(self, sig, signode):
# type: (unicode, addnodes.desc_signature) -> unicode
"""Transform a C signature into RST nodes."""
# first try the function pointer signature regex, it's more specific
m = c_funcptr_sig_re.match(sig)
m = c_funcptr_sig_re.match(sig) # type: ignore
if m is None:
m = c_sig_re.match(sig)
m = c_sig_re.match(sig) # type: ignore
if m is None:
raise ValueError('no match')
rettype, name, arglist, const = m.groups()
@ -151,7 +161,7 @@ class CObject(ObjectDescription):
arg = arg.strip()
param = addnodes.desc_parameter('', '', noemph=True)
try:
m = c_funcptr_arg_sig_re.match(arg)
m = c_funcptr_arg_sig_re.match(arg) # type: ignore
if m:
self._parse_type(param, m.group(1) + '(')
param += nodes.emphasis(m.group(2), m.group(2))
@ -173,6 +183,7 @@ class CObject(ObjectDescription):
return fullname
def get_index_text(self, name):
# type: (unicode) -> unicode
if self.objtype == 'function':
return _('%s (C function)') % name
elif self.objtype == 'member':
@ -187,6 +198,7 @@ class CObject(ObjectDescription):
return ''
def add_target_and_index(self, name, sig, signode):
# type: (unicode, unicode, addnodes.desc_signature) -> None
# for C API items we add a prefix since names are usually not qualified
# by a module name and so easily clash with e.g. section titles
targetname = 'c.' + name
@ -209,6 +221,7 @@ class CObject(ObjectDescription):
targetname, '', None))
def before_content(self):
# type: () -> None
self.typename_set = False
if self.name == 'c:type':
if self.names:
@ -216,12 +229,14 @@ class CObject(ObjectDescription):
self.typename_set = True
def after_content(self):
# type: () -> None
if self.typename_set:
self.env.ref_context.pop('c:type', None)
class CXRefRole(XRefRole):
def process_link(self, env, refnode, has_explicit_title, title, target):
# type: (BuildEnvironment, nodes.Node, bool, unicode, unicode) -> Tuple[unicode, unicode] # NOQA
if not has_explicit_title:
target = target.lstrip('~') # only has a meaning for the title
# if the first character is a tilde, don't display the module/class
@ -262,14 +277,16 @@ class CDomain(Domain):
}
initial_data = {
'objects': {}, # fullname -> docname, objtype
}
} # type: Dict[unicode, Dict[unicode, Tuple[unicode, Any]]]
def clear_doc(self, docname):
# type: (unicode) -> None
for fullname, (fn, _l) in list(self.data['objects'].items()):
if fn == docname:
del self.data['objects'][fullname]
def merge_domaindata(self, docnames, otherdata):
# type: (List[unicode], Dict) -> None
# XXX check duplicates
for fullname, (fn, objtype) in otherdata['objects'].items():
if fn in docnames:
@ -277,6 +294,7 @@ class CDomain(Domain):
def resolve_xref(self, env, fromdocname, builder,
typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
# strip pointer asterisk
target = target.rstrip(' *')
# becase TypedField can generate xrefs
@ -290,6 +308,7 @@ class CDomain(Domain):
def resolve_any_xref(self, env, fromdocname, builder, target,
node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, nodes.Node, nodes.Node) -> List[Tuple[unicode, nodes.Node]] # NOQA
# strip pointer asterisk
target = target.rstrip(' *')
if target not in self.data['objects']:
@ -300,11 +319,13 @@ class CDomain(Domain):
contnode, target))]
def get_objects(self):
# type: () -> Iterator[Tuple[unicode, unicode, unicode, unicode, unicode, int]]
for refname, (docname, type) in list(self.data['objects'].items()):
yield (refname, refname, type, docname, 'c.' + refname, 1)
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_domain(CDomain)
return {

684
sphinx/domains/cpp.py
View File

File diff suppressed because it is too large Load Diff

23
sphinx/domains/javascript.py
View File

@ -18,6 +18,14 @@ from sphinx.domains.python import _pseudo_parse_arglist
from sphinx.util.nodes import make_refnode
from sphinx.util.docfields import Field, GroupedField, TypedField
if False:
# For type annotation
from typing import Any, Iterator, Tuple # NOQA
from docutils import nodes # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
class JSObject(ObjectDescription):
"""
@ -28,9 +36,10 @@ class JSObject(ObjectDescription):
has_arguments = False
#: what is displayed right before the documentation entry
display_prefix = None
display_prefix = None # type: unicode
def handle_signature(self, sig, signode):
# type: (unicode, addnodes.desc_signature) -> Tuple[unicode, unicode]
sig = sig.strip()
if '(' in sig and sig[-1:] == ')':
prefix, arglist = sig.split('(', 1)
@ -76,6 +85,7 @@ class JSObject(ObjectDescription):
return fullname, nameprefix
def add_target_and_index(self, name_obj, sig, signode):
# type: (Tuple[unicode, unicode], unicode, addnodes.desc_signature) -> None
objectname = self.options.get(
'object', self.env.ref_context.get('js:object'))
fullname = name_obj[0]
@ -100,6 +110,7 @@ class JSObject(ObjectDescription):
'', None))
def get_index_text(self, objectname, name_obj):
# type: (unicode, Tuple[unicode, unicode]) -> unicode
name, obj = name_obj
if self.objtype == 'function':
if not obj:
@ -139,6 +150,7 @@ class JSConstructor(JSCallable):
class JSXRefRole(XRefRole):
def process_link(self, env, refnode, has_explicit_title, title, target):
# type: (BuildEnvironment, nodes.Node, bool, unicode, unicode) -> Tuple[unicode, unicode] # NOQA
# basically what sphinx.domains.python.PyXRefRole does
refnode['js:object'] = env.ref_context.get('js:object')
if not has_explicit_title:
@ -180,20 +192,23 @@ class JavaScriptDomain(Domain):
}
initial_data = {
'objects': {}, # fullname -> docname, objtype
}
} # type: Dict[unicode, Dict[unicode, Tuple[unicode, unicode]]]
def clear_doc(self, docname):
# type: (unicode) -> None
for fullname, (fn, _l) in list(self.data['objects'].items()):
if fn == docname:
del self.data['objects'][fullname]
def merge_domaindata(self, docnames, otherdata):
# type: (List[unicode], Dict) -> None
# XXX check duplicates
for fullname, (fn, objtype) in otherdata['objects'].items():
if fn in docnames:
self.data['objects'][fullname] = (fn, objtype)
def find_obj(self, env, obj, name, typ, searchorder=0):
# type: (BuildEnvironment, unicode, unicode, unicode, int) -> Tuple[unicode, Tuple[unicode, unicode]] # NOQA
if name[-2:] == '()':
name = name[:-2]
objects = self.data['objects']
@ -212,6 +227,7 @@ class JavaScriptDomain(Domain):
def resolve_xref(self, env, fromdocname, builder, typ, target, node,
contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
objectname = node.get('js:object')
searchorder = node.hasattr('refspecific') and 1 or 0
name, obj = self.find_obj(env, objectname, target, typ, searchorder)
@ -222,6 +238,7 @@ class JavaScriptDomain(Domain):
def resolve_any_xref(self, env, fromdocname, builder, target, node,
contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, nodes.Node, nodes.Node) -> List[Tuple[unicode, nodes.Node]] # NOQA
objectname = node.get('js:object')
name, obj = self.find_obj(env, objectname, target, None, 1)
if not obj:
@ -231,12 +248,14 @@ class JavaScriptDomain(Domain):
name.replace('$', '_S_'), contnode, name))]
def get_objects(self):
# type: () -> Iterator[Tuple[unicode, unicode, unicode, unicode, unicode, int]]
for refname, (docname, type) in list(self.data['objects'].items()):
yield refname, refname, type, docname, \
refname.replace('$', '_S_'), 1
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_domain(JavaScriptDomain)
return {

83
sphinx/domains/python.py
View File

@ -12,18 +12,28 @@
import re
from six import iteritems
from docutils import nodes
from docutils.parsers.rst import directives
from docutils.parsers.rst import Directive, directives
from sphinx import addnodes
from sphinx.roles import XRefRole
from sphinx.locale import l_, _
from sphinx.domains import Domain, ObjType, Index
from sphinx.directives import ObjectDescription
from sphinx.util import logging
from sphinx.util.nodes import make_refnode
from sphinx.util.compat import Directive
from sphinx.util.docfields import Field, GroupedField, TypedField
if False:
# For type annotation
from typing import Any, Iterable, Iterator, Tuple, Union # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
logger = logging.getLogger(__name__)
# REs for Python signatures
py_sig_re = re.compile(
@ -36,6 +46,7 @@ py_sig_re = re.compile(
def _pseudo_parse_arglist(signode, arglist):
# type: (addnodes.desc_signature, unicode) -> None
""""Parse" a list of arguments separated by commas.
Arguments can have "optional" annotations given by enclosing them in
@ -87,7 +98,8 @@ def _pseudo_parse_arglist(signode, arglist):
class PyXrefMixin(object):
def make_xref(self, rolename, domain, target, innernode=nodes.emphasis,
contnode=None):
result = super(PyXrefMixin, self).make_xref(rolename, domain, target,
# type: (unicode, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node
result = super(PyXrefMixin, self).make_xref(rolename, domain, target, # type: ignore
innernode, contnode)
result['refspecific'] = True
if target.startswith(('.', '~')):
@ -103,6 +115,7 @@ class PyXrefMixin(object):
def make_xrefs(self, rolename, domain, target, innernode=nodes.emphasis,
contnode=None):
# type: (unicode, unicode, unicode, nodes.Node, nodes.Node) -> List[nodes.Node]
delims = '(\s*[\[\]\(\),](?:\s*or\s)?\s*|\s+or\s+)'
delims_re = re.compile(delims)
sub_targets = re.split(delims, target)
@ -114,7 +127,7 @@ class PyXrefMixin(object):
if split_contnode:
contnode = nodes.Text(sub_target)
if delims_re.match(sub_target):
if delims_re.match(sub_target): # type: ignore
results.append(contnode or innernode(sub_target, sub_target))
else:
results.append(self.make_xref(rolename, domain, sub_target,
@ -165,18 +178,21 @@ class PyObject(ObjectDescription):
]
def get_signature_prefix(self, sig):
# type: (unicode) -> unicode
"""May return a prefix to put before the object name in the
signature.
"""
return ''
def needs_arglist(self):
# type: () -> bool
"""May return true if an empty argument list is to be generated even if
the document contains none.
"""
return False
def handle_signature(self, sig, signode):
# type: (unicode, addnodes.desc_signature) -> Tuple[unicode, unicode]
"""Transform a Python signature into RST nodes.
Return (fully qualified name of the thing, classname if any).
@ -185,7 +201,7 @@ class PyObject(ObjectDescription):
* it is stripped from the displayed name if present
* it is added to the full name (return value) if not present
"""
m = py_sig_re.match(sig)
m = py_sig_re.match(sig) # type: ignore
if m is None:
raise ValueError
name_prefix, name, arglist, retann = m.groups()
@ -256,10 +272,12 @@ class PyObject(ObjectDescription):
return fullname, name_prefix
def get_index_text(self, modname, name):
# type: (unicode, unicode) -> unicode
"""Return the text for the index entry of the object."""
raise NotImplementedError('must be implemented in subclasses')
def add_target_and_index(self, name_cls, sig, signode):
# type: (unicode, unicode, addnodes.desc_signature) -> None
modname = self.options.get(
'module', self.env.ref_context.get('py:module'))
fullname = (modname and modname + '.' or '') + name_cls[0]
@ -285,10 +303,12 @@ class PyObject(ObjectDescription):
fullname, '', None))
def before_content(self):
# type: () -> None
# needed for automatic qualification of members (reset in subclasses)
self.clsname_set = False
def after_content(self):
# type: () -> None
if self.clsname_set:
self.env.ref_context.pop('py:class', None)
@ -299,9 +319,11 @@ class PyModulelevel(PyObject):
"""
def needs_arglist(self):
# type: () -> bool
return self.objtype == 'function'
def get_index_text(self, modname, name_cls):
# type: (unicode, unicode) -> unicode
if self.objtype == 'function':
if not modname:
return _('%s() (built-in function)') % name_cls[0]
@ -320,9 +342,11 @@ class PyClasslike(PyObject):
"""
def get_signature_prefix(self, sig):
# type: (unicode) -> unicode
return self.objtype + ' '
def get_index_text(self, modname, name_cls):
# type: (unicode, unicode) -> unicode
if self.objtype == 'class':
if not modname:
return _('%s (built-in class)') % name_cls[0]
@ -333,6 +357,7 @@ class PyClasslike(PyObject):
return ''
def before_content(self):
# type: () -> None
PyObject.before_content(self)
if self.names:
self.env.ref_context['py:class'] = self.names[0][0]
@ -345,9 +370,11 @@ class PyClassmember(PyObject):
"""
def needs_arglist(self):
# type: () -> bool
return self.objtype.endswith('method')
def get_signature_prefix(self, sig):
# type: (unicode) -> unicode
if self.objtype == 'staticmethod':
return 'static '
elif self.objtype == 'classmethod':
@ -355,6 +382,7 @@ class PyClassmember(PyObject):
return ''
def get_index_text(self, modname, name_cls):
# type: (unicode, unicode) -> unicode
name, cls = name_cls
add_modules = self.env.config.add_module_names
if self.objtype == 'method':
@ -411,6 +439,7 @@ class PyClassmember(PyObject):
return ''
def before_content(self):
# type: () -> None
PyObject.before_content(self)
lastname = self.names and self.names[-1][1]
if lastname and not self.env.ref_context.get('py:class'):
@ -423,11 +452,13 @@ class PyDecoratorMixin(object):
Mixin for decorator directives.
"""
def handle_signature(self, sig, signode):
ret = super(PyDecoratorMixin, self).handle_signature(sig, signode)
# type: (unicode, addnodes.desc_signature) -> Tuple[unicode, unicode]
ret = super(PyDecoratorMixin, self).handle_signature(sig, signode) # type: ignore
signode.insert(0, addnodes.desc_addname('@', '@'))
return ret
def needs_arglist(self):
# type: () -> bool
return False
@ -436,6 +467,7 @@ class PyDecoratorFunction(PyDecoratorMixin, PyModulelevel):
Directive to mark functions meant to be used as decorators.
"""
def run(self):
# type: () -> List[nodes.Node]
# a decorator function is a function after all
self.name = 'py:function'
return PyModulelevel.run(self)
@ -446,6 +478,7 @@ class PyDecoratorMethod(PyDecoratorMixin, PyClassmember):
Directive to mark methods meant to be used as decorators.
"""
def run(self):
# type: () -> List[nodes.Node]
self.name = 'py:method'
return PyClassmember.run(self)
@ -467,6 +500,7 @@ class PyModule(Directive):
}
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
modname = self.arguments[0].strip()
noindex = 'noindex' in self.options
@ -502,9 +536,10 @@ class PyCurrentModule(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = False
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
modname = self.arguments[0].strip()
if modname == 'None':
@ -516,6 +551,7 @@ class PyCurrentModule(Directive):
class PyXRefRole(XRefRole):
def process_link(self, env, refnode, has_explicit_title, title, target):
# type: (BuildEnvironment, nodes.Node, bool, unicode, unicode) -> Tuple[unicode, unicode] # NOQA
refnode['py:module'] = env.ref_context.get('py:module')
refnode['py:class'] = env.ref_context.get('py:class')
if not has_explicit_title:
@ -546,9 +582,11 @@ class PythonModuleIndex(Index):
shortname = l_('modules')
def generate(self, docnames=None):
content = {}
# type: (Iterable[unicode]) -> Tuple[List[Tuple[unicode, List[List[Union[unicode, int]]]]], bool] # NOQA
content = {} # type: Dict[unicode, List]
# list of prefixes to ignore
ignores = self.domain.env.config['modindex_common_prefix']
ignores = None # type: List[unicode]
ignores = self.domain.env.config['modindex_common_prefix'] # type: ignore
ignores = sorted(ignores, key=len, reverse=True)
# list of all modules, sorted by module name
modules = sorted(iteritems(self.domain.data['modules']),
@ -601,9 +639,9 @@ class PythonModuleIndex(Index):
collapse = len(modules) - num_toplevels < num_toplevels
# sort by first letter
content = sorted(iteritems(content))
sorted_content = sorted(iteritems(content))
return content, collapse
return sorted_content, collapse
class PythonDomain(Domain):
@ -620,7 +658,7 @@ class PythonDomain(Domain):
'staticmethod': ObjType(l_('static method'), 'meth', 'obj'),
'attribute': ObjType(l_('attribute'), 'attr', 'obj'),
'module': ObjType(l_('module'), 'mod', 'obj'),
}
} # type: Dict[unicode, ObjType]
directives = {
'function': PyModulelevel,
@ -650,12 +688,13 @@ class PythonDomain(Domain):
initial_data = {
'objects': {}, # fullname -> docname, objtype
'modules': {}, # modname -> docname, synopsis, platform, deprecated
}
} # type: Dict[unicode, Dict[unicode, Tuple[Any]]]
indices = [
PythonModuleIndex,
]
def clear_doc(self, docname):
# type: (unicode) -> None
for fullname, (fn, _l) in list(self.data['objects'].items()):
if fn == docname:
del self.data['objects'][fullname]
@ -664,6 +703,7 @@ class PythonDomain(Domain):
del self.data['modules'][modname]
def merge_domaindata(self, docnames, otherdata):
# type: (List[unicode], Dict) -> None
# XXX check duplicates?
for fullname, (fn, objtype) in otherdata['objects'].items():
if fn in docnames:
@ -673,6 +713,7 @@ class PythonDomain(Domain):
self.data['modules'][modname] = data
def find_obj(self, env, modname, classname, name, type, searchmode=0):
# type: (BuildEnvironment, unicode, unicode, unicode, unicode, int) -> List[Tuple[unicode, Any]] # NOQA
"""Find a Python object for "name", perhaps using the given module
and/or classname. Returns a list of (name, object entry) tuples.
"""
@ -684,7 +725,7 @@ class PythonDomain(Domain):
return []
objects = self.data['objects']
matches = []
matches = [] # type: List[Tuple[unicode, Any]]
newname = None
if searchmode == 1:
@ -737,6 +778,7 @@ class PythonDomain(Domain):
def resolve_xref(self, env, fromdocname, builder,
type, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
modname = node.get('py:module')
clsname = node.get('py:class')
searchmode = node.hasattr('refspecific') and 1 or 0
@ -745,10 +787,9 @@ class PythonDomain(Domain):
if not matches:
return None
elif len(matches) > 1:
env.warn_node(
'more than one target found for cross-reference '
'%r: %s' % (target, ', '.join(match[0] for match in matches)),
node)
logger.warning('more than one target found for cross-reference %r: %s',
target, ', '.join(match[0] for match in matches),
location=node)
name, obj = matches[0]
if obj[1] == 'module':
@ -760,9 +801,10 @@ class PythonDomain(Domain):
def resolve_any_xref(self, env, fromdocname, builder, target,
node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, nodes.Node, nodes.Node) -> List[Tuple[unicode, nodes.Node]] # NOQA
modname = node.get('py:module')
clsname = node.get('py:class')
results = []
results = [] # type: List[Tuple[unicode, nodes.Node]]
# always search in "refspecific" mode with the :any: role
matches = self.find_obj(env, modname, clsname, target, None, 1)
@ -778,6 +820,7 @@ class PythonDomain(Domain):
return results
def _make_module_refnode(self, builder, fromdocname, name, contnode):
# type: (Builder, unicode, unicode, nodes.Node) -> nodes.Node
# get additional info for modules
docname, synopsis, platform, deprecated = self.data['modules'][name]
title = name
@ -791,6 +834,7 @@ class PythonDomain(Domain):
'module-' + name, contnode, title)
def get_objects(self):
# type: () -> Iterator[Tuple[unicode, unicode, unicode, unicode, unicode, int]]
for modname, info in iteritems(self.data['modules']):
yield (modname, modname, 'module', info[0], 'module-' + modname, 0)
for refname, (docname, type) in iteritems(self.data['objects']):
@ -799,6 +843,7 @@ class PythonDomain(Domain):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_domain(PythonDomain)
return {

23
sphinx/domains/rst.py
View File

@ -20,6 +20,14 @@ from sphinx.directives import ObjectDescription
from sphinx.roles import XRefRole
from sphinx.util.nodes import make_refnode
if False:
# For type annotation
from typing import Any, Iterator, Tuple # NOQA
from docutils import nodes # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
dir_sig_re = re.compile(r'\.\. (.+?)::(.*)$')
@ -30,6 +38,7 @@ class ReSTMarkup(ObjectDescription):
"""
def add_target_and_index(self, name, sig, signode):
# type: (unicode, unicode, addnodes.desc_signature) -> None
targetname = self.objtype + '-' + name
if targetname not in self.state.document.ids:
signode['names'].append(targetname)
@ -51,6 +60,7 @@ class ReSTMarkup(ObjectDescription):
targetname, '', None))
def get_index_text(self, objectname, name):
# type: (unicode, unicode) -> unicode
if self.objtype == 'directive':
return _('%s (directive)') % name
elif self.objtype == 'role':
@ -59,6 +69,7 @@ class ReSTMarkup(ObjectDescription):
def parse_directive(d):
# type: (unicode) -> Tuple[unicode, unicode]
"""Parse a directive signature.
Returns (directive, arguments) string tuple. If no arguments are given,
@ -68,7 +79,7 @@ def parse_directive(d):
if not dir.startswith('.'):
# Assume it is a directive without syntax
return (dir, '')
m = dir_sig_re.match(dir)
m = dir_sig_re.match(dir) # type: ignore
if not m:
return (dir, '')
parsed_dir, parsed_args = m.groups()
@ -80,6 +91,7 @@ class ReSTDirective(ReSTMarkup):
Description of a reST directive.
"""
def handle_signature(self, sig, signode):
# type: (unicode, addnodes.desc_signature) -> unicode
name, args = parse_directive(sig)
desc_name = '.. %s::' % name
signode += addnodes.desc_name(desc_name, desc_name)
@ -93,6 +105,7 @@ class ReSTRole(ReSTMarkup):
Description of a reST role.
"""
def handle_signature(self, sig, signode):
# type: (unicode, addnodes.desc_signature) -> unicode
signode += addnodes.desc_name(':%s:' % sig, ':%s:' % sig)
return sig
@ -116,14 +129,16 @@ class ReSTDomain(Domain):
}
initial_data = {
'objects': {}, # fullname -> docname, objtype
}
} # type: Dict[unicode, Dict[unicode, Tuple[unicode, ObjType]]]
def clear_doc(self, docname):
# type: (unicode) -> None
for (typ, name), doc in list(self.data['objects'].items()):
if doc == docname:
del self.data['objects'][typ, name]
def merge_domaindata(self, docnames, otherdata):
# type: (List[unicode], Dict) -> None
# XXX check duplicates
for (typ, name), doc in otherdata['objects'].items():
if doc in docnames:
@ -131,6 +146,7 @@ class ReSTDomain(Domain):
def resolve_xref(self, env, fromdocname, builder, typ, target, node,
contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
objects = self.data['objects']
objtypes = self.objtypes_for_role(typ)
for objtype in objtypes:
@ -142,6 +158,7 @@ class ReSTDomain(Domain):
def resolve_any_xref(self, env, fromdocname, builder, target,
node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, nodes.Node, nodes.Node) -> List[nodes.Node] # NOQA
objects = self.data['objects']
results = []
for objtype in self.object_types:
@ -154,11 +171,13 @@ class ReSTDomain(Domain):
return results
def get_objects(self):
# type: () -> Iterator[Tuple[unicode, unicode, unicode, unicode, unicode, int]]
for (typ, name), docname in iteritems(self.data['objects']):
yield name, name, typ, docname, typ + '-' + name, 1
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_domain(ReSTDomain)
return {

140
sphinx/domains/std.py
View File

@ -12,9 +12,10 @@
import re
import unicodedata
from six import iteritems
from six import PY3, iteritems
from docutils import nodes
from docutils.parsers.rst import directives
from docutils.parsers.rst import Directive, directives
from docutils.statemachine import ViewList
from sphinx import addnodes
@ -22,9 +23,25 @@ from sphinx.roles import XRefRole
from sphinx.locale import l_, _
from sphinx.domains import Domain, ObjType
from sphinx.directives import ObjectDescription
from sphinx.util import ws_re
from sphinx.util import ws_re, logging, docname_join
from sphinx.util.nodes import clean_astext, make_refnode
from sphinx.util.compat import Directive
if False:
# For type annotation
from typing import Any, Callable, Dict, Iterator, List, Tuple, Type, Union # NOQA
from docutils.parsers.rst.states import Inliner # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
from sphinx.util.typing import Role # NOQA
if PY3:
unicode = str
RoleFunction = Callable[[unicode, unicode, unicode, int, Inliner, Dict, List[unicode]],
Tuple[List[nodes.Node], List[nodes.Node]]]
logger = logging.getLogger(__name__)
# RE for option descriptions
@ -38,9 +55,10 @@ class GenericObject(ObjectDescription):
A generic x-ref directive registered with Sphinx.add_object_type().
"""
indextemplate = ''
parse_node = None
parse_node = None # type: Callable[[GenericObject, BuildEnvironment, unicode, addnodes.desc_signature], unicode] # NOQA
def handle_signature(self, sig, signode):
# type: (unicode, addnodes.desc_signature) -> unicode
if self.parse_node:
name = self.parse_node(self.env, sig, signode)
else:
@ -51,6 +69,7 @@ class GenericObject(ObjectDescription):
return name
def add_target_and_index(self, name, sig, signode):
# type: (unicode, unicode, addnodes.desc_signature) -> None
targetname = '%s-%s' % (self.objtype, name)
signode['ids'].append(targetname)
self.state.document.note_explicit_target(signode)
@ -78,6 +97,7 @@ class EnvVarXRefRole(XRefRole):
"""
def result_nodes(self, document, env, node, is_ref):
# type: (nodes.Node, BuildEnvironment, nodes.Node, bool) -> Tuple[List[nodes.Node], List[nodes.Node]] # NOQA
if not is_ref:
return [node], []
varname = node['reftarget']
@ -102,9 +122,10 @@ class Target(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
# normalize whitespace in fullname like XRefRole does
fullname = ws_re.sub(' ', self.arguments[0].strip())
@ -136,19 +157,18 @@ class Cmdoption(ObjectDescription):
"""
def handle_signature(self, sig, signode):
# type: (unicode, addnodes.desc_signature) -> unicode
"""Transform an option description into RST nodes."""
count = 0
firstname = ''
for potential_option in sig.split(', '):
potential_option = potential_option.strip()
m = option_desc_re.match(potential_option)
m = option_desc_re.match(potential_option) # type: ignore
if not m:
self.env.warn(
self.env.docname,
'Malformed option description %r, should '
'look like "opt", "-opt args", "--opt args", '
'"/opt args" or "+opt args"' % potential_option,
self.lineno)
logger.warning('Malformed option description %r, should '
'look like "opt", "-opt args", "--opt args", '
'"/opt args" or "+opt args"', potential_option,
location=(self.env.docname, self.lineno))
continue
optname, args = m.groups()
if count:
@ -166,6 +186,7 @@ class Cmdoption(ObjectDescription):
return firstname
def add_target_and_index(self, firstname, sig, signode):
# type: (unicode, unicode, addnodes.desc_signature) -> None
currprogram = self.env.ref_context.get('std:program')
for optname in signode.get('allnames', []):
targetname = optname.replace('/', '-')
@ -197,9 +218,10 @@ class Program(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
program = ws_re.sub('-', self.arguments[0].strip())
if program == 'None':
@ -211,17 +233,20 @@ class Program(Directive):
class OptionXRefRole(XRefRole):
def process_link(self, env, refnode, has_explicit_title, title, target):
# type: (BuildEnvironment, nodes.Node, bool, unicode, unicode) -> Tuple[unicode, unicode] # NOQA
refnode['std:program'] = env.ref_context.get('std:program')
return title, target
def split_term_classifiers(line):
# type: (unicode) -> List[Union[unicode, None]]
# split line into a term and classifiers. if no classifier, None is used..
parts = re.split(' +: +', line) + [None]
return parts
def make_glossary_term(env, textnodes, index_key, source, lineno, new_id=None):
# type: (BuildEnvironment, List[nodes.Node], unicode, unicode, int, unicode) -> nodes.term
# get a text-only representation of the term and register it
# as a cross-reference target
term = nodes.term('', '', *textnodes)
@ -265,6 +290,7 @@ class Glossary(Directive):
}
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
node = addnodes.glossary()
node.document = self.state.document
@ -275,7 +301,7 @@ class Glossary(Directive):
# be* a definition list.
# first, collect single entries
entries = []
entries = [] # type: List[Tuple[List[Tuple[unicode, unicode, int]], ViewList]]
in_definition = True
was_empty = True
messages = []
@ -329,7 +355,7 @@ class Glossary(Directive):
for terms, definition in entries:
termtexts = []
termnodes = []
system_messages = []
system_messages = [] # type: List[unicode]
for line, source, lineno in terms:
parts = split_term_classifiers(line)
# parse the term with inline markup
@ -365,9 +391,10 @@ class Glossary(Directive):
def token_xrefs(text):
# type: (unicode) -> List[nodes.Node]
retnodes = []
pos = 0
for m in token_re.finditer(text):
for m in token_re.finditer(text): # type: ignore
if m.start() > pos:
txt = text[pos:m.start()]
retnodes.append(nodes.Text(txt, txt))
@ -390,13 +417,14 @@ class ProductionList(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
env = self.state.document.settings.env
objects = env.domaindata['std']['objects']
node = addnodes.productionlist()
messages = []
messages = [] # type: List[nodes.Node]
i = 0
for rule in self.arguments[0].split('\n'):
@ -437,7 +465,8 @@ class StandardDomain(Domain):
searchprio=-1),
'envvar': ObjType(l_('environment variable'), 'envvar'),
'cmdoption': ObjType(l_('program option'), 'option'),
}
'doc': ObjType(l_('document'), 'doc', searchprio=-1)
} # type: Dict[unicode, ObjType]
directives = {
'program': Program,
@ -446,7 +475,7 @@ class StandardDomain(Domain):
'envvar': EnvVar,
'glossary': Glossary,
'productionlist': ProductionList,
}
} # type: Dict[unicode, Type[Directive]]
roles = {
'option': OptionXRefRole(warn_dangling=True),
'envvar': EnvVarXRefRole(),
@ -463,7 +492,9 @@ class StandardDomain(Domain):
warn_dangling=True),
# links to labels, without a different title
'keyword': XRefRole(warn_dangling=True),
}
# links to documents
'doc': XRefRole(warn_dangling=True, innernodeclass=nodes.inline),
} # type: Dict[unicode, Union[RoleFunction, XRefRole]]
initial_data = {
'progoptions': {}, # (program, name) -> docname, labelid
@ -487,6 +518,7 @@ class StandardDomain(Domain):
'the label must precede a section header)',
'numref': 'undefined label: %(target)s',
'keyword': 'unknown keyword: %(target)s',
'doc': 'unknown document: %(target)s',
'option': 'unknown option: %(target)s',
'citation': 'citation not found: %(target)s',
}
@ -495,9 +527,10 @@ class StandardDomain(Domain):
nodes.figure: ('figure', None),
nodes.table: ('table', None),
nodes.container: ('code-block', None),
}
} # type: Dict[nodes.Node, Tuple[unicode, Callable]]
def clear_doc(self, docname):
# type: (unicode) -> None
for key, (fn, _l) in list(self.data['progoptions'].items()):
if fn == docname:
del self.data['progoptions'][key]
@ -515,6 +548,7 @@ class StandardDomain(Domain):
del self.data['anonlabels'][key]
def merge_domaindata(self, docnames, otherdata):
# type: (List[unicode], Dict) -> None
# XXX duplicates?
for key, data in otherdata['progoptions'].items():
if data[0] in docnames:
@ -533,19 +567,22 @@ class StandardDomain(Domain):
self.data['anonlabels'][key] = data
def process_doc(self, env, docname, document):
# type: (BuildEnvironment, unicode, nodes.Node) -> None
self.note_citations(env, docname, document)
self.note_labels(env, docname, document)
def note_citations(self, env, docname, document):
# type: (BuildEnvironment, unicode, nodes.Node) -> None
for node in document.traverse(nodes.citation):
label = node[0].astext()
if label in self.data['citations']:
path = env.doc2path(self.data['citations'][label][0])
env.warn_node('duplicate citation %s, other instance in %s' %
(label, path), node)
logger.warning('duplicate citation %s, other instance in %s', label, path,
location=node)
self.data['citations'][label] = (docname, node['ids'][0])
def note_labels(self, env, docname, document):
# type: (BuildEnvironment, unicode, nodes.Node) -> None
labels, anonlabels = self.data['labels'], self.data['anonlabels']
for name, explicit in iteritems(document.nametypes):
if not explicit:
@ -563,8 +600,9 @@ class StandardDomain(Domain):
# link and object descriptions
continue
if name in labels:
env.warn_node('duplicate label %s, ' % name + 'other instance '
'in ' + env.doc2path(labels[name][0]), node)
logger.warning('duplicate label %s, ' % name + 'other instance '
'in ' + env.doc2path(labels[name][0]),
location=node)
anonlabels[name] = docname, labelid
if node.tagname in ('section', 'rubric'):
sectname = clean_astext(node[0]) # node[0] == title node
@ -585,6 +623,7 @@ class StandardDomain(Domain):
def build_reference_node(self, fromdocname, builder, docname, labelid,
sectname, rolename, **options):
# type: (unicode, Builder, unicode, unicode, unicode, unicode, Any) -> nodes.Node
nodeclass = options.pop('nodeclass', nodes.reference)
newnode = nodeclass('', '', internal=True, **options)
innernode = nodes.inline(sectname, sectname)
@ -608,12 +647,15 @@ class StandardDomain(Domain):
return newnode
def resolve_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
if typ == 'ref':
resolver = self._resolve_ref_xref
elif typ == 'numref':
resolver = self._resolve_numref_xref
elif typ == 'keyword':
resolver = self._resolve_keyword_xref
elif typ == 'doc':
resolver = self._resolve_doc_xref
elif typ == 'option':
resolver = self._resolve_option_xref
elif typ == 'citation':
@ -624,6 +666,7 @@ class StandardDomain(Domain):
return resolver(env, fromdocname, builder, typ, target, node, contnode)
def _resolve_ref_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
if node['refexplicit']:
# reference to anonymous label; the reference uses
# the supplied link caption
@ -641,6 +684,7 @@ class StandardDomain(Domain):
docname, labelid, sectname, 'ref')
def _resolve_numref_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
if target in self.data['labels']:
docname, labelid, figname = self.data['labels'].get(target, ('', '', ''))
else:
@ -651,7 +695,7 @@ class StandardDomain(Domain):
return None
if env.config.numfig is False:
env.warn_node('numfig is disabled. :numref: is ignored.', node)
logger.warning('numfig is disabled. :numref: is ignored.', location=node)
return contnode
target_node = env.get_doctree(docname).ids.get(labelid)
@ -664,7 +708,8 @@ class StandardDomain(Domain):
if fignumber is None:
return contnode
except ValueError:
env.warn_node("no number is assigned for %s: %s" % (figtype, labelid), node)
logger.warning("no number is assigned for %s: %s", figtype, labelid,
location=node)
return contnode
try:
@ -674,7 +719,7 @@ class StandardDomain(Domain):
title = env.config.numfig_format.get(figtype, '')
if figname is None and '{name}' in title:
env.warn_node('the link has no caption: %s' % title, node)
logger.warning('the link has no caption: %s', title, location=node)
return contnode
else:
fignum = '.'.join(map(str, fignumber))
@ -688,10 +733,10 @@ class StandardDomain(Domain):
# old style format (cf. "Fig.%s")
newtitle = title % fignum
except KeyError as exc:
env.warn_node('invalid numfig_format: %s (%r)' % (title, exc), node)
logger.warning('invalid numfig_format: %s (%r)', title, exc, location=node)
return contnode
except TypeError:
env.warn_node('invalid numfig_format: %s' % title, node)
logger.warning('invalid numfig_format: %s', title, location=node)
return contnode
return self.build_reference_node(fromdocname, builder,
@ -700,6 +745,7 @@ class StandardDomain(Domain):
title=title)
def _resolve_keyword_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
# keywords are oddballs: they are referenced by named labels
docname, labelid, _ = self.data['labels'].get(target, ('', '', ''))
if not docname:
@ -707,7 +753,24 @@ class StandardDomain(Domain):
return make_refnode(builder, fromdocname, docname,
labelid, contnode)
def _resolve_doc_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
# directly reference to document by source name; can be absolute or relative
refdoc = node.get('refdoc', fromdocname)
docname = docname_join(refdoc, node['reftarget'])
if docname not in env.all_docs:
return None
else:
if node['refexplicit']:
# reference with explicit title
caption = node.astext()
else:
caption = clean_astext(env.titles[docname])
innernode = nodes.inline(caption, caption, classes=['doc'])
return make_refnode(builder, fromdocname, docname, None, innernode)
def _resolve_option_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
progname = node.get('std:program')
target = target.strip()
docname, labelid = self.data['progoptions'].get((progname, target), ('', ''))
@ -729,6 +792,7 @@ class StandardDomain(Domain):
labelid, contnode)
def _resolve_citation_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
from sphinx.environment import NoUri
docname, labelid = self.data['citations'].get(target, ('', ''))
@ -751,6 +815,7 @@ class StandardDomain(Domain):
raise
def _resolve_obj_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
objtypes = self.objtypes_for_role(typ) or []
for objtype in objtypes:
if (objtype, target) in self.data['objects']:
@ -764,7 +829,8 @@ class StandardDomain(Domain):
labelid, contnode)
def resolve_any_xref(self, env, fromdocname, builder, target, node, contnode):
results = []
# type: (BuildEnvironment, unicode, Builder, unicode, nodes.Node, nodes.Node) -> List[Tuple[unicode, nodes.Node]] # NOQA
results = [] # type: List[Tuple[unicode, nodes.Node]]
ltarget = target.lower() # :ref: lowercases its target automatically
for role in ('ref', 'option'): # do not try "keyword"
res = self.resolve_xref(env, fromdocname, builder, role,
@ -785,6 +851,7 @@ class StandardDomain(Domain):
return results
def get_objects(self):
# type: () -> Iterator[Tuple[unicode, unicode, unicode, unicode, unicode, int]]
# handle the special 'doc' reference here
for doc in self.env.all_docs:
yield (doc, clean_astext(self.env.titles[doc]), 'doc', doc, '', -1)
@ -802,13 +869,16 @@ class StandardDomain(Domain):
yield (name, name, 'label', info[0], info[1], -1)
def get_type_name(self, type, primary=False):
# type: (ObjType, bool) -> unicode
# never prepend "Default"
return type.lname
def is_enumerable_node(self, node):
# type: (nodes.Node) -> bool
return node.__class__ in self.enumerable_nodes
def get_numfig_title(self, node):
# type: (nodes.Node) -> unicode
"""Get the title of enumerable nodes to refer them using its title"""
if self.is_enumerable_node(node):
_, title_getter = self.enumerable_nodes.get(node.__class__, (None, None))
@ -822,8 +892,10 @@ class StandardDomain(Domain):
return None
def get_figtype(self, node):
# type: (nodes.Node) -> unicode
"""Get figure type of nodes."""
def has_child(node, cls):
# type: (nodes.Node, Type) -> bool
return any(isinstance(child, cls) for child in node)
if isinstance(node, nodes.section):
@ -838,6 +910,7 @@ class StandardDomain(Domain):
return figtype
def get_fignumber(self, env, builder, figtype, docname, target_node):
# type: (BuildEnvironment, Builder, unicode, unicode, nodes.Node) -> Tuple[int, ...]
if figtype == 'section':
if builder.name == 'latex':
return tuple()
@ -861,6 +934,7 @@ class StandardDomain(Domain):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_domain(StandardDomain)
return {

551
sphinx/environment/__init__.py
View File

File diff suppressed because it is too large Load Diff

10
sphinx/environment/adapters/__init__.py Normal file
View File

@ -0,0 +1,10 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.adapters
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sphinx environment adapters
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""

64
sphinx/environment/managers/indexentries.py → sphinx/environment/adapters/indexentries.py
View File

@ -1,9 +1,9 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.managers.indexentries
sphinx.environment.adapters.indexentries
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Index entries manager for sphinx.environment.
Index entries adapters for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
@ -16,51 +16,33 @@ from itertools import groupby
from six import text_type
from sphinx import addnodes
from sphinx.util import iteritems, split_index_msg, split_into
from sphinx.locale import _
from sphinx.environment.managers import EnvironmentManager
from sphinx.util import iteritems, split_into, logging
if False:
# For type annotation
from typing import Any, Pattern, Tuple # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
logger = logging.getLogger(__name__)
class IndexEntries(EnvironmentManager):
name = 'indices'
class IndexEntries(object):
def __init__(self, env):
super(IndexEntries, self).__init__(env)
self.data = env.indexentries
def clear_doc(self, docname):
self.data.pop(docname, None)
def merge_other(self, docnames, other):
for docname in docnames:
self.data[docname] = other.indexentries[docname]
def process_doc(self, docname, doctree):
entries = self.data[docname] = []
for node in doctree.traverse(addnodes.index):
try:
for entry in node['entries']:
split_index_msg(entry[0], entry[1])
except ValueError as exc:
self.env.warn_node(exc, node)
node.parent.remove(node)
else:
for entry in node['entries']:
if len(entry) == 5:
# Since 1.4: new index structure including index_key (5th column)
entries.append(entry)
else:
entries.append(entry + (None,))
# type: (BuildEnvironment) -> None
self.env = env
def create_index(self, builder, group_entries=True,
_fixre=re.compile(r'(.*) ([(][^()]*[)])')):
# type: (Builder, bool, Pattern) -> List[Tuple[unicode, List[Tuple[unicode, Any]]]] # NOQA
"""Create the real index from the collected index entries."""
from sphinx.environment import NoUri
new = {}
new = {} # type: Dict[unicode, List]
def add_entry(word, subword, main, link=True, dic=new, key=None):
# type: (unicode, unicode, unicode, bool, Dict, unicode) -> None
# Force the word to be unicode if it's a ASCII bytestring.
# This will solve problems with unicode normalization later.
# For instance the RFC role will add bytestrings at the moment
@ -79,7 +61,7 @@ class IndexEntries(EnvironmentManager):
# maintain links in sorted/deterministic order
bisect.insort(entry[0], (main, uri))
for fn, entries in iteritems(self.data):
for fn, entries in iteritems(self.env.indexentries):
# new entry types must be listed in directives/other.py!
for type, value, tid, main, index_key in entries:
try:
@ -108,13 +90,14 @@ class IndexEntries(EnvironmentManager):
add_entry(first, _('see also %s') % second, None,
link=False, key=index_key)
else:
self.env.warn(fn, 'unknown index entry type %r' % type)
logger.warning('unknown index entry type %r', type, location=fn)
except ValueError as err:
self.env.warn(fn, str(err))
logger.warning(str(err), location=fn)
# sort the index entries; put all symbols at the front, even those
# following the letters in ASCII, this is where the chr(127) comes from
def keyfunc(entry, lcletters=string.ascii_lowercase + '_'):
# type: (Tuple[unicode, List], unicode) -> Tuple[unicode, unicode]
key, (void, void, category_key) = entry
if category_key:
# using specified category key to sort
@ -135,8 +118,8 @@ class IndexEntries(EnvironmentManager):
# func()
# (in module foo)
# (in module bar)
oldkey = ''
oldsubitems = None
oldkey = '' # type: unicode
oldsubitems = None # type: Dict[unicode, List]
i = 0
while i < len(newlist):
key, (targets, subitems, _key) = newlist[i]
@ -159,6 +142,7 @@ class IndexEntries(EnvironmentManager):
# group the entries by letter
def keyfunc2(item, letters=string.ascii_uppercase + '_'):
# type: (Tuple[unicode, List], unicode) -> unicode
# hack: mutating the subitems dicts to a list in the keyfunc
k, v = item
v[1] = sorted((si, se) for (si, (se, void, void)) in iteritems(v[1]))

325
sphinx/environment/adapters/toctree.py Normal file
View File

@ -0,0 +1,325 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.adapters.toctree
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Toctree adapter for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
from six import iteritems
from docutils import nodes
from sphinx import addnodes
from sphinx.util import url_re, logging
from sphinx.util.nodes import clean_astext, process_only_nodes
if False:
# For type annotation
from typing import Any # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
logger = logging.getLogger(__name__)
class TocTree(object):
def __init__(self, env):
# type: (BuildEnvironment) -> None
self.env = env
def note(self, docname, toctreenode):
# type: (unicode, addnodes.toctree) -> None
"""Note a TOC tree directive in a document and gather information about
file relations from it.
"""
if toctreenode['glob']:
self.env.glob_toctrees.add(docname)
if toctreenode.get('numbered'):
self.env.numbered_toctrees.add(docname)
includefiles = toctreenode['includefiles']
for includefile in includefiles:
# note that if the included file is rebuilt, this one must be
# too (since the TOC of the included file could have changed)
self.env.files_to_rebuild.setdefault(includefile, set()).add(docname)
self.env.toctree_includes.setdefault(docname, []).extend(includefiles)
def resolve(self, docname, builder, toctree, prune=True, maxdepth=0,
titles_only=False, collapse=False, includehidden=False):
# type: (unicode, Builder, addnodes.toctree, bool, int, bool, bool, bool) -> nodes.Node
"""Resolve a *toctree* node into individual bullet lists with titles
as items, returning None (if no containing titles are found) or
a new node.
If *prune* is True, the tree is pruned to *maxdepth*, or if that is 0,
to the value of the *maxdepth* option on the *toctree* node.
If *titles_only* is True, only toplevel document titles will be in the
resulting tree.
If *collapse* is True, all branches not containing docname will
be collapsed.
"""
if toctree.get('hidden', False) and not includehidden:
return None
# For reading the following two helper function, it is useful to keep
# in mind the node structure of a toctree (using HTML-like node names
# for brevity):
#
# <ul>
# <li>
# <p><a></p>
# <p><a></p>
# ...
# <ul>
# ...
# </ul>
# </li>
# </ul>
#
# The transformation is made in two passes in order to avoid
# interactions between marking and pruning the tree (see bug #1046).
toctree_ancestors = self.get_toctree_ancestors(docname)
def _toctree_add_classes(node, depth):
# type: (nodes.Node, int) -> None
"""Add 'toctree-l%d' and 'current' classes to the toctree."""
for subnode in node.children:
if isinstance(subnode, (addnodes.compact_paragraph,
nodes.list_item)):
# for <p> and <li>, indicate the depth level and recurse
subnode['classes'].append('toctree-l%d' % (depth - 1))
_toctree_add_classes(subnode, depth)
elif isinstance(subnode, nodes.bullet_list):
# for <ul>, just recurse
_toctree_add_classes(subnode, depth + 1)
elif isinstance(subnode, nodes.reference):
# for <a>, identify which entries point to the current
# document and therefore may not be collapsed
if subnode['refuri'] == docname:
if not subnode['anchorname']:
# give the whole branch a 'current' class
# (useful for styling it differently)
branchnode = subnode
while branchnode:
branchnode['classes'].append('current')
branchnode = branchnode.parent
# mark the list_item as "on current page"
if subnode.parent.parent.get('iscurrent'):
# but only if it's not already done
return
while subnode:
subnode['iscurrent'] = True
subnode = subnode.parent
def _entries_from_toctree(toctreenode, parents, separate=False, subtree=False):
# type: (addnodes.toctree, List[nodes.Node], bool, bool) -> List[nodes.Node]
"""Return TOC entries for a toctree node."""
refs = [(e[0], e[1]) for e in toctreenode['entries']]
entries = []
for (title, ref) in refs:
try:
refdoc = None
if url_re.match(ref):
if title is None:
title = ref
reference = nodes.reference('', '', internal=False,
refuri=ref, anchorname='',
*[nodes.Text(title)])
para = addnodes.compact_paragraph('', '', reference)
item = nodes.list_item('', para)
toc = nodes.bullet_list('', item)
elif ref == 'self':
# 'self' refers to the document from which this
# toctree originates
ref = toctreenode['parent']
if not title:
title = clean_astext(self.env.titles[ref])
reference = nodes.reference('', '', internal=True,
refuri=ref,
anchorname='',
*[nodes.Text(title)])
para = addnodes.compact_paragraph('', '', reference)
item = nodes.list_item('', para)
# don't show subitems
toc = nodes.bullet_list('', item)
else:
if ref in parents:
logger.warning('circular toctree references '
'detected, ignoring: %s <- %s',
ref, ' <- '.join(parents),
location=ref)
continue
refdoc = ref
toc = self.env.tocs[ref].deepcopy()
maxdepth = self.env.metadata[ref].get('tocdepth', 0)
if ref not in toctree_ancestors or (prune and maxdepth > 0):
self._toctree_prune(toc, 2, maxdepth, collapse)
process_only_nodes(toc, builder.tags)
if title and toc.children and len(toc.children) == 1:
child = toc.children[0]
for refnode in child.traverse(nodes.reference):
if refnode['refuri'] == ref and \
not refnode['anchorname']:
refnode.children = [nodes.Text(title)]
if not toc.children:
# empty toc means: no titles will show up in the toctree
logger.warning('toctree contains reference to document %r that '
'doesn\'t have a title: no link will be generated',
ref, location=toctreenode)
except KeyError:
# this is raised if the included file does not exist
logger.warning('toctree contains reference to nonexisting document %r',
ref, location=toctreenode)
else:
# if titles_only is given, only keep the main title and
# sub-toctrees
if titles_only:
# delete everything but the toplevel title(s)
# and toctrees
for toplevel in toc:
# nodes with length 1 don't have any children anyway
if len(toplevel) > 1:
subtrees = toplevel.traverse(addnodes.toctree)
if subtrees:
toplevel[1][:] = subtrees
else:
toplevel.pop(1)
# resolve all sub-toctrees
for subtocnode in toc.traverse(addnodes.toctree):
if not (subtocnode.get('hidden', False) and
not includehidden):
i = subtocnode.parent.index(subtocnode) + 1
for item in _entries_from_toctree(
subtocnode, [refdoc] + parents,
subtree=True):
subtocnode.parent.insert(i, item)
i += 1
subtocnode.parent.remove(subtocnode)
if separate:
entries.append(toc)
else:
entries.extend(toc.children)
if not subtree and not separate:
ret = nodes.bullet_list()
ret += entries
return [ret]
return entries
maxdepth = maxdepth or toctree.get('maxdepth', -1)
if not titles_only and toctree.get('titlesonly', False):
titles_only = True
if not includehidden and toctree.get('includehidden', False):
includehidden = True
# NOTE: previously, this was separate=True, but that leads to artificial
# separation when two or more toctree entries form a logical unit, so
# separating mode is no longer used -- it's kept here for history's sake
tocentries = _entries_from_toctree(toctree, [], separate=False)
if not tocentries:
return None
newnode = addnodes.compact_paragraph('', '')
caption = toctree.attributes.get('caption')
if caption:
caption_node = nodes.caption(caption, '', *[nodes.Text(caption)])
caption_node.line = toctree.line
caption_node.source = toctree.source
caption_node.rawsource = toctree['rawcaption']
if hasattr(toctree, 'uid'):
# move uid to caption_node to translate it
caption_node.uid = toctree.uid
del toctree.uid
newnode += caption_node
newnode.extend(tocentries)
newnode['toctree'] = True
# prune the tree to maxdepth, also set toc depth and current classes
_toctree_add_classes(newnode, 1)
self._toctree_prune(newnode, 1, prune and maxdepth or 0, collapse)
if len(newnode[-1]) == 0: # No titles found
return None
# set the target paths in the toctrees (they are not known at TOC
# generation time)
for refnode in newnode.traverse(nodes.reference):
if not url_re.match(refnode['refuri']):
refnode['refuri'] = builder.get_relative_uri(
docname, refnode['refuri']) + refnode['anchorname']
return newnode
def get_toctree_ancestors(self, docname):
# type: (unicode) -> List[unicode]
parent = {}
for p, children in iteritems(self.env.toctree_includes):
for child in children:
parent[child] = p
ancestors = [] # type: List[unicode]
d = docname
while d in parent and d not in ancestors:
ancestors.append(d)
d = parent[d]
return ancestors
def _toctree_prune(self, node, depth, maxdepth, collapse=False):
# type: (nodes.Node, int, int, bool) -> None
"""Utility: Cut a TOC at a specified depth."""
for subnode in node.children[:]:
if isinstance(subnode, (addnodes.compact_paragraph,
nodes.list_item)):
# for <p> and <li>, just recurse
self._toctree_prune(subnode, depth, maxdepth, collapse)
elif isinstance(subnode, nodes.bullet_list):
# for <ul>, determine if the depth is too large or if the
# entry is to be collapsed
if maxdepth > 0 and depth > maxdepth:
subnode.parent.replace(subnode, [])
else:
# cull sub-entries whose parents aren't 'current'
if (collapse and depth > 1 and
'iscurrent' not in subnode.parent):
subnode.parent.remove(subnode)
else:
# recurse on visible children
self._toctree_prune(subnode, depth + 1, maxdepth, collapse)
def get_toc_for(self, docname, builder):
# type: (unicode, Builder) -> Dict[unicode, nodes.Node]
"""Return a TOC nodetree -- for use on the same page only!"""
tocdepth = self.env.metadata[docname].get('tocdepth', 0)
try:
toc = self.env.tocs[docname].deepcopy()
self._toctree_prune(toc, 2, tocdepth)
except KeyError:
# the document does not exist anymore: return a dummy node that
# renders to nothing
return nodes.paragraph()
process_only_nodes(toc, builder.tags)
for node in toc.traverse(nodes.reference):
node['refuri'] = node['anchorname'] or '#'
return toc
def get_toctree_for(self, docname, builder, collapse, **kwds):
# type: (unicode, Builder, bool, Any) -> nodes.Node
"""Return the global TOC nodetree."""
doctree = self.env.get_doctree(self.env.config.master_doc)
toctrees = []
if 'includehidden' not in kwds:
kwds['includehidden'] = True
if 'maxdepth' not in kwds:
kwds['maxdepth'] = 0
kwds['collapse'] = collapse
for toctreenode in doctree.traverse(addnodes.toctree):
toctree = self.resolve(docname, builder, toctreenode, prune=True, **kwds)
if toctree:
toctrees.append(toctree)
if not toctrees:
return None
result = toctrees[0]
for toctree in toctrees[1:]:
result.extend(toctree.children)
return result

84
sphinx/environment/collectors/__init__.py Normal file
View File

@ -0,0 +1,84 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.collectors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The data collector components for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
from six import itervalues
if False:
# For type annotation
from docutils import nodes # NOQA
from sphinx.sphinx import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
class EnvironmentCollector(object):
"""An EnvironmentCollector is a specific data collector from each document.
It gathers data and stores :py:class:`BuildEnvironment
<sphinx.environment.BuildEnvironment>` as a database. Examples of specific
data would be images, download files, section titles, metadatas, index
entries and toctrees, etc.
"""
listener_ids = None # type: Dict[unicode, int]
def enable(self, app):
# type: (Sphinx) -> None
assert self.listener_ids is None
self.listener_ids = {
'doctree-read': app.connect('doctree-read', self.process_doc),
'env-merge-info': app.connect('env-merge-info', self.merge_other),
'env-purge-doc': app.connect('env-purge-doc', self.clear_doc),
'env-get-updated': app.connect('env-get-updated', self.get_updated_docs),
'env-get-outdated': app.connect('env-get-outdated', self.get_outdated_docs),
}
def disable(self, app):
# type: (Sphinx) -> None
assert self.listener_ids is not None
for listener_id in itervalues(self.listener_ids):
app.disconnect(listener_id)
self.listener_ids = None
def clear_doc(self, app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
"""Remove specified data of a document.
This method is called on the removal of the document."""
raise NotImplementedError
def merge_other(self, app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Set[unicode], BuildEnvironment) -> None
"""Merge in specified data regarding docnames from a different `BuildEnvironment`
object which coming from a subprocess in parallel builds."""
raise NotImplementedError
def process_doc(self, app, doctree):
# type: (Sphinx, nodes.Node) -> None
"""Process a document and gather specific data from it.
This method is called after the document is read."""
raise NotImplementedError
def get_updated_docs(self, app, env):
# type: (Sphinx, BuildEnvironment) -> List[unicode]
"""Return a list of docnames to re-read.
This methods is called after reading the whole of documents (experimental).
"""
return []
def get_outdated_docs(self, app, env, added, changed, removed):
# type: (Sphinx, BuildEnvironment, unicode, Set[unicode], Set[unicode], Set[unicode]) -> List[unicode] # NOQA
"""Return a list of docnames to re-read.
This methods is called before reading the documents.
"""
return []

148
sphinx/environment/collectors/asset.py Normal file
View File

@ -0,0 +1,148 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.collectors.asset
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The image collector for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
import os
from os import path
from glob import glob
from six import iteritems, itervalues
from docutils import nodes
from docutils.utils import relative_path
from sphinx import addnodes
from sphinx.environment.collectors import EnvironmentCollector
from sphinx.util import logging
from sphinx.util.i18n import get_image_filename_for_language, search_image_for_language
from sphinx.util.images import guess_mimetype
if False:
# For type annotation
from typing import Tuple # NOQA
from docutils import nodes # NOQA
from sphinx.sphinx import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
logger = logging.getLogger(__name__)
class ImageCollector(EnvironmentCollector):
"""Image files collector for sphinx.environment."""
def clear_doc(self, app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
env.images.purge_doc(docname)
def merge_other(self, app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Set[unicode], BuildEnvironment) -> None
env.images.merge_other(docnames, other.images)
def process_doc(self, app, doctree):
# type: (Sphinx, nodes.Node) -> None
"""Process and rewrite image URIs."""
docname = app.env.docname
for node in doctree.traverse(nodes.image):
# Map the mimetype to the corresponding image. The writer may
# choose the best image from these candidates. The special key * is
# set if there is only single candidate to be used by a writer.
# The special key ? is set for nonlocal URIs.
candidates = {} # type: Dict[unicode, unicode]
node['candidates'] = candidates
imguri = node['uri']
if imguri.startswith('data:'):
logger.warning('image data URI found. some builders might not support',
location=node, type='image', subtype='data_uri')
candidates['?'] = imguri
continue
elif imguri.find('://') != -1:
logger.warning('nonlocal image URI found: %s' % imguri,
location=node,
type='image', subtype='nonlocal_uri')
candidates['?'] = imguri
continue
rel_imgpath, full_imgpath = app.env.relfn2path(imguri, docname)
if app.config.language:
# substitute figures (ex. foo.png -> foo.en.png)
i18n_full_imgpath = search_image_for_language(full_imgpath, app.env)
if i18n_full_imgpath != full_imgpath:
full_imgpath = i18n_full_imgpath
rel_imgpath = relative_path(path.join(app.srcdir, 'dummy'),
i18n_full_imgpath)
# set imgpath as default URI
node['uri'] = rel_imgpath
if rel_imgpath.endswith(os.extsep + '*'):
if app.config.language:
# Search language-specific figures at first
i18n_imguri = get_image_filename_for_language(imguri, app.env)
_, full_i18n_imgpath = app.env.relfn2path(i18n_imguri, docname)
self.collect_candidates(app.env, full_i18n_imgpath, candidates, node)
self.collect_candidates(app.env, full_imgpath, candidates, node)
else:
candidates['*'] = rel_imgpath
# map image paths to unique image names (so that they can be put
# into a single directory)
for imgpath in itervalues(candidates):
app.env.dependencies[docname].add(imgpath)
if not os.access(path.join(app.srcdir, imgpath), os.R_OK):
logger.warning('image file not readable: %s' % imgpath,
location=node, type='image', subtype='not_readable')
continue
app.env.images.add_file(docname, imgpath)
def collect_candidates(self, env, imgpath, candidates, node):
# type: (BuildEnvironment, unicode, Dict[unicode, unicode], nodes.Node) -> None
globbed = {} # type: Dict[unicode, List[unicode]]
for filename in glob(imgpath):
new_imgpath = relative_path(path.join(env.srcdir, 'dummy'),
filename)
try:
mimetype = guess_mimetype(filename)
if mimetype not in candidates:
globbed.setdefault(mimetype, []).append(new_imgpath)
except (OSError, IOError) as err:
logger.warning('image file %s not readable: %s' % (filename, err),
location=node, type='image', subtype='not_readable')
for key, files in iteritems(globbed):
candidates[key] = sorted(files, key=len)[0] # select by similarity
class DownloadFileCollector(EnvironmentCollector):
"""Download files collector for sphinx.environment."""
def clear_doc(self, app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
env.dlfiles.purge_doc(docname)
def merge_other(self, app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Set[unicode], BuildEnvironment) -> None
env.dlfiles.merge_other(docnames, other.dlfiles)
def process_doc(self, app, doctree):
# type: (Sphinx, nodes.Node) -> None
"""Process downloadable file paths. """
for node in doctree.traverse(addnodes.download_reference):
targetname = node['reftarget']
rel_filename, filename = app.env.relfn2path(targetname, app.env.docname)
app.env.dependencies[app.env.docname].add(rel_filename)
if not os.access(filename, os.R_OK):
logger.warning('download file not readable: %s' % filename,
location=node, type='download', subtype='not_readable')
continue
node['filename'] = app.env.dlfiles.add_file(app.env.docname, filename)
def setup(app):
# type: (Sphinx) -> None
app.add_env_collector(ImageCollector)
app.add_env_collector(DownloadFileCollector)

59
sphinx/environment/collectors/dependencies.py Normal file
View File

@ -0,0 +1,59 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.collectors.dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The dependencies collector components for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
from os import path
from docutils.utils import relative_path
from sphinx.util.osutil import getcwd, fs_encoding
from sphinx.environment.collectors import EnvironmentCollector
if False:
# For type annotation
from docutils import nodes # NOQA
from sphinx.sphinx import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
class DependenciesCollector(EnvironmentCollector):
"""dependencies collector for sphinx.environment."""
def clear_doc(self, app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
env.dependencies.pop(docname, None)
def merge_other(self, app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Set[unicode], BuildEnvironment) -> None
for docname in docnames:
if docname in other.dependencies:
env.dependencies[docname] = other.dependencies[docname]
def process_doc(self, app, doctree):
# type: (Sphinx, nodes.Node) -> None
"""Process docutils-generated dependency info."""
cwd = getcwd()
frompath = path.join(path.normpath(app.srcdir), 'dummy')
deps = doctree.settings.record_dependencies
if not deps:
return
for dep in deps.list:
# the dependency path is relative to the working dir, so get
# one relative to the srcdir
if isinstance(dep, bytes):
dep = dep.decode(fs_encoding)
relpath = relative_path(frompath,
path.normpath(path.join(cwd, dep)))
app.env.dependencies[app.env.docname].add(relpath)
def setup(app):
# type: (Sphinx) -> None
app.add_env_collector(DependenciesCollector)

59
sphinx/environment/collectors/indexentries.py Normal file
View File

@ -0,0 +1,59 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.collectors.indexentries
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Index entries collector for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
from sphinx import addnodes
from sphinx.util import split_index_msg, logging
from sphinx.environment.collectors import EnvironmentCollector
if False:
# For type annotation
from docutils import nodes # NOQA
from sphinx.applicatin import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
logger = logging.getLogger(__name__)
class IndexEntriesCollector(EnvironmentCollector):
name = 'indices'
def clear_doc(self, app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
env.indexentries.pop(docname, None)
def merge_other(self, app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Set[unicode], BuildEnvironment) -> None
for docname in docnames:
env.indexentries[docname] = other.indexentries[docname]
def process_doc(self, app, doctree):
# type: (Sphinx, nodes.Node) -> None
docname = app.env.docname
entries = app.env.indexentries[docname] = []
for node in doctree.traverse(addnodes.index):
try:
for entry in node['entries']:
split_index_msg(entry[0], entry[1])
except ValueError as exc:
logger.warning(str(exc), location=node)
node.parent.remove(node)
else:
for entry in node['entries']:
if len(entry) == 5:
# Since 1.4: new index structure including index_key (5th column)
entries.append(entry)
else:
entries.append(entry + (None,))
def setup(app):
# type: (Sphinx) -> None
app.add_env_collector(IndexEntriesCollector)

72
sphinx/environment/collectors/metadata.py Normal file
View File

@ -0,0 +1,72 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.collectors.metadata
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The metadata collector components for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
from docutils import nodes
from sphinx.environment.collectors import EnvironmentCollector
if False:
# For type annotation
from docutils import nodes # NOQA
from sphinx.sphinx import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
class MetadataCollector(EnvironmentCollector):
"""metadata collector for sphinx.environment."""
def clear_doc(self, app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
env.metadata.pop(docname, None)
def merge_other(self, app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Set[unicode], BuildEnvironment) -> None
for docname in docnames:
env.metadata[docname] = other.metadata[docname]
def process_doc(self, app, doctree):
# type: (Sphinx, nodes.Node) -> None
"""Process the docinfo part of the doctree as metadata.
Keep processing minimal -- just return what docutils says.
"""
md = app.env.metadata[app.env.docname]
try:
docinfo = doctree[0]
except IndexError:
# probably an empty document
return
if docinfo.__class__ is not nodes.docinfo:
# nothing to see here
return
for node in docinfo:
# nodes are multiply inherited...
if isinstance(node, nodes.authors):
md['authors'] = [author.astext() for author in node]
elif isinstance(node, nodes.TextElement): # e.g. author
md[node.__class__.__name__] = node.astext()
else:
name, body = node
md[name.astext()] = body.astext()
for name, value in md.items():
if name in ('tocdepth',):
try:
value = int(value)
except ValueError:
value = 0
md[name] = value
del doctree[0]
def setup(app):
# type: (Sphinx) -> None
app.add_env_collector(MetadataCollector)

65
sphinx/environment/collectors/title.py Normal file
View File

@ -0,0 +1,65 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.collectors.title
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The title collector components for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
from docutils import nodes
from sphinx.environment.collectors import EnvironmentCollector
from sphinx.transforms import SphinxContentsFilter
if False:
# For type annotation
from docutils import nodes # NOQA
from sphinx.sphinx import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
class TitleCollector(EnvironmentCollector):
"""title collector for sphinx.environment."""
def clear_doc(self, app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
env.titles.pop(docname, None)
env.longtitles.pop(docname, None)
def merge_other(self, app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Set[unicode], BuildEnvironment) -> None
for docname in docnames:
env.titles[docname] = other.titles[docname]
env.longtitles[docname] = other.longtitles[docname]
def process_doc(self, app, doctree):
# type: (Sphinx, nodes.Node) -> None
"""Add a title node to the document (just copy the first section title),
and store that title in the environment.
"""
titlenode = nodes.title()
longtitlenode = titlenode
# explicit title set with title directive; use this only for
# the <title> tag in HTML output
if 'title' in doctree:
longtitlenode = nodes.title()
longtitlenode += nodes.Text(doctree['title'])
# look for first section title and use that as the title
for node in doctree.traverse(nodes.section):
visitor = SphinxContentsFilter(doctree)
node[0].walkabout(visitor)
titlenode += visitor.get_entry_text()
break
else:
# document has no title
titlenode += nodes.Text('<no title>')
app.env.titles[app.env.docname] = titlenode
app.env.longtitles[app.env.docname] = longtitlenode
def setup(app):
# type: (Sphinx) -> None
app.add_env_collector(TitleCollector)

288
sphinx/environment/collectors/toctree.py Normal file
View File

@ -0,0 +1,288 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.collectors.toctree
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Toctree collector for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
from six import iteritems
from docutils import nodes
from sphinx import addnodes
from sphinx.util import url_re, logging
from sphinx.transforms import SphinxContentsFilter
from sphinx.environment.adapters.toctree import TocTree
from sphinx.environment.collectors import EnvironmentCollector
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
logger = logging.getLogger(__name__)
class TocTreeCollector(EnvironmentCollector):
def clear_doc(self, app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
env.tocs.pop(docname, None)
env.toc_secnumbers.pop(docname, None)
env.toc_fignumbers.pop(docname, None)
env.toc_num_entries.pop(docname, None)
env.toctree_includes.pop(docname, None)
env.glob_toctrees.discard(docname)
env.numbered_toctrees.discard(docname)
for subfn, fnset in list(env.files_to_rebuild.items()):
fnset.discard(docname)
if not fnset:
del env.files_to_rebuild[subfn]
def merge_other(self, app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Set[unicode], BuildEnvironment) -> None
for docname in docnames:
env.tocs[docname] = other.tocs[docname]
env.toc_num_entries[docname] = other.toc_num_entries[docname]
if docname in other.toctree_includes:
env.toctree_includes[docname] = other.toctree_includes[docname]
if docname in other.glob_toctrees:
env.glob_toctrees.add(docname)
if docname in other.numbered_toctrees:
env.numbered_toctrees.add(docname)
for subfn, fnset in other.files_to_rebuild.items():
env.files_to_rebuild.setdefault(subfn, set()).update(fnset & set(docnames))
def process_doc(self, app, doctree):
# type: (Sphinx, nodes.Node) -> None
"""Build a TOC from the doctree and store it in the inventory."""
docname = app.env.docname
numentries = [0] # nonlocal again...
def traverse_in_section(node, cls):
"""Like traverse(), but stay within the same section."""
result = []
if isinstance(node, cls):
result.append(node)
for child in node.children:
if isinstance(child, nodes.section):
continue
result.extend(traverse_in_section(child, cls))
return result
def build_toc(node, depth=1):
entries = []
for sectionnode in node:
# find all toctree nodes in this section and add them
# to the toc (just copying the toctree node which is then
# resolved in self.get_and_resolve_doctree)
if isinstance(sectionnode, addnodes.only):
onlynode = addnodes.only(expr=sectionnode['expr'])
blist = build_toc(sectionnode, depth)
if blist:
onlynode += blist.children
entries.append(onlynode)
continue
if not isinstance(sectionnode, nodes.section):
for toctreenode in traverse_in_section(sectionnode,
addnodes.toctree):
item = toctreenode.copy()
entries.append(item)
# important: do the inventory stuff
TocTree(app.env).note(docname, toctreenode)
continue
title = sectionnode[0]
# copy the contents of the section title, but without references
# and unnecessary stuff
visitor = SphinxContentsFilter(doctree)
title.walkabout(visitor)
nodetext = visitor.get_entry_text()
if not numentries[0]:
# for the very first toc entry, don't add an anchor
# as it is the file's title anyway
anchorname = ''
else:
anchorname = '#' + sectionnode['ids'][0]
numentries[0] += 1
# make these nodes:
# list_item -> compact_paragraph -> reference
reference = nodes.reference(
'', '', internal=True, refuri=docname,
anchorname=anchorname, *nodetext)
para = addnodes.compact_paragraph('', '', reference)
item = nodes.list_item('', para)
sub_item = build_toc(sectionnode, depth + 1)
item += sub_item
entries.append(item)
if entries:
return nodes.bullet_list('', *entries)
return []
toc = build_toc(doctree)
if toc:
app.env.tocs[docname] = toc
else:
app.env.tocs[docname] = nodes.bullet_list('')
app.env.toc_num_entries[docname] = numentries[0]
def get_updated_docs(self, app, env):
# type: (Sphinx, BuildEnvironment) -> List[unicode]
return self.assign_section_numbers(env) + self.assign_figure_numbers(env)
def assign_section_numbers(self, env):
# type: (BuildEnvironment) -> List[unicode]
"""Assign a section number to each heading under a numbered toctree."""
# a list of all docnames whose section numbers changed
rewrite_needed = []
assigned = set() # type: Set[unicode]
old_secnumbers = env.toc_secnumbers
env.toc_secnumbers = {}
def _walk_toc(node, secnums, depth, titlenode=None):
# titlenode is the title of the document, it will get assigned a
# secnumber too, so that it shows up in next/prev/parent rellinks
for subnode in node.children:
if isinstance(subnode, nodes.bullet_list):
numstack.append(0)
_walk_toc(subnode, secnums, depth - 1, titlenode)
numstack.pop()
titlenode = None
elif isinstance(subnode, nodes.list_item):
_walk_toc(subnode, secnums, depth, titlenode)
titlenode = None
elif isinstance(subnode, addnodes.only):
# at this stage we don't know yet which sections are going
# to be included; just include all of them, even if it leads
# to gaps in the numbering
_walk_toc(subnode, secnums, depth, titlenode)
titlenode = None
elif isinstance(subnode, addnodes.compact_paragraph):
numstack[-1] += 1
if depth > 0:
number = tuple(numstack)
else:
number = None
secnums[subnode[0]['anchorname']] = \
subnode[0]['secnumber'] = number
if titlenode:
titlenode['secnumber'] = number
titlenode = None
elif isinstance(subnode, addnodes.toctree):
_walk_toctree(subnode, depth)
def _walk_toctree(toctreenode, depth):
if depth == 0:
return
for (title, ref) in toctreenode['entries']:
if url_re.match(ref) or ref == 'self':
# don't mess with those
continue
elif ref in assigned:
logger.warning('%s is already assigned section numbers '
'(nested numbered toctree?)', ref,
location=toctreenode, type='toc', subtype='secnum')
elif ref in env.tocs:
secnums = env.toc_secnumbers[ref] = {}
assigned.add(ref)
_walk_toc(env.tocs[ref], secnums, depth,
env.titles.get(ref))
if secnums != old_secnumbers.get(ref):
rewrite_needed.append(ref)
for docname in env.numbered_toctrees:
assigned.add(docname)
doctree = env.get_doctree(docname)
for toctreenode in doctree.traverse(addnodes.toctree):
depth = toctreenode.get('numbered', 0)
if depth:
# every numbered toctree gets new numbering
numstack = [0]
_walk_toctree(toctreenode, depth)
return rewrite_needed
def assign_figure_numbers(self, env):
# type: (BuildEnvironment) -> List[unicode]
"""Assign a figure number to each figure under a numbered toctree."""
rewrite_needed = []
assigned = set() # type: Set[unicode]
old_fignumbers = env.toc_fignumbers
env.toc_fignumbers = {}
fignum_counter = {} # type: Dict[unicode, Dict[Tuple[int], int]]
def get_section_number(docname, section):
anchorname = '#' + section['ids'][0]
secnumbers = env.toc_secnumbers.get(docname, {})
if anchorname in secnumbers:
secnum = secnumbers.get(anchorname)
else:
secnum = secnumbers.get('')
return secnum or tuple()
def get_next_fignumber(figtype, secnum):
counter = fignum_counter.setdefault(figtype, {})
secnum = secnum[:env.config.numfig_secnum_depth]
counter[secnum] = counter.get(secnum, 0) + 1
return secnum + (counter[secnum],)
def register_fignumber(docname, secnum, figtype, fignode):
env.toc_fignumbers.setdefault(docname, {})
fignumbers = env.toc_fignumbers[docname].setdefault(figtype, {})
figure_id = fignode['ids'][0]
fignumbers[figure_id] = get_next_fignumber(figtype, secnum)
def _walk_doctree(docname, doctree, secnum):
for subnode in doctree.children:
if isinstance(subnode, nodes.section):
next_secnum = get_section_number(docname, subnode)
if next_secnum:
_walk_doctree(docname, subnode, next_secnum)
else:
_walk_doctree(docname, subnode, secnum)
continue
elif isinstance(subnode, addnodes.toctree):
for title, subdocname in subnode['entries']:
if url_re.match(subdocname) or subdocname == 'self':
# don't mess with those
continue
_walk_doc(subdocname, secnum)
continue
figtype = env.get_domain('std').get_figtype(subnode) # type: ignore
if figtype and subnode['ids']:
register_fignumber(docname, secnum, figtype, subnode)
_walk_doctree(docname, subnode, secnum)
def _walk_doc(docname, secnum):
if docname not in assigned:
assigned.add(docname)
doctree = env.get_doctree(docname)
_walk_doctree(docname, doctree, secnum)
if env.config.numfig:
_walk_doc(env.config.master_doc, tuple())
for docname, fignums in iteritems(env.toc_fignumbers):
if fignums != old_fignumbers.get(docname):
rewrite_needed.append(docname)
return rewrite_needed
def setup(app):
# type: (Sphinx) -> None
app.add_env_collector(TocTreeCollector)

37
sphinx/environment/managers/__init__.py
View File

@ -1,37 +0,0 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.managers
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Manager components for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
class EnvironmentManager(object):
"""Base class for sphinx.environment managers."""
name = None
def __init__(self, env):
self.env = env
def attach(self, env):
self.env = env
if self.name:
setattr(env, self.name, self)
def detach(self, env):
self.env = None
if self.name:
delattr(env, self.name)
def clear_doc(self, docname):
raise NotImplementedError
def merge_other(self, docnames, other):
raise NotImplementedError
def process_doc(self, docname, doctree):
raise NotImplementedError

565
sphinx/environment/managers/toctree.py
View File

@ -1,565 +0,0 @@
# -*- coding: utf-8 -*-
"""
sphinx.environment.managers.toctree
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Toctree manager for sphinx.environment.
:copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""
from six import iteritems
from docutils import nodes
from sphinx import addnodes
from sphinx.util import url_re
from sphinx.util.nodes import clean_astext, process_only_nodes
from sphinx.transforms import SphinxContentsFilter
from sphinx.environment.managers import EnvironmentManager
class Toctree(EnvironmentManager):
name = 'toctree'
def __init__(self, env):
super(Toctree, self).__init__(env)
self.tocs = env.tocs
self.toc_num_entries = env.toc_num_entries
self.toc_secnumbers = env.toc_secnumbers
self.toc_fignumbers = env.toc_fignumbers
self.toctree_includes = env.toctree_includes
self.files_to_rebuild = env.files_to_rebuild
self.glob_toctrees = env.glob_toctrees
self.numbered_toctrees = env.numbered_toctrees
def clear_doc(self, docname):
self.tocs.pop(docname, None)
self.toc_secnumbers.pop(docname, None)
self.toc_fignumbers.pop(docname, None)
self.toc_num_entries.pop(docname, None)
self.toctree_includes.pop(docname, None)
self.glob_toctrees.discard(docname)
self.numbered_toctrees.discard(docname)
for subfn, fnset in list(self.files_to_rebuild.items()):
fnset.discard(docname)
if not fnset:
del self.files_to_rebuild[subfn]
def merge_other(self, docnames, other):
for docname in docnames:
self.tocs[docname] = other.tocs[docname]
self.toc_num_entries[docname] = other.toc_num_entries[docname]
if docname in other.toctree_includes:
self.toctree_includes[docname] = other.toctree_includes[docname]
if docname in other.glob_toctrees:
self.glob_toctrees.add(docname)
if docname in other.numbered_toctrees:
self.numbered_toctrees.add(docname)
for subfn, fnset in other.files_to_rebuild.items():
self.files_to_rebuild.setdefault(subfn, set()).update(fnset & docnames)
def process_doc(self, docname, doctree):
"""Build a TOC from the doctree and store it in the inventory."""
numentries = [0] # nonlocal again...
def traverse_in_section(node, cls):
"""Like traverse(), but stay within the same section."""
result = []
if isinstance(node, cls):
result.append(node)
for child in node.children:
if isinstance(child, nodes.section):
continue
result.extend(traverse_in_section(child, cls))
return result
def build_toc(node, depth=1):
entries = []
for sectionnode in node:
# find all toctree nodes in this section and add them
# to the toc (just copying the toctree node which is then
# resolved in self.get_and_resolve_doctree)
if isinstance(sectionnode, addnodes.only):
onlynode = addnodes.only(expr=sectionnode['expr'])
blist = build_toc(sectionnode, depth)
if blist:
onlynode += blist.children
entries.append(onlynode)
continue
if not isinstance(sectionnode, nodes.section):
for toctreenode in traverse_in_section(sectionnode,
addnodes.toctree):
item = toctreenode.copy()
entries.append(item)
# important: do the inventory stuff
self.note_toctree(docname, toctreenode)
continue
title = sectionnode[0]
# copy the contents of the section title, but without references
# and unnecessary stuff
visitor = SphinxContentsFilter(doctree)
title.walkabout(visitor)
nodetext = visitor.get_entry_text()
if not numentries[0]:
# for the very first toc entry, don't add an anchor
# as it is the file's title anyway
anchorname = ''
else:
anchorname = '#' + sectionnode['ids'][0]
numentries[0] += 1
# make these nodes:
# list_item -> compact_paragraph -> reference
reference = nodes.reference(
'', '', internal=True, refuri=docname,
anchorname=anchorname, *nodetext)
para = addnodes.compact_paragraph('', '', reference)
item = nodes.list_item('', para)
sub_item = build_toc(sectionnode, depth + 1)
item += sub_item
entries.append(item)
if entries:
return nodes.bullet_list('', *entries)
return []
toc = build_toc(doctree)
if toc:
self.tocs[docname] = toc
else:
self.tocs[docname] = nodes.bullet_list('')
self.toc_num_entries[docname] = numentries[0]
def note_toctree(self, docname, toctreenode):
"""Note a TOC tree directive in a document and gather information about
file relations from it.
"""
if toctreenode['glob']:
self.glob_toctrees.add(docname)
if toctreenode.get('numbered'):
self.numbered_toctrees.add(docname)
includefiles = toctreenode['includefiles']
for includefile in includefiles:
# note that if the included file is rebuilt, this one must be
# too (since the TOC of the included file could have changed)
self.files_to_rebuild.setdefault(includefile, set()).add(docname)
self.toctree_includes.setdefault(docname, []).extend(includefiles)
def get_toc_for(self, docname, builder):
"""Return a TOC nodetree -- for use on the same page only!"""
tocdepth = self.env.metadata[docname].get('tocdepth', 0)
try:
toc = self.tocs[docname].deepcopy()
self._toctree_prune(toc, 2, tocdepth)
except KeyError:
# the document does not exist anymore: return a dummy node that
# renders to nothing
return nodes.paragraph()
process_only_nodes(toc, builder.tags, warn_node=self.env.warn_node)
for node in toc.traverse(nodes.reference):
node['refuri'] = node['anchorname'] or '#'
return toc
def get_toctree_for(self, docname, builder, collapse, **kwds):
"""Return the global TOC nodetree."""
doctree = self.env.get_doctree(self.env.config.master_doc)
toctrees = []
if 'includehidden' not in kwds:
kwds['includehidden'] = True
if 'maxdepth' not in kwds:
kwds['maxdepth'] = 0
kwds['collapse'] = collapse
for toctreenode in doctree.traverse(addnodes.toctree):
toctree = self.env.resolve_toctree(docname, builder, toctreenode,
prune=True, **kwds)
if toctree:
toctrees.append(toctree)
if not toctrees:
return None
result = toctrees[0]
for toctree in toctrees[1:]:
result.extend(toctree.children)
return result
def resolve_toctree(self, docname, builder, toctree, prune=True, maxdepth=0,
titles_only=False, collapse=False, includehidden=False):
"""Resolve a *toctree* node into individual bullet lists with titles
as items, returning None (if no containing titles are found) or
a new node.
If *prune* is True, the tree is pruned to *maxdepth*, or if that is 0,
to the value of the *maxdepth* option on the *toctree* node.
If *titles_only* is True, only toplevel document titles will be in the
resulting tree.
If *collapse* is True, all branches not containing docname will
be collapsed.
"""
if toctree.get('hidden', False) and not includehidden:
return None
# For reading the following two helper function, it is useful to keep
# in mind the node structure of a toctree (using HTML-like node names
# for brevity):
#
# <ul>
# <li>
# <p><a></p>
# <p><a></p>
# ...
# <ul>
# ...
# </ul>
# </li>
# </ul>
#
# The transformation is made in two passes in order to avoid
# interactions between marking and pruning the tree (see bug #1046).
toctree_ancestors = self.get_toctree_ancestors(docname)
def _toctree_add_classes(node, depth):
"""Add 'toctree-l%d' and 'current' classes to the toctree."""
for subnode in node.children:
if isinstance(subnode, (addnodes.compact_paragraph,
nodes.list_item)):
# for <p> and <li>, indicate the depth level and recurse
subnode['classes'].append('toctree-l%d' % (depth - 1))
_toctree_add_classes(subnode, depth)
elif isinstance(subnode, nodes.bullet_list):
# for <ul>, just recurse
_toctree_add_classes(subnode, depth + 1)
elif isinstance(subnode, nodes.reference):
# for <a>, identify which entries point to the current
# document and therefore may not be collapsed
if subnode['refuri'] == docname:
if not subnode['anchorname']:
# give the whole branch a 'current' class
# (useful for styling it differently)
branchnode = subnode
while branchnode:
branchnode['classes'].append('current')
branchnode = branchnode.parent
# mark the list_item as "on current page"
if subnode.parent.parent.get('iscurrent'):
# but only if it's not already done
return
while subnode:
subnode['iscurrent'] = True
subnode = subnode.parent
def _entries_from_toctree(toctreenode, parents,
separate=False, subtree=False):
"""Return TOC entries for a toctree node."""
refs = [(e[0], e[1]) for e in toctreenode['entries']]
entries = []
for (title, ref) in refs:
try:
refdoc = None
if url_re.match(ref):
if title is None:
title = ref
reference = nodes.reference('', '', internal=False,
refuri=ref, anchorname='',
*[nodes.Text(title)])
para = addnodes.compact_paragraph('', '', reference)
item = nodes.list_item('', para)
toc = nodes.bullet_list('', item)
elif ref == 'self':
# 'self' refers to the document from which this
# toctree originates
ref = toctreenode['parent']
if not title:
title = clean_astext(self.env.titles[ref])
reference = nodes.reference('', '', internal=True,
refuri=ref,
anchorname='',
*[nodes.Text(title)])
para = addnodes.compact_paragraph('', '', reference)
item = nodes.list_item('', para)
# don't show subitems
toc = nodes.bullet_list('', item)
else:
if ref in parents:
self.env.warn(ref, 'circular toctree references '
'detected, ignoring: %s <- %s' %
(ref, ' <- '.join(parents)))
continue
refdoc = ref
toc = self.tocs[ref].deepcopy()
maxdepth = self.env.metadata[ref].get('tocdepth', 0)
if ref not in toctree_ancestors or (prune and maxdepth > 0):
self._toctree_prune(toc, 2, maxdepth, collapse)
process_only_nodes(toc, builder.tags, warn_node=self.env.warn_node)
if title and toc.children and len(toc.children) == 1:
child = toc.children[0]
for refnode in child.traverse(nodes.reference):
if refnode['refuri'] == ref and \
not refnode['anchorname']:
refnode.children = [nodes.Text(title)]
if not toc.children:
# empty toc means: no titles will show up in the toctree
self.env.warn_node(
'toctree contains reference to document %r that '
'doesn\'t have a title: no link will be generated'
% ref, toctreenode)
except KeyError:
# this is raised if the included file does not exist
self.env.warn_node(
'toctree contains reference to nonexisting document %r'
% ref, toctreenode)
else:
# if titles_only is given, only keep the main title and
# sub-toctrees
if titles_only:
# delete everything but the toplevel title(s)
# and toctrees
for toplevel in toc:
# nodes with length 1 don't have any children anyway
if len(toplevel) > 1:
subtrees = toplevel.traverse(addnodes.toctree)
if subtrees:
toplevel[1][:] = subtrees
else:
toplevel.pop(1)
# resolve all sub-toctrees
for subtocnode in toc.traverse(addnodes.toctree):
if not (subtocnode.get('hidden', False) and
not includehidden):
i = subtocnode.parent.index(subtocnode) + 1
for item in _entries_from_toctree(
subtocnode, [refdoc] + parents,
subtree=True):
subtocnode.parent.insert(i, item)
i += 1
subtocnode.parent.remove(subtocnode)
if separate:
entries.append(toc)
else:
entries.extend(toc.children)
if not subtree and not separate:
ret = nodes.bullet_list()
ret += entries
return [ret]
return entries
maxdepth = maxdepth or toctree.get('maxdepth', -1)
if not titles_only and toctree.get('titlesonly', False):
titles_only = True
if not includehidden and toctree.get('includehidden', False):
includehidden = True
# NOTE: previously, this was separate=True, but that leads to artificial
# separation when two or more toctree entries form a logical unit, so
# separating mode is no longer used -- it's kept here for history's sake
tocentries = _entries_from_toctree(toctree, [], separate=False)
if not tocentries:
return None
newnode = addnodes.compact_paragraph('', '')
caption = toctree.attributes.get('caption')
if caption:
caption_node = nodes.caption(caption, '', *[nodes.Text(caption)])
caption_node.line = toctree.line
caption_node.source = toctree.source
caption_node.rawsource = toctree['rawcaption']
if hasattr(toctree, 'uid'):
# move uid to caption_node to translate it
caption_node.uid = toctree.uid
del toctree.uid
newnode += caption_node
newnode.extend(tocentries)
newnode['toctree'] = True
# prune the tree to maxdepth, also set toc depth and current classes
_toctree_add_classes(newnode, 1)
self._toctree_prune(newnode, 1, prune and maxdepth or 0, collapse)
if len(newnode[-1]) == 0: # No titles found
return None
# set the target paths in the toctrees (they are not known at TOC
# generation time)
for refnode in newnode.traverse(nodes.reference):
if not url_re.match(refnode['refuri']):
refnode['refuri'] = builder.get_relative_uri(
docname, refnode['refuri']) + refnode['anchorname']
return newnode
def get_toctree_ancestors(self, docname):
parent = {}
for p, children in iteritems(self.toctree_includes):
for child in children:
parent[child] = p
ancestors = []
d = docname
while d in parent and d not in ancestors:
ancestors.append(d)
d = parent[d]
return ancestors
def _toctree_prune(self, node, depth, maxdepth, collapse=False):
"""Utility: Cut a TOC at a specified depth."""
for subnode in node.children[:]:
if isinstance(subnode, (addnodes.compact_paragraph,
nodes.list_item)):
# for <p> and <li>, just recurse
self._toctree_prune(subnode, depth, maxdepth, collapse)
elif isinstance(subnode, nodes.bullet_list):
# for <ul>, determine if the depth is too large or if the
# entry is to be collapsed
if maxdepth > 0 and depth > maxdepth:
subnode.parent.replace(subnode, [])
else:
# cull sub-entries whose parents aren't 'current'
if (collapse and depth > 1 and
'iscurrent' not in subnode.parent):
subnode.parent.remove(subnode)
else:
# recurse on visible children
self._toctree_prune(subnode, depth + 1, maxdepth, collapse)
def assign_section_numbers(self):
"""Assign a section number to each heading under a numbered toctree."""
# a list of all docnames whose section numbers changed
rewrite_needed = []
assigned = set()
old_secnumbers = self.toc_secnumbers
self.toc_secnumbers = self.env.toc_secnumbers = {}
def _walk_toc(node, secnums, depth, titlenode=None):
# titlenode is the title of the document, it will get assigned a
# secnumber too, so that it shows up in next/prev/parent rellinks
for subnode in node.children:
if isinstance(subnode, nodes.bullet_list):
numstack.append(0)
_walk_toc(subnode, secnums, depth - 1, titlenode)
numstack.pop()
titlenode = None
elif isinstance(subnode, nodes.list_item):
_walk_toc(subnode, secnums, depth, titlenode)
titlenode = None
elif isinstance(subnode, addnodes.only):
# at this stage we don't know yet which sections are going
# to be included; just include all of them, even if it leads
# to gaps in the numbering
_walk_toc(subnode, secnums, depth, titlenode)
titlenode = None
elif isinstance(subnode, addnodes.compact_paragraph):
numstack[-1] += 1
if depth > 0:
number = tuple(numstack)
else:
number = None
secnums[subnode[0]['anchorname']] = \
subnode[0]['secnumber'] = number
if titlenode:
titlenode['secnumber'] = number
titlenode = None
elif isinstance(subnode, addnodes.toctree):
_walk_toctree(subnode, depth)
def _walk_toctree(toctreenode, depth):
if depth == 0:
return
for (title, ref) in toctreenode['entries']:
if url_re.match(ref) or ref == 'self':
# don't mess with those
continue
elif ref in assigned:
self.env.warn_node('%s is already assigned section numbers '
'(nested numbered toctree?)' % ref,
toctreenode, type='toc', subtype='secnum')
elif ref in self.tocs:
secnums = self.toc_secnumbers[ref] = {}
assigned.add(ref)
_walk_toc(self.tocs[ref], secnums, depth,
self.env.titles.get(ref))
if secnums != old_secnumbers.get(ref):
rewrite_needed.append(ref)
for docname in self.numbered_toctrees:
assigned.add(docname)
doctree = self.env.get_doctree(docname)
for toctreenode in doctree.traverse(addnodes.toctree):
depth = toctreenode.get('numbered', 0)
if depth:
# every numbered toctree gets new numbering
numstack = [0]
_walk_toctree(toctreenode, depth)
return rewrite_needed
def assign_figure_numbers(self):
"""Assign a figure number to each figure under a numbered toctree."""
rewrite_needed = []
assigned = set()
old_fignumbers = self.toc_fignumbers
self.toc_fignumbers = self.env.toc_fignumbers = {}
fignum_counter = {}
def get_section_number(docname, section):
anchorname = '#' + section['ids'][0]
secnumbers = self.toc_secnumbers.get(docname, {})
if anchorname in secnumbers:
secnum = secnumbers.get(anchorname)
else:
secnum = secnumbers.get('')
return secnum or tuple()
def get_next_fignumber(figtype, secnum):
counter = fignum_counter.setdefault(figtype, {})
secnum = secnum[:self.env.config.numfig_secnum_depth]
counter[secnum] = counter.get(secnum, 0) + 1
return secnum + (counter[secnum],)
def register_fignumber(docname, secnum, figtype, fignode):
self.toc_fignumbers.setdefault(docname, {})
fignumbers = self.toc_fignumbers[docname].setdefault(figtype, {})
figure_id = fignode['ids'][0]
fignumbers[figure_id] = get_next_fignumber(figtype, secnum)
def _walk_doctree(docname, doctree, secnum):
for subnode in doctree.children:
if isinstance(subnode, nodes.section):
next_secnum = get_section_number(docname, subnode)
if next_secnum:
_walk_doctree(docname, subnode, next_secnum)
else:
_walk_doctree(docname, subnode, secnum)
continue
elif isinstance(subnode, addnodes.toctree):
for title, subdocname in subnode['entries']:
if url_re.match(subdocname) or subdocname == 'self':
# don't mess with those
continue
_walk_doc(subdocname, secnum)
continue
figtype = self.env.domains['std'].get_figtype(subnode)
if figtype and subnode['ids']:
register_fignumber(docname, secnum, figtype, subnode)
_walk_doctree(docname, subnode, secnum)
def _walk_doc(docname, secnum):
if docname not in assigned:
assigned.add(docname)
doctree = self.env.get_doctree(docname)
_walk_doctree(docname, doctree, secnum)
if self.env.config.numfig:
_walk_doc(self.env.config.master_doc, tuple())
for docname, fignums in iteritems(self.toc_fignumbers):
if fignums != old_fignumbers.get(docname):
rewrite_needed.append(docname)
return rewrite_needed

10
sphinx/errors.py
View File

@ -10,6 +10,10 @@
:license: BSD, see LICENSE for details.
"""
if False:
# For type annotation
from typing import Any # NOQA
class SphinxError(Exception):
"""
@ -29,16 +33,19 @@ class ExtensionError(SphinxError):
category = 'Extension error'
def __init__(self, message, orig_exc=None):
# type: (unicode, Exception) -> None
SphinxError.__init__(self, message)
self.orig_exc = orig_exc
def __repr__(self):
# type: () -> str
if self.orig_exc:
return '%s(%r, %r)' % (self.__class__.__name__,
self.message, self.orig_exc)
return '%s(%r)' % (self.__class__.__name__, self.message)
def __str__(self):
# type: () -> str
parent_str = SphinxError.__str__(self)
if self.orig_exc:
return '%s (exception: %s)' % (parent_str, self.orig_exc)
@ -59,6 +66,7 @@ class VersionRequirementError(SphinxError):
class PycodeError(Exception):
def __str__(self):
# type: () -> str
res = self.args[0]
if len(self.args) > 1:
res += ' (exception was: %r)' % self.args[1]
@ -70,8 +78,10 @@ class SphinxParallelError(SphinxError):
category = 'Sphinx parallel build error'
def __init__(self, message, traceback):
# type: (str, Any) -> None
self.message = message
self.traceback = traceback
def __str__(self):
# type: () -> str
return self.message

223
sphinx/ext/autodoc.py
View File

@ -20,8 +20,10 @@ from types import FunctionType, BuiltinFunctionType, MethodType
from six import PY2, iterkeys, iteritems, itervalues, text_type, class_types, \
string_types, StringIO
from docutils import nodes
from docutils.utils import assemble_option_dict
from docutils.parsers.rst import Directive
from docutils.statemachine import ViewList
import sphinx
@ -29,21 +31,30 @@ from sphinx.util import rpartition, force_decode
from sphinx.locale import _
from sphinx.pycode import ModuleAnalyzer, PycodeError
from sphinx.application import ExtensionError
from sphinx.util import logging
from sphinx.util.nodes import nested_parse_with_titles
from sphinx.util.compat import Directive
from sphinx.util.inspect import getargspec, isdescriptor, safe_getmembers, \
safe_getattr, object_description, is_builtin_class_method, \
isenumclass, isenumattribute
from sphinx.util.docstrings import prepare_docstring
if False:
# For type annotation
from typing import Any, Callable, Iterator, Sequence, Tuple, Type, Union # NOQA
from types import ModuleType # NOQA
from docutils.utils import Reporter # NOQA
from sphinx.application import Sphinx # NOQA
try:
if sys.version_info >= (3,):
import typing
else:
typing = None
typing = None # type: ignore
except ImportError:
typing = None
logger = logging.getLogger(__name__)
# This type isn't exposed directly in any modules, but can be found
# here in most Python versions
MethodDescriptorType = type(type.__subclasses__)
@ -63,28 +74,33 @@ py_ext_sig_re = re.compile(
class DefDict(dict):
"""A dict that returns a default on nonexisting keys."""
def __init__(self, default):
# type: (Any) -> None
dict.__init__(self)
self.default = default
def __getitem__(self, key):
# type: (Any) -> Any
try:
return dict.__getitem__(self, key)
except KeyError:
return self.default
def __bool__(self):
# type: () -> bool
# docutils check "if option_spec"
return True
__nonzero__ = __bool__ # for python2 compatibility
def identity(x):
# type: (Any) -> Any
return x
class Options(dict):
"""A dict/attribute hybrid that returns None on nonexisting keys."""
def __getattr__(self, name):
# type: (unicode) -> Any
try:
return self[name.replace('_', '-')]
except KeyError:
@ -97,22 +113,26 @@ class _MockModule(object):
__path__ = '/dev/null'
def __init__(self, *args, **kwargs):
self.__all__ = []
# type: (Any, Any) -> None
self.__all__ = [] # type: List[str]
def __call__(self, *args, **kwargs):
# type: (Any, Any) -> _MockModule
if args and type(args[0]) in [FunctionType, MethodType]:
# Appears to be a decorator, pass through unchanged
return args[0]
return _MockModule()
def _append_submodule(self, submod):
# type: (str) -> None
self.__all__.append(submod)
@classmethod
def __getattr__(cls, name):
# type: (unicode) -> Any
if name[0] == name[0].upper():
# Not very good, we assume Uppercase names are classes...
mocktype = type(name, (), {})
mocktype = type(name, (), {}) # type: ignore
mocktype.__module__ = __name__
return mocktype
else:
@ -120,15 +140,16 @@ class _MockModule(object):
def mock_import(modname):
# type: (str) -> None
if '.' in modname:
pkg, _n, mods = modname.rpartition('.')
mock_import(pkg)
if isinstance(sys.modules[pkg], _MockModule):
sys.modules[pkg]._append_submodule(mods)
sys.modules[pkg]._append_submodule(mods) # type: ignore
if modname not in sys.modules:
mod = _MockModule()
sys.modules[modname] = mod
sys.modules[modname] = mod # type: ignore
ALL = object()
@ -136,6 +157,7 @@ INSTANCEATTR = object()
def members_option(arg):
# type: (Any) -> Union[object, List[unicode]]
"""Used to convert the :members: option to auto directives."""
if arg is None:
return ALL
@ -143,6 +165,7 @@ def members_option(arg):
def members_set_option(arg):
# type: (Any) -> Union[object, Set[unicode]]
"""Used to convert the :members: option to auto directives."""
if arg is None:
return ALL
@ -153,6 +176,7 @@ SUPPRESS = object()
def annotation_option(arg):
# type: (Any) -> Any
if arg is None:
# suppress showing the representation of the object
return SUPPRESS
@ -161,6 +185,7 @@ def annotation_option(arg):
def bool_option(arg):
# type: (Any) -> bool
"""Used to convert flag options to auto directives. (Instead of
directives.flag(), which returns None).
"""
@ -173,13 +198,16 @@ class AutodocReporter(object):
and line number to a system message, as recorded in a ViewList.
"""
def __init__(self, viewlist, reporter):
# type: (ViewList, Reporter) -> None
self.viewlist = viewlist
self.reporter = reporter
def __getattr__(self, name):
# type: (unicode) -> Any
return getattr(self.reporter, name)
def system_message(self, level, message, *children, **kwargs):
# type: (int, unicode, Any, Any) -> nodes.system_message
if 'line' in kwargs and 'source' not in kwargs:
try:
source, line = self.viewlist.items[kwargs['line']]
@ -192,25 +220,31 @@ class AutodocReporter(object):
*children, **kwargs)
def debug(self, *args, **kwargs):
# type: (Any, Any) -> nodes.system_message
if self.reporter.debug_flag:
return self.system_message(0, *args, **kwargs)
def info(self, *args, **kwargs):
# type: (Any, Any) -> nodes.system_message
return self.system_message(1, *args, **kwargs)
def warning(self, *args, **kwargs):
# type: (Any, Any) -> nodes.system_message
return self.system_message(2, *args, **kwargs)
def error(self, *args, **kwargs):
# type: (Any, Any) -> nodes.system_message
return self.system_message(3, *args, **kwargs)
def severe(self, *args, **kwargs):
# type: (Any, Any) -> nodes.system_message
return self.system_message(4, *args, **kwargs)
# Some useful event listener factories for autodoc-process-docstring.
def cut_lines(pre, post=0, what=None):
# type: (int, int, unicode) -> Callable
"""Return a listener that removes the first *pre* and last *post*
lines of every docstring. If *what* is a sequence of strings,
only docstrings of a type in *what* will be processed.
@ -223,6 +257,7 @@ def cut_lines(pre, post=0, what=None):
This can (and should) be used in place of :confval:`automodule_skip_lines`.
"""
def process(app, what_, name, obj, options, lines):
# type: (Sphinx, unicode, unicode, Any, Any, List[unicode]) -> None
if what and what_ not in what:
return
del lines[:pre]
@ -238,6 +273,7 @@ def cut_lines(pre, post=0, what=None):
def between(marker, what=None, keepempty=False, exclude=False):
# type: (unicode, Sequence[unicode], bool, bool) -> Callable
"""Return a listener that either keeps, or if *exclude* is True excludes,
lines between lines that match the *marker* regular expression. If no line
matches, the resulting docstring would be empty, so no change will be made
@ -249,6 +285,7 @@ def between(marker, what=None, keepempty=False, exclude=False):
marker_re = re.compile(marker)
def process(app, what_, name, obj, options, lines):
# type: (Sphinx, unicode, unicode, Any, Any, List[unicode]) -> None
if what and what_ not in what:
return
deleted = 0
@ -272,6 +309,7 @@ def between(marker, what=None, keepempty=False, exclude=False):
def format_annotation(annotation):
# type: (Any) -> str
"""Return formatted representation of a type annotation.
Show qualified names for types and additional details for types from
@ -279,18 +317,18 @@ def format_annotation(annotation):
Displaying complex types from ``typing`` relies on its private API.
"""
if typing and isinstance(annotation, typing.TypeVar):
if typing and isinstance(annotation, typing.TypeVar): # type: ignore
return annotation.__name__
if annotation == Ellipsis:
return '...'
if not isinstance(annotation, type):
return repr(annotation)
qualified_name = (annotation.__module__ + '.' + annotation.__qualname__
qualified_name = (annotation.__module__ + '.' + annotation.__qualname__ # type: ignore
if annotation else repr(annotation))
if annotation.__module__ == 'builtins':
return annotation.__qualname__
return annotation.__qualname__ # type: ignore
elif typing:
if hasattr(typing, 'GenericMeta') and \
isinstance(annotation, typing.GenericMeta):
@ -351,6 +389,7 @@ def format_annotation(annotation):
def formatargspec(function, args, varargs=None, varkw=None, defaults=None,
kwonlyargs=(), kwonlydefaults={}, annotations={}):
# type: (Callable, Tuple[str, ...], str, str, Any, Tuple, Dict, Dict[str, Any]) -> str
"""Return a string representation of an ``inspect.FullArgSpec`` tuple.
An enhanced version of ``inspect.formatargspec()`` that handles typing
@ -358,18 +397,20 @@ def formatargspec(function, args, varargs=None, varkw=None, defaults=None,
"""
def format_arg_with_annotation(name):
# type: (str) -> str
if name in annotations:
return '%s: %s' % (name, format_annotation(get_annotation(name)))
return name
def get_annotation(name):
# type: (str) -> str
value = annotations[name]
if isinstance(value, string_types):
return introspected_hints.get(name, value)
else:
return value
introspected_hints = (typing.get_type_hints(function)
introspected_hints = (typing.get_type_hints(function) # type: ignore
if typing and hasattr(function, '__code__') else {})
fd = StringIO()
@ -383,7 +424,7 @@ def formatargspec(function, args, varargs=None, varkw=None, defaults=None,
arg_fd.write(format_arg_with_annotation(arg))
if defaults and i >= defaults_start:
arg_fd.write(' = ' if arg in annotations else '=')
arg_fd.write(object_description(defaults[i - defaults_start]))
arg_fd.write(object_description(defaults[i - defaults_start])) # type: ignore
formatted.append(arg_fd.getvalue())
if varargs:
@ -398,7 +439,7 @@ def formatargspec(function, args, varargs=None, varkw=None, defaults=None,
arg_fd.write(format_arg_with_annotation(kwarg))
if kwonlydefaults and kwarg in kwonlydefaults:
arg_fd.write(' = ' if kwarg in annotations else '=')
arg_fd.write(object_description(kwonlydefaults[kwarg]))
arg_fd.write(object_description(kwonlydefaults[kwarg])) # type: ignore
formatted.append(arg_fd.getvalue())
if varkw:
@ -445,6 +486,7 @@ class Documenter(object):
@staticmethod
def get_attr(obj, name, *defargs):
# type: (Any, unicode, Any) -> Any
"""getattr() override for types such as Zope interfaces."""
for typ, func in iteritems(AutoDirective._special_attrgetters):
if isinstance(obj, typ):
@ -453,10 +495,12 @@ class Documenter(object):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
"""Called to see if a member can be documented by this documenter."""
raise NotImplementedError('must be implemented in subclasses')
def __init__(self, directive, name, indent=u''):
# type: (Directive, unicode, unicode) -> None
self.directive = directive
self.env = directive.env
self.options = directive.genopt
@ -464,27 +508,29 @@ class Documenter(object):
self.indent = indent
# the module and object path within the module, and the fully
# qualified name (all set after resolve_name succeeds)
self.modname = None
self.module = None
self.objpath = None
self.fullname = None
self.modname = None # type: str
self.module = None # type: ModuleType
self.objpath = None # type: List[unicode]
self.fullname = None # type: unicode
# extra signature items (arguments and return annotation,
# also set after resolve_name succeeds)
self.args = None
self.retann = None
self.args = None # type: unicode
self.retann = None # type: unicode
# the object to document (set after import_object succeeds)
self.object = None
self.object_name = None
self.object = None # type: Any
self.object_name = None # type: unicode
# the parent/owner of the object to document
self.parent = None
self.parent = None # type: Any
# the module analyzer to get at attribute docs, or None
self.analyzer = None
self.analyzer = None # type: Any
def add_line(self, line, source, *lineno):
# type: (unicode, unicode, int) -> None
"""Append one line of generated reST to the output."""
self.directive.result.append(self.indent + line, source, *lineno)
def resolve_name(self, modname, parents, path, base):
# type: (str, Any, str, Any) -> Tuple[str, List[unicode]]
"""Resolve the module and name of the object to document given by the
arguments and the current module/class.
@ -495,6 +541,7 @@ class Documenter(object):
raise NotImplementedError('must be implemented in subclasses')
def parse_name(self):
# type: () -> bool
"""Determine what module to import and what attribute to document.
Returns True and sets *self.modname*, *self.objpath*, *self.fullname*,
@ -505,7 +552,7 @@ class Documenter(object):
# an autogenerated one
try:
explicit_modname, path, base, args, retann = \
py_ext_sig_re.match(self.name).groups()
py_ext_sig_re.match(self.name).groups() # type: ignore
except AttributeError:
self.directive.warn('invalid signature for auto%s (%r)' %
(self.objtype, self.name))
@ -519,8 +566,7 @@ class Documenter(object):
modname = None
parents = []
self.modname, self.objpath = \
self.resolve_name(modname, parents, path, base)
self.modname, self.objpath = self.resolve_name(modname, parents, path, base)
if not self.modname:
return False
@ -532,31 +578,31 @@ class Documenter(object):
return True
def import_object(self):
# type: () -> bool
"""Import the object given by *self.modname* and *self.objpath* and set
it as *self.object*.
Returns True if successful, False if an error occurred.
"""
dbg = self.env.app.debug
if self.objpath:
dbg('[autodoc] from %s import %s',
self.modname, '.'.join(self.objpath))
logger.debug('[autodoc] from %s import %s',
self.modname, '.'.join(self.objpath))
try:
dbg('[autodoc] import %s', self.modname)
logger.debug('[autodoc] import %s', self.modname)
for modname in self.env.config.autodoc_mock_imports:
dbg('[autodoc] adding a mock module %s!', modname)
logger.debug('[autodoc] adding a mock module %s!', modname)
mock_import(modname)
with warnings.catch_warnings():
warnings.filterwarnings("ignore", category=ImportWarning)
__import__(self.modname)
parent = None
obj = self.module = sys.modules[self.modname]
dbg('[autodoc] => %r', obj)
logger.debug('[autodoc] => %r', obj)
for part in self.objpath:
parent = obj
dbg('[autodoc] getattr(_, %r)', part)
logger.debug('[autodoc] getattr(_, %r)', part)
obj = self.get_attr(obj, part)
dbg('[autodoc] => %r', obj)
logger.debug('[autodoc] => %r', obj)
self.object_name = part
self.parent = parent
self.object = obj
@ -577,13 +623,14 @@ class Documenter(object):
errmsg += '; the following exception was raised:\n%s' % \
traceback.format_exc()
if PY2:
errmsg = errmsg.decode('utf-8')
dbg(errmsg)
errmsg = errmsg.decode('utf-8') # type: ignore
logger.debug(errmsg)
self.directive.warn(errmsg)
self.env.note_reread()
return False
def get_real_modname(self):
# type: () -> str
"""Get the real module name of an object to document.
It can differ from the name of the module through which the object was
@ -592,6 +639,7 @@ class Documenter(object):
return self.get_attr(self.object, '__module__', None) or self.modname
def check_module(self):
# type: () -> bool
"""Check if *self.object* is really defined in the module given by
*self.modname*.
"""
@ -604,6 +652,7 @@ class Documenter(object):
return True
def format_args(self):
# type: () -> unicode
"""Format the argument signature of *self.object*.
Should return None if the object does not have a signature.
@ -611,6 +660,7 @@ class Documenter(object):
return None
def format_name(self):
# type: () -> unicode
"""Format the name of *self.object*.
This normally should be something that can be parsed by the generated
@ -622,13 +672,14 @@ class Documenter(object):
return '.'.join(self.objpath) or self.modname
def format_signature(self):
# type: () -> unicode
"""Format the signature (arguments and return annotation) of the object.
Let the user process it via the ``autodoc-process-signature`` event.
"""
if self.args is not None:
# signature given explicitly
args = "(%s)" % self.args
args = "(%s)" % self.args # type: unicode
else:
# try to introspect the signature
try:
@ -652,6 +703,7 @@ class Documenter(object):
return ''
def add_directive_header(self, sig):
# type: (unicode) -> None
"""Add the directive header and options to the generated content."""
domain = getattr(self, 'domain', 'py')
directive = getattr(self, 'directivetype', self.objtype)
@ -667,6 +719,7 @@ class Documenter(object):
self.add_line(u' :module: %s' % self.modname, sourcename)
def get_doc(self, encoding=None, ignore=1):
# type: (unicode, int) -> List[List[unicode]]
"""Decode and return lines of the docstring(s) for the object."""
docstring = self.get_attr(self.object, '__doc__', None)
# make sure we have Unicode docstrings, then sanitize and split
@ -680,6 +733,7 @@ class Documenter(object):
return []
def process_doc(self, docstrings):
# type: (List[List[unicode]]) -> Iterator[unicode]
"""Let the user process the docstrings before adding them."""
for docstringlines in docstrings:
if self.env.app:
@ -691,6 +745,7 @@ class Documenter(object):
yield line
def get_sourcename(self):
# type: () -> unicode
if self.analyzer:
# prevent encoding errors when the file name is non-ASCII
if not isinstance(self.analyzer.srcname, text_type):
@ -702,6 +757,7 @@ class Documenter(object):
return u'docstring of %s' % self.fullname
def add_content(self, more_content, no_docstring=False):
# type: (Any, bool) -> None
"""Add content from docstrings, attribute documentation and user."""
# set sourcename and add content from attribute documentation
sourcename = self.get_sourcename()
@ -733,6 +789,7 @@ class Documenter(object):
self.add_line(line, src[0], src[1])
def get_object_members(self, want_all):
# type: (bool) -> Tuple[bool, List[Tuple[unicode, object]]]
"""Return `(members_check_module, members)` where `members` is a
list of `(membername, member)` pairs of the members of *self.object*.
@ -792,6 +849,7 @@ class Documenter(object):
return False, sorted(members)
def filter_members(self, members, want_all):
# type: (List[Tuple[unicode, Any]], bool) -> List[Tuple[unicode, Any, bool]]
"""Filter the given member list.
Members are skipped if
@ -869,6 +927,7 @@ class Documenter(object):
return ret
def document_members(self, all_members=False):
# type: (bool) -> None
"""Generate reST for member documentation.
If *all_members* is True, do all members, else those given by
@ -890,7 +949,7 @@ class Documenter(object):
if membername not in self.options.exclude_members]
# document non-skipped members
memberdocumenters = []
memberdocumenters = [] # type: List[Tuple[Documenter, bool]]
for (mname, member, isattr) in self.filter_members(members, want_all):
classes = [cls for cls in itervalues(AutoDirective._registry)
if cls.can_document_member(member, mname, isattr, self)]
@ -916,6 +975,7 @@ class Documenter(object):
tagorder = self.analyzer.tagorder
def keyfunc(entry):
# type: (Tuple[Documenter, bool]) -> int
fullname = entry[0].name.split('::')[1]
return tagorder.get(fullname, len(tagorder))
memberdocumenters.sort(key=keyfunc)
@ -931,6 +991,7 @@ class Documenter(object):
def generate(self, more_content=None, real_modname=None,
check_module=False, all_members=False):
# type: (Any, str, bool, bool) -> None
"""Generate reST for the object given by *self.name*, and possibly for
its members.
@ -966,7 +1027,7 @@ class Documenter(object):
# be cached anyway)
self.analyzer.find_attr_docs()
except PycodeError as err:
self.env.app.debug('[autodoc] module analyzer failed: %s', err)
logger.debug('[autodoc] module analyzer failed: %s', err)
# no source file -- e.g. for builtin and C modules
self.analyzer = None
# at least add the module.__file__ as a dependency
@ -1024,15 +1085,18 @@ class ModuleDocumenter(Documenter):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
# don't document submodules automatically
return False
def resolve_name(self, modname, parents, path, base):
# type: (str, Any, str, Any) -> Tuple[str, List[unicode]]
if modname is not None:
self.directive.warn('"::" in automodule name doesn\'t make sense')
return (path or '') + base, []
def parse_name(self):
# type: () -> bool
ret = Documenter.parse_name(self)
if self.args or self.retann:
self.directive.warn('signature arguments or return annotation '
@ -1040,6 +1104,7 @@ class ModuleDocumenter(Documenter):
return ret
def add_directive_header(self, sig):
# type: (unicode) -> None
Documenter.add_directive_header(self, sig)
sourcename = self.get_sourcename()
@ -1055,6 +1120,7 @@ class ModuleDocumenter(Documenter):
self.add_line(u' :deprecated:', sourcename)
def get_object_members(self, want_all):
# type: (bool) -> Tuple[bool, List[Tuple[unicode, object]]]
if want_all:
if not hasattr(self.object, '__all__'):
# for implicit module members, check __module__ to avoid
@ -1091,6 +1157,7 @@ class ModuleLevelDocumenter(Documenter):
classes, data/constants).
"""
def resolve_name(self, modname, parents, path, base):
# type: (str, Any, str, Any) -> Tuple[str, List[unicode]]
if modname is None:
if path:
modname = path.rstrip('.')
@ -1111,6 +1178,7 @@ class ClassLevelDocumenter(Documenter):
attributes).
"""
def resolve_name(self, modname, parents, path, base):
# type: (str, Any, str, Any) -> Tuple[str, List[unicode]]
if modname is None:
if path:
mod_cls = path.rstrip('.')
@ -1126,7 +1194,7 @@ class ClassLevelDocumenter(Documenter):
# ... if still None, there's no way to know
if mod_cls is None:
return None, []
modname, cls = rpartition(mod_cls, '.')
modname, cls = rpartition(mod_cls, '.') # type: ignore
parents = [cls]
# if the module name is still missing, get it like above
if not modname:
@ -1144,6 +1212,7 @@ class DocstringSignatureMixin(object):
"""
def _find_signature(self, encoding=None):
# type: (unicode) -> Tuple[str, str]
docstrings = self.get_doc(encoding)
self._new_docstrings = docstrings[:]
result = None
@ -1152,12 +1221,12 @@ class DocstringSignatureMixin(object):
if not doclines:
continue
# match first line of docstring against signature RE
match = py_ext_sig_re.match(doclines[0])
match = py_ext_sig_re.match(doclines[0]) # type: ignore
if not match:
continue
exmod, path, base, args, retann = match.groups()
# the base name must match ours
valid_names = [self.objpath[-1]]
valid_names = [self.objpath[-1]] # type: ignore
if isinstance(self, ClassDocumenter):
valid_names.append('__init__')
if hasattr(self.object, '__mro__'):
@ -1172,19 +1241,21 @@ class DocstringSignatureMixin(object):
return result
def get_doc(self, encoding=None, ignore=1):
# type: (unicode, int) -> List[List[unicode]]
lines = getattr(self, '_new_docstrings', None)
if lines is not None:
return lines
return Documenter.get_doc(self, encoding, ignore)
return Documenter.get_doc(self, encoding, ignore) # type: ignore
def format_signature(self):
if self.args is None and self.env.config.autodoc_docstring_signature:
# type: () -> unicode
if self.args is None and self.env.config.autodoc_docstring_signature: # type: ignore
# only act if a signature is not explicitly given already, and if
# the feature is enabled
result = self._find_signature()
if result is not None:
self.args, self.retann = result
return Documenter.format_signature(self)
return Documenter.format_signature(self) # type: ignore
class DocstringStripSignatureMixin(DocstringSignatureMixin):
@ -1193,7 +1264,8 @@ class DocstringStripSignatureMixin(DocstringSignatureMixin):
feature of stripping any function signature from the docstring.
"""
def format_signature(self):
if self.args is None and self.env.config.autodoc_docstring_signature:
# type: () -> unicode
if self.args is None and self.env.config.autodoc_docstring_signature: # type: ignore
# only act if a signature is not explicitly given already, and if
# the feature is enabled
result = self._find_signature()
@ -1202,10 +1274,10 @@ class DocstringStripSignatureMixin(DocstringSignatureMixin):
# DocstringSignatureMixin.format_signature.
# Documenter.format_signature use self.args value to format.
_args, self.retann = result
return Documenter.format_signature(self)
return Documenter.format_signature(self) # type: ignore
class FunctionDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
class FunctionDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter): # type: ignore
"""
Specialized Documenter subclass for functions.
"""
@ -1214,9 +1286,11 @@ class FunctionDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
return isinstance(member, (FunctionType, BuiltinFunctionType))
def format_args(self):
# type: () -> unicode
if inspect.isbuiltin(self.object) or \
inspect.ismethoddescriptor(self.object):
# cannot introspect arguments of a C function or method
@ -1243,10 +1317,11 @@ class FunctionDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
return args
def document_members(self, all_members=False):
# type: (bool) -> None
pass
class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter): # type: ignore
"""
Specialized Documenter subclass for classes.
"""
@ -1262,9 +1337,11 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
return isinstance(member, class_types)
def import_object(self):
# type: () -> Any
ret = ModuleLevelDocumenter.import_object(self)
# if the class is documented under another name, document it
# as data/attribute
@ -1276,6 +1353,7 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
return ret
def format_args(self):
# type: () -> unicode
# for classes, the relevant signature is the __init__ method's
initmeth = self.get_attr(self.object, '__init__', None)
# classes without __init__ method, default __init__ or
@ -1295,12 +1373,14 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
return formatargspec(initmeth, *argspec)
def format_signature(self):
# type: () -> unicode
if self.doc_as_attr:
return ''
return DocstringSignatureMixin.format_signature(self)
def add_directive_header(self, sig):
# type: (unicode) -> None
if self.doc_as_attr:
self.directivetype = 'attribute'
Documenter.add_directive_header(self, sig)
@ -1318,6 +1398,7 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
sourcename)
def get_doc(self, encoding=None, ignore=1):
# type: (unicode, int) -> List[List[unicode]]
lines = getattr(self, '_new_docstrings', None)
if lines is not None:
return lines
@ -1363,6 +1444,7 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
return doc
def add_content(self, more_content, no_docstring=False):
# type: (Any, bool) -> None
if self.doc_as_attr:
classname = safe_getattr(self.object, '__name__', None)
if classname:
@ -1374,6 +1456,7 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter):
ModuleLevelDocumenter.add_content(self, more_content)
def document_members(self, all_members=False):
# type: (bool) -> None
if self.doc_as_attr:
return
ModuleLevelDocumenter.document_members(self, all_members)
@ -1391,8 +1474,9 @@ class ExceptionDocumenter(ClassDocumenter):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
return isinstance(member, class_types) and \
issubclass(member, BaseException)
issubclass(member, BaseException) # type: ignore
class DataDocumenter(ModuleLevelDocumenter):
@ -1407,9 +1491,11 @@ class DataDocumenter(ModuleLevelDocumenter):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
return isinstance(parent, ModuleDocumenter) and isattr
def add_directive_header(self, sig):
# type: (unicode) -> None
ModuleLevelDocumenter.add_directive_header(self, sig)
sourcename = self.get_sourcename()
if not self.options.annotation:
@ -1426,10 +1512,11 @@ class DataDocumenter(ModuleLevelDocumenter):
sourcename)
def document_members(self, all_members=False):
# type: (bool) -> None
pass
class MethodDocumenter(DocstringSignatureMixin, ClassLevelDocumenter):
class MethodDocumenter(DocstringSignatureMixin, ClassLevelDocumenter): # type: ignore
"""
Specialized Documenter subclass for methods (normal, static and class).
"""
@ -1439,10 +1526,12 @@ class MethodDocumenter(DocstringSignatureMixin, ClassLevelDocumenter):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
return inspect.isroutine(member) and \
not isinstance(parent, ModuleDocumenter)
def import_object(self):
# type: () -> Any
ret = ClassLevelDocumenter.import_object(self)
if not ret:
return ret
@ -1463,6 +1552,7 @@ class MethodDocumenter(DocstringSignatureMixin, ClassLevelDocumenter):
return ret
def format_args(self):
# type: () -> unicode
if inspect.isbuiltin(self.object) or \
inspect.ismethoddescriptor(self.object):
# can never get arguments of a C function or method
@ -1476,10 +1566,11 @@ class MethodDocumenter(DocstringSignatureMixin, ClassLevelDocumenter):
return args
def document_members(self, all_members=False):
# type: (bool) -> None
pass
class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter):
class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter): # type: ignore
"""
Specialized Documenter subclass for attributes.
"""
@ -1496,6 +1587,7 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
non_attr_types = cls.method_types + (type, MethodDescriptorType)
isdatadesc = isdescriptor(member) and not \
isinstance(member, non_attr_types) and not \
@ -1508,9 +1600,11 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter):
not isinstance(member, class_types))
def document_members(self, all_members=False):
# type: (bool) -> None
pass
def import_object(self):
# type: () -> Any
ret = ClassLevelDocumenter.import_object(self)
if isenumattribute(self.object):
self.object = self.object.value
@ -1523,10 +1617,12 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter):
return ret
def get_real_modname(self):
# type: () -> str
return self.get_attr(self.parent or self.object, '__module__', None) \
or self.modname
def add_directive_header(self, sig):
# type: (unicode) -> None
ClassLevelDocumenter.add_directive_header(self, sig)
sourcename = self.get_sourcename()
if not self.options.annotation:
@ -1544,6 +1640,7 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter):
sourcename)
def add_content(self, more_content, no_docstring=False):
# type: (Any, bool) -> None
if not self._datadescriptor:
# if it's not a data descriptor, its docstring is very probably the
# wrong thing to display
@ -1565,10 +1662,12 @@ class InstanceAttributeDocumenter(AttributeDocumenter):
@classmethod
def can_document_member(cls, member, membername, isattr, parent):
# type: (Any, unicode, bool, Any) -> bool
"""This documents only INSTANCEATTR members."""
return isattr and (member is INSTANCEATTR)
def import_object(self):
# type: () -> bool
"""Never import anything."""
# disguise as an attribute
self.objtype = 'attribute'
@ -1576,6 +1675,7 @@ class InstanceAttributeDocumenter(AttributeDocumenter):
return True
def add_content(self, more_content, no_docstring=False):
# type: (Any, bool) -> None
"""Never try to get a docstring from the object."""
AttributeDocumenter.add_content(self, more_content, no_docstring=True)
@ -1596,10 +1696,10 @@ class AutoDirective(Directive):
attributes of the parents.
"""
# a registry of objtype -> documenter class
_registry = {}
_registry = {} # type: Dict[unicode, Type[Documenter]]
# a registry of type -> getattr function
_special_attrgetters = {}
_special_attrgetters = {} # type: Dict[Type, Callable]
# flags that can be given in autodoc_default_flags
_default_flags = set([
@ -1617,21 +1717,24 @@ class AutoDirective(Directive):
option_spec = DefDict(identity)
def warn(self, msg):
# type: (unicode) -> None
self.warnings.append(self.reporter.warning(msg, line=self.lineno))
def run(self):
self.filename_set = set() # a set of dependent filenames
# type: () -> List[nodes.Node]
self.filename_set = set() # type: Set[unicode]
# a set of dependent filenames
self.reporter = self.state.document.reporter
self.env = self.state.document.settings.env
self.warnings = []
self.warnings = [] # type: List[unicode]
self.result = ViewList()
try:
source, lineno = self.reporter.get_source_and_line(self.lineno)
except AttributeError:
source = lineno = None
self.env.app.debug('[autodoc] %s:%s: input:\n%s',
source, lineno, self.block_text)
logger.debug('[autodoc] %s:%s: input:\n%s',
source, lineno, self.block_text)
# find out what documenter to call
objtype = self.name[4:]
@ -1660,7 +1763,7 @@ class AutoDirective(Directive):
if not self.result:
return self.warnings
self.env.app.debug2('[autodoc] output:\n%s', '\n'.join(self.result))
logger.debug('[autodoc] output:\n%s', '\n'.join(self.result))
# record all filenames as dependencies -- this will at least
# partially make automatic invalidation possible
@ -1687,6 +1790,7 @@ class AutoDirective(Directive):
def add_documenter(cls):
# type: (Type[Documenter]) -> None
"""Register a new Documenter."""
if not issubclass(cls, Documenter):
raise ExtensionError('autodoc documenter %r must be a subclass '
@ -1699,6 +1803,7 @@ def add_documenter(cls):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_autodocumenter(ModuleDocumenter)
app.add_autodocumenter(ClassDocumenter)
app.add_autodocumenter(ExceptionDocumenter)
@ -1724,7 +1829,9 @@ class testcls:
"""test doc string"""
def __getattr__(self, x):
# type: (Any) -> Any
return x
def __setattr__(self, x, y):
# type: (Any, Any) -> None
"""Attr setter."""

8
sphinx/ext/autosectionlabel.py
View File

@ -10,8 +10,11 @@
"""
from docutils import nodes
from sphinx.util import logging
from sphinx.util.nodes import clean_astext
logger = logging.getLogger(__name__)
def register_sections_as_label(app, document):
labels = app.env.domaindata['std']['labels']
@ -23,8 +26,9 @@ def register_sections_as_label(app, document):
sectname = clean_astext(node[0])
if name in labels:
app.env.warn_node('duplicate label %s, ' % name + 'other instance '
'in ' + app.env.doc2path(labels[name][0]), node)
logger.warning('duplicate label %s, ' % name + 'other instance '
'in ' + app.env.doc2path(labels[name][0]),
location=node)
anonlabels[name] = docname, labelid
labels[name] = docname, labelid, sectname

75
sphinx/ext/autosummary/__init__.py
View File

@ -62,17 +62,28 @@ from six import string_types
from types import ModuleType
from six import text_type
from docutils.parsers.rst import directives
from docutils.parsers.rst import Directive, directives
from docutils.statemachine import ViewList
from docutils import nodes
import sphinx
from sphinx import addnodes
from sphinx.util import import_object, rst
from sphinx.util.compat import Directive
from sphinx.environment.adapters.toctree import TocTree
from sphinx.util import import_object, rst, logging
from sphinx.pycode import ModuleAnalyzer, PycodeError
from sphinx.ext.autodoc import Options
if False:
# For type annotation
from typing import Any, Tuple, Type, Union # NOQA
from docutils.utils import Inliner # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
from sphinx.ext.autodoc import Documenter # NOQA
logger = logging.getLogger(__name__)
# -- autosummary_toc node ------------------------------------------------------
@ -81,6 +92,7 @@ class autosummary_toc(nodes.comment):
def process_autosummary_toc(app, doctree):
# type: (Sphinx, nodes.Node) -> None
"""Insert items described in autosummary:: to the TOC tree, but do
not generate the toctree:: list.
"""
@ -93,7 +105,7 @@ def process_autosummary_toc(app, doctree):
try:
if (isinstance(subnode, autosummary_toc) and
isinstance(subnode[0], addnodes.toctree)):
env.note_toctree(env.docname, subnode[0])
TocTree(env).note(env.docname, subnode[0])
continue
except IndexError:
continue
@ -105,11 +117,13 @@ def process_autosummary_toc(app, doctree):
def autosummary_toc_visit_html(self, node):
# type: (nodes.NodeVisitor, autosummary_toc) -> None
"""Hide autosummary toctree list in HTML output."""
raise nodes.SkipNode
def autosummary_noop(self, node):
# type: (nodes.NodeVisitor, nodes.Node) -> None
pass
@ -120,6 +134,7 @@ class autosummary_table(nodes.comment):
def autosummary_table_visit_html(self, node):
# type: (nodes.NodeVisitor, autosummary_table) -> None
"""Make the first column of the table non-breaking."""
try:
tbody = node[0][0][-1]
@ -138,11 +153,12 @@ def autosummary_table_visit_html(self, node):
# -- autodoc integration -------------------------------------------------------
class FakeDirective(object):
env = {}
env = {} # type: Dict
genopt = Options()
def get_documenter(obj, parent):
# type: (Any, Any) -> Type[Documenter]
"""Get an autodoc.Documenter class suitable for documenting the given
object.
@ -198,13 +214,15 @@ class Autosummary(Directive):
}
def warn(self, msg):
# type: (unicode) -> None
self.warnings.append(self.state.document.reporter.warning(
msg, line=self.lineno))
def run(self):
# type: () -> List[nodes.Node]
self.env = env = self.state.document.settings.env
self.genopt = Options()
self.warnings = []
self.warnings = [] # type: List[nodes.Node]
self.result = ViewList()
names = [x.strip().split()[0] for x in self.content
@ -237,6 +255,7 @@ class Autosummary(Directive):
return self.warnings + nodes
def get_items(self, names):
# type: (List[unicode]) -> List[Tuple[unicode, unicode, unicode, unicode]]
"""Try to import the given names, and return a list of
``[(name, signature, summary_string, real_name), ...]``.
"""
@ -244,7 +263,7 @@ class Autosummary(Directive):
prefixes = get_import_prefixes_from_env(env)
items = []
items = [] # type: List[Tuple[unicode, unicode, unicode, unicode]]
max_item_chars = 50
@ -289,8 +308,7 @@ class Autosummary(Directive):
# be cached anyway)
documenter.analyzer.find_attr_docs()
except PycodeError as err:
documenter.env.app.debug(
'[autodoc] module analyzer failed: %s', err)
logger.debug('[autodoc] module analyzer failed: %s', err)
# no source file -- e.g. for builtin and C modules
documenter.analyzer = None
@ -333,6 +351,7 @@ class Autosummary(Directive):
return items
def get_table(self, items):
# type: (List[Tuple[unicode, unicode, unicode, unicode]]) -> List[Union[addnodes.tabular_col_spec, autosummary_table]] # NOQA
"""Generate a proper list of table nodes for autosummary:: directive.
*items* is a list produced by :meth:`get_items`.
@ -351,6 +370,7 @@ class Autosummary(Directive):
group.append(body)
def append_row(*column_texts):
# type: (unicode) -> None
row = nodes.row('')
for text in column_texts:
node = nodes.paragraph('')
@ -368,7 +388,7 @@ class Autosummary(Directive):
for name, sig, summary, real_name in items:
qualifier = 'obj'
if 'nosignatures' not in self.options:
col1 = ':%s:`%s <%s>`\ %s' % (qualifier, name, real_name, rst.escape(sig))
col1 = ':%s:`%s <%s>`\ %s' % (qualifier, name, real_name, rst.escape(sig)) # type: unicode # NOQA
else:
col1 = ':%s:`%s <%s>`' % (qualifier, name, real_name)
col2 = summary
@ -378,6 +398,7 @@ class Autosummary(Directive):
def mangle_signature(sig, max_chars=30):
# type: (unicode, int) -> unicode
"""Reformat a function signature to a more compact form."""
s = re.sub(r"^\((.*)\)$", r"\1", sig).strip()
@ -387,12 +408,12 @@ def mangle_signature(sig, max_chars=30):
s = re.sub(r"'[^']*'", "", s)
# Parse the signature to arguments + options
args = []
opts = []
args = [] # type: List[unicode]
opts = [] # type: List[unicode]
opt_re = re.compile(r"^(.*, |)([a-zA-Z0-9_*]+)=")
while s:
m = opt_re.search(s)
m = opt_re.search(s) # type: ignore
if not m:
# The rest are arguments
args = s.split(', ')
@ -414,6 +435,7 @@ def mangle_signature(sig, max_chars=30):
def limited_join(sep, items, max_chars=30, overflow_marker="..."):
# type: (unicode, List[unicode], int, unicode) -> unicode
"""Join a number of strings to one, limiting the length to *max_chars*.
If the string overflows this limit, replace the last fitting item by
@ -440,11 +462,12 @@ def limited_join(sep, items, max_chars=30, overflow_marker="..."):
# -- Importing items -----------------------------------------------------------
def get_import_prefixes_from_env(env):
# type: (BuildEnvironment) -> List
"""
Obtain current Python import prefixes (for `import_by_name`)
from ``document.env``
"""
prefixes = [None]
prefixes = [None] # type: List
currmodule = env.ref_context.get('py:module')
if currmodule:
@ -461,6 +484,7 @@ def get_import_prefixes_from_env(env):
def import_by_name(name, prefixes=[None]):
# type: (unicode, List) -> Tuple[unicode, Any, Any, unicode]
"""Import a Python object that has the given *name*, under one of the
*prefixes*. The first name that succeeds is used.
"""
@ -479,6 +503,7 @@ def import_by_name(name, prefixes=[None]):
def _import_by_name(name):
# type: (str) -> Tuple[Any, Any, unicode]
"""Import a Python object given its full name."""
try:
name_parts = name.split('.')
@ -523,6 +548,7 @@ def _import_by_name(name):
def autolink_role(typ, rawtext, etext, lineno, inliner,
options={}, content=[]):
# type: (unicode, unicode, unicode, int, Inliner, Dict, List[unicode]) -> Tuple[List[nodes.Node], List[nodes.Node]] # NOQA
"""Smart linking role.
Expands to ':obj:`text`' if `text` is an object that can be imported;
@ -538,21 +564,24 @@ def autolink_role(typ, rawtext, etext, lineno, inliner,
name, obj, parent, modname = import_by_name(pnode['reftarget'], prefixes)
except ImportError:
content = pnode[0]
r[0][0] = nodes.emphasis(rawtext, content[0].astext(),
classes=content['classes'])
r[0][0] = nodes.emphasis(rawtext, content[0].astext(), # type: ignore
classes=content['classes']) # type: ignore
return r
def get_rst_suffix(app):
# type: (Sphinx) -> unicode
def get_supported_format(suffix):
# type: (unicode) -> Tuple[unicode]
parser_class = app.config.source_parsers.get(suffix)
if parser_class is None:
return ('restructuredtext',)
if isinstance(parser_class, string_types):
parser_class = import_object(parser_class, 'source parser')
parser_class = import_object(parser_class, 'source parser') # type: ignore
return parser_class.supported
for suffix in app.config.source_suffix:
suffix = None # type: unicode
for suffix in app.config.source_suffix: # type: ignore
if 'restructuredtext' in get_supported_format(suffix):
return suffix
@ -560,6 +589,7 @@ def get_rst_suffix(app):
def process_generate_options(app):
# type: (Sphinx) -> None
genfiles = app.config.autosummary_generate
if genfiles and not hasattr(genfiles, '__len__'):
@ -578,16 +608,17 @@ def process_generate_options(app):
suffix = get_rst_suffix(app)
if suffix is None:
app.warn('autosummary generats .rst files internally. '
'But your source_suffix does not contain .rst. Skipped.')
logger.warning('autosummary generats .rst files internally. '
'But your source_suffix does not contain .rst. Skipped.')
return
generate_autosummary_docs(genfiles, builder=app.builder,
warn=app.warn, info=app.info, suffix=suffix,
base_path=app.srcdir)
warn=logger.warning, info=logger.info,
suffix=suffix, base_path=app.srcdir)
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
# I need autodoc
app.setup_extension('sphinx.ext.autodoc')
app.add_node(autosummary_toc,

70
sphinx/ext/autosummary/generate.py
View File

@ -49,8 +49,16 @@ add_documenter(MethodDocumenter)
add_documenter(AttributeDocumenter)
add_documenter(InstanceAttributeDocumenter)
if False:
# For type annotation
from typing import Any, Callable, Tuple, List # NOQA
from sphinx import addnodes # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
def main(argv=sys.argv):
# type: (List[str]) -> None
usage = """%prog [OPTIONS] SOURCEFILE ..."""
p = optparse.OptionParser(usage.strip())
p.add_option("-o", "--output-dir", action="store", type="string",
@ -62,6 +70,9 @@ def main(argv=sys.argv):
p.add_option("-t", "--templates", action="store", type="string",
dest="templates", default=None,
help="Custom template directory (default: %default)")
p.add_option("-i", "--imported-members", action="store_true",
dest="imported_members", default=False,
help="Document imported members (default: %default)")
options, args = p.parse_args(argv[1:])
if len(args) < 1:
@ -69,14 +80,17 @@ def main(argv=sys.argv):
generate_autosummary_docs(args, options.output_dir,
"." + options.suffix,
template_dir=options.templates)
template_dir=options.templates,
imported_members=options.imported_members)
def _simple_info(msg):
# type: (unicode) -> None
print(msg)
def _simple_warn(msg):
# type: (unicode) -> None
print('WARNING: ' + msg, file=sys.stderr)
@ -84,7 +98,9 @@ def _simple_warn(msg):
def generate_autosummary_docs(sources, output_dir=None, suffix='.rst',
warn=_simple_warn, info=_simple_info,
base_path=None, builder=None, template_dir=None):
base_path=None, builder=None, template_dir=None,
imported_members=False):
# type: (List[unicode], unicode, unicode, Callable, Callable, unicode, Builder, unicode, bool) -> None # NOQA
showed_sources = list(sorted(sources))
if len(showed_sources) > 20:
@ -99,6 +115,7 @@ def generate_autosummary_docs(sources, output_dir=None, suffix='.rst',
sources = [os.path.join(base_path, filename) for filename in sources]
# create our own templating environment
template_dirs = None # type: List[unicode]
template_dirs = [os.path.join(package_dir, 'ext',
'autosummary', 'templates')]
if builder is not None:
@ -153,36 +170,38 @@ def generate_autosummary_docs(sources, output_dir=None, suffix='.rst',
except TemplateNotFound:
template = template_env.get_template('autosummary/base.rst')
def get_members(obj, typ, include_public=[]):
items = []
def get_members(obj, typ, include_public=[], imported=False):
# type: (Any, unicode, List[unicode], bool) -> Tuple[List[unicode], List[unicode]] # NOQA
items = [] # type: List[unicode]
for name in dir(obj):
try:
documenter = get_documenter(safe_getattr(obj, name),
obj)
value = safe_getattr(obj, name)
except AttributeError:
continue
documenter = get_documenter(value, obj)
if documenter.objtype == typ:
items.append(name)
if imported or getattr(value, '__module__', None) == obj.__name__:
items.append(name)
public = [x for x in items
if x in include_public or not x.startswith('_')]
return public, items
ns = {}
ns = {} # type: Dict[unicode, Any]
if doc.objtype == 'module':
ns['members'] = dir(obj)
ns['functions'], ns['all_functions'] = \
get_members(obj, 'function')
get_members(obj, 'function', imported=imported_members)
ns['classes'], ns['all_classes'] = \
get_members(obj, 'class')
get_members(obj, 'class', imported=imported_members)
ns['exceptions'], ns['all_exceptions'] = \
get_members(obj, 'exception')
get_members(obj, 'exception', imported=imported_members)
elif doc.objtype == 'class':
ns['members'] = dir(obj)
ns['methods'], ns['all_methods'] = \
get_members(obj, 'method', ['__init__'])
get_members(obj, 'method', ['__init__'], imported=imported_members)
ns['attributes'], ns['all_attributes'] = \
get_members(obj, 'attribute')
get_members(obj, 'attribute', imported=imported_members)
parts = name.split('.')
if doc.objtype in ('method', 'attribute'):
@ -215,21 +234,23 @@ def generate_autosummary_docs(sources, output_dir=None, suffix='.rst',
# -- Finding documented entries in files ---------------------------------------
def find_autosummary_in_files(filenames):
# type: (List[unicode]) -> List[Tuple[unicode, unicode, unicode]]
"""Find out what items are documented in source/*.rst.
See `find_autosummary_in_lines`.
"""
documented = []
documented = [] # type: List[Tuple[unicode, unicode, unicode]]
for filename in filenames:
with codecs.open(filename, 'r', encoding='utf-8',
with codecs.open(filename, 'r', encoding='utf-8', # type: ignore
errors='ignore') as f:
lines = f.read().splitlines()
documented.extend(find_autosummary_in_lines(lines,
documented.extend(find_autosummary_in_lines(lines, # type: ignore
filename=filename))
return documented
def find_autosummary_in_docstring(name, module=None, filename=None):
# type: (unicode, Any, unicode) -> List[Tuple[unicode, unicode, unicode]]
"""Find out what items are documented in the given object's docstring.
See `find_autosummary_in_lines`.
@ -249,6 +270,7 @@ def find_autosummary_in_docstring(name, module=None, filename=None):
def find_autosummary_in_lines(lines, module=None, filename=None):
# type: (List[unicode], Any, unicode) -> List[Tuple[unicode, unicode, unicode]]
"""Find out what items appear in autosummary:: directives in the
given lines.
@ -268,9 +290,9 @@ def find_autosummary_in_lines(lines, module=None, filename=None):
toctree_arg_re = re.compile(r'^\s+:toctree:\s*(.*?)\s*$')
template_arg_re = re.compile(r'^\s+:template:\s*(.*?)\s*$')
documented = []
documented = [] # type: List[Tuple[unicode, unicode, unicode]]
toctree = None
toctree = None # type: unicode
template = None
current_module = module
in_autosummary = False
@ -278,7 +300,7 @@ def find_autosummary_in_lines(lines, module=None, filename=None):
for line in lines:
if in_autosummary:
m = toctree_arg_re.match(line)
m = toctree_arg_re.match(line) # type: ignore
if m:
toctree = m.group(1)
if filename:
@ -286,7 +308,7 @@ def find_autosummary_in_lines(lines, module=None, filename=None):
toctree)
continue
m = template_arg_re.match(line)
m = template_arg_re.match(line) # type: ignore
if m:
template = m.group(1).strip()
continue
@ -294,7 +316,7 @@ def find_autosummary_in_lines(lines, module=None, filename=None):
if line.strip().startswith(':'):
continue # skip options
m = autosummary_item_re.match(line)
m = autosummary_item_re.match(line) # type: ignore
if m:
name = m.group(1).strip()
if name.startswith('~'):
@ -310,7 +332,7 @@ def find_autosummary_in_lines(lines, module=None, filename=None):
in_autosummary = False
m = autosummary_re.match(line)
m = autosummary_re.match(line) # type: ignore
if m:
in_autosummary = True
base_indent = m.group(1)
@ -318,7 +340,7 @@ def find_autosummary_in_lines(lines, module=None, filename=None):
template = None
continue
m = automodule_re.search(line)
m = automodule_re.search(line) # type: ignore
if m:
current_module = m.group(1).strip()
# recurse into the automodule docstring
@ -326,7 +348,7 @@ def find_autosummary_in_lines(lines, module=None, filename=None):
current_module, filename=filename))
continue
m = module_re.match(line)
m = module_re.match(line) # type: ignore
if m:
current_module = m.group(2)
continue

63
sphinx/ext/coverage.py
View File

@ -20,22 +20,32 @@ from six.moves import cPickle as pickle
import sphinx
from sphinx.builders import Builder
from sphinx.util import logging
from sphinx.util.inspect import safe_getattr
if False:
# For type annotation
from typing import Any, Callable, IO, Pattern, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
# utility
def write_header(f, text, char='-'):
# type:(IO, unicode, unicode) -> None
f.write(text + '\n')
f.write(char * len(text) + '\n')
def compile_regex_list(name, exps, warnfunc):
def compile_regex_list(name, exps):
# type: (unicode, unicode) -> List[Pattern]
lst = []
for exp in exps:
try:
lst.append(re.compile(exp))
except Exception:
warnfunc('invalid regex %r in %s' % (exp, name))
logger.warning('invalid regex %r in %s', exp, name)
return lst
@ -44,45 +54,46 @@ class CoverageBuilder(Builder):
name = 'coverage'
def init(self):
self.c_sourcefiles = []
# type: () -> None
self.c_sourcefiles = [] # type: List[unicode]
for pattern in self.config.coverage_c_path:
pattern = path.join(self.srcdir, pattern)
self.c_sourcefiles.extend(glob.glob(pattern))
self.c_regexes = []
self.c_regexes = [] # type: List[Tuple[unicode, Pattern]]
for (name, exp) in self.config.coverage_c_regexes.items():
try:
self.c_regexes.append((name, re.compile(exp)))
except Exception:
self.warn('invalid regex %r in coverage_c_regexes' % exp)
logger.warning('invalid regex %r in coverage_c_regexes', exp)
self.c_ignorexps = {}
self.c_ignorexps = {} # type: Dict[unicode, List[Pattern]]
for (name, exps) in iteritems(self.config.coverage_ignore_c_items):
self.c_ignorexps[name] = compile_regex_list(
'coverage_ignore_c_items', exps, self.warn)
self.mod_ignorexps = compile_regex_list(
'coverage_ignore_modules', self.config.coverage_ignore_modules,
self.warn)
self.cls_ignorexps = compile_regex_list(
'coverage_ignore_classes', self.config.coverage_ignore_classes,
self.warn)
self.fun_ignorexps = compile_regex_list(
'coverage_ignore_functions', self.config.coverage_ignore_functions,
self.warn)
self.c_ignorexps[name] = compile_regex_list('coverage_ignore_c_items',
exps)
self.mod_ignorexps = compile_regex_list('coverage_ignore_modules',
self.config.coverage_ignore_modules)
self.cls_ignorexps = compile_regex_list('coverage_ignore_classes',
self.config.coverage_ignore_classes)
self.fun_ignorexps = compile_regex_list('coverage_ignore_functions',
self.config.coverage_ignore_functions)
def get_outdated_docs(self):
# type: () -> unicode
return 'coverage overview'
def write(self, *ignored):
self.py_undoc = {}
# type: (Any) -> None
self.py_undoc = {} # type: Dict[unicode, Dict[unicode, Any]]
self.build_py_coverage()
self.write_py_coverage()
self.c_undoc = {}
self.c_undoc = {} # type: Dict[unicode, Set[Tuple[unicode, unicode]]]
self.build_c_coverage()
self.write_c_coverage()
def build_c_coverage(self):
# type: () -> None
# Fetch all the info from the header files
c_objects = self.env.domaindata['c']['objects']
for filename in self.c_sourcefiles:
@ -94,7 +105,7 @@ class CoverageBuilder(Builder):
if match:
name = match.groups()[0]
if name not in c_objects:
for exp in self.c_ignorexps.get(key, ()):
for exp in self.c_ignorexps.get(key, []):
if exp.match(name):
break
else:
@ -104,6 +115,7 @@ class CoverageBuilder(Builder):
self.c_undoc[filename] = undoc
def write_c_coverage(self):
# type: () -> None
output_file = path.join(self.outdir, 'c.txt')
with open(output_file, 'w') as op:
if self.config.coverage_write_headline:
@ -117,6 +129,7 @@ class CoverageBuilder(Builder):
op.write('\n')
def build_py_coverage(self):
# type: () -> None
objects = self.env.domaindata['py']['objects']
modules = self.env.domaindata['py']['modules']
@ -134,13 +147,12 @@ class CoverageBuilder(Builder):
try:
mod = __import__(mod_name, fromlist=['foo'])
except ImportError as err:
self.warn('module %s could not be imported: %s' %
(mod_name, err))
logger.warning('module %s could not be imported: %s', mod_name, err)
self.py_undoc[mod_name] = {'error': err}
continue
funcs = []
classes = {}
classes = {} # type: Dict[unicode, List[unicode]]
for name, obj in inspect.getmembers(mod):
# diverse module attributes are ignored:
@ -177,7 +189,7 @@ class CoverageBuilder(Builder):
classes[name] = []
continue
attrs = []
attrs = [] # type: List[unicode]
for attr_name in dir(obj):
if attr_name not in obj.__dict__:
@ -207,6 +219,7 @@ class CoverageBuilder(Builder):
self.py_undoc[mod_name] = {'funcs': funcs, 'classes': classes}
def write_py_coverage(self):
# type: () -> None
output_file = path.join(self.outdir, 'python.txt')
failed = []
with open(output_file, 'w') as op:
@ -242,6 +255,7 @@ class CoverageBuilder(Builder):
op.writelines(' * %s -- %s\n' % x for x in failed)
def finish(self):
# type: () -> None
# dump the coverage data to a pickle file too
picklepath = path.join(self.outdir, 'undoc.pickle')
with open(picklepath, 'wb') as dumpfile:
@ -249,6 +263,7 @@ class CoverageBuilder(Builder):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_builder(CoverageBuilder)
app.add_config_value('coverage_ignore_modules', [], False)
app.add_config_value('coverage_ignore_functions', [], False)

164
sphinx/ext/doctest.py
View File

@ -15,26 +15,37 @@ import re
import sys
import time
import codecs
import platform
from os import path
import doctest
from six import itervalues, StringIO, binary_type, text_type, PY2
from distutils.version import LooseVersion
from docutils import nodes
from docutils.parsers.rst import directives
from docutils.parsers.rst import Directive, directives
import sphinx
from sphinx.builders import Builder
from sphinx.util import force_decode
from sphinx.util import force_decode, logging
from sphinx.util.nodes import set_source_info
from sphinx.util.compat import Directive
from sphinx.util.console import bold
from sphinx.util.console import bold # type: ignore
from sphinx.util.osutil import fs_encoding
from sphinx.locale import _
if False:
# For type annotation
from typing import Any, Callable, IO, Iterable, Sequence, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
blankline_re = re.compile(r'^\s*<BLANKLINE>', re.MULTILINE)
doctestopt_re = re.compile(r'#\s*doctest:.+$', re.MULTILINE)
if PY2:
def doctest_encode(text, encoding):
# type: (str, unicode) -> unicode
if isinstance(text, text_type):
text = text.encode(encoding)
if text.startswith(codecs.BOM_UTF8):
@ -42,9 +53,34 @@ if PY2:
return text
else:
def doctest_encode(text, encoding):
# type: (unicode, unicode) -> unicode
return text
def compare_version(ver1, ver2, operand):
# type: (unicode, unicode, unicode) -> bool
"""Compare `ver1` to `ver2`, relying on `operand`.
Some examples:
>>> compare_version('3.3', '3.5', '<=')
True
>>> compare_version('3.3', '3.2', '<=')
False
>>> compare_version('3.3a0', '3.3', '<=')
True
"""
if operand not in ('<=', '<', '==', '>=', '>'):
raise ValueError("'%s' is not a valid operand.")
v1 = LooseVersion(ver1)
v2 = LooseVersion(ver2)
return ((operand == '<=' and (v1 <= v2)) or
(operand == '<' and (v1 < v2)) or
(operand == '==' and (v1 == v2)) or
(operand == '>=' and (v1 >= v2)) or
(operand == '>' and (v1 > v2)))
# set up the necessary directives
class TestDirective(Directive):
@ -58,6 +94,7 @@ class TestDirective(Directive):
final_argument_whitespace = True
def run(self):
# type: () -> List[nodes.Node]
# use ordinary docutils nodes for test code: they get special attributes
# so that our builder recognizes them, and the other builders are happy.
code = '\n'.join(self.content)
@ -91,33 +128,55 @@ class TestDirective(Directive):
# parse doctest-like output comparison flags
option_strings = self.options['options'].replace(',', ' ').split()
for option in option_strings:
if (option[0] not in '+-' or option[1:] not in
doctest.OPTIONFLAGS_BY_NAME):
# XXX warn?
prefix, option_name = option[0], option[1:]
if prefix not in '+-':
self.state.document.reporter.warning(
_("missing '+' or '-' in '%s' option.") % option,
line=self.lineno)
continue
flag = doctest.OPTIONFLAGS_BY_NAME[option[1:]]
if option_name not in doctest.OPTIONFLAGS_BY_NAME: # type: ignore
self.state.document.reporter.warning(
_("'%s' is not a valid option.") % option_name,
line=self.lineno)
continue
flag = doctest.OPTIONFLAGS_BY_NAME[option[1:]] # type: ignore
node['options'][flag] = (option[0] == '+')
if self.name == 'doctest' and 'pyversion' in self.options:
try:
option = self.options['pyversion']
# :pyversion: >= 3.6 --> operand='>=', option_version='3.6'
operand, option_version = [item.strip() for item in option.split()]
running_version = platform.python_version()
if not compare_version(running_version, option_version, operand):
flag = doctest.OPTIONFLAGS_BY_NAME['SKIP'] # type: ignore
node['options'][flag] = True # Skip the test
except ValueError:
self.state.document.reporter.warning(
_("'%s' is not a valid pyversion option") % option,
line=self.lineno)
return [node]
class TestsetupDirective(TestDirective):
option_spec = {}
option_spec = {} # type: Dict
class TestcleanupDirective(TestDirective):
option_spec = {}
option_spec = {} # type: Dict
class DoctestDirective(TestDirective):
option_spec = {
'hide': directives.flag,
'options': directives.unchanged,
'pyversion': directives.unchanged_required,
}
class TestcodeDirective(TestDirective):
option_spec = {
'hide': directives.flag,
'pyversion': directives.unchanged_required,
}
@ -125,22 +184,25 @@ class TestoutputDirective(TestDirective):
option_spec = {
'hide': directives.flag,
'options': directives.unchanged,
'pyversion': directives.unchanged_required,
}
parser = doctest.DocTestParser()
parser = doctest.DocTestParser() # type: ignore
# helper classes
class TestGroup(object):
def __init__(self, name):
# type: (unicode) -> None
self.name = name
self.setup = []
self.tests = []
self.cleanup = []
self.setup = [] # type: List[TestCode]
self.tests = [] # type: List[List[TestCode]]
self.cleanup = [] # type: List[TestCode]
def add_code(self, code, prepend=False):
# type: (TestCode, bool) -> None
if code.type == 'testsetup':
if prepend:
self.setup.insert(0, code)
@ -158,30 +220,34 @@ class TestGroup(object):
else:
raise RuntimeError('invalid TestCode type')
def __repr__(self):
def __repr__(self): # type: ignore
# type: () -> unicode
return 'TestGroup(name=%r, setup=%r, cleanup=%r, tests=%r)' % (
self.name, self.setup, self.cleanup, self.tests)
class TestCode(object):
def __init__(self, code, type, lineno, options=None):
# type: (unicode, unicode, int, Dict) -> None
self.code = code
self.type = type
self.lineno = lineno
self.options = options or {}
def __repr__(self):
def __repr__(self): # type: ignore
# type: () -> unicode
return 'TestCode(%r, %r, %r, options=%r)' % (
self.code, self.type, self.lineno, self.options)
class SphinxDocTestRunner(doctest.DocTestRunner):
class SphinxDocTestRunner(doctest.DocTestRunner): # type: ignore
def summarize(self, out, verbose=None):
# type: (Callable, bool) -> Tuple[int, int]
string_io = StringIO()
old_stdout = sys.stdout
sys.stdout = string_io
try:
res = doctest.DocTestRunner.summarize(self, verbose)
res = doctest.DocTestRunner.summarize(self, verbose) # type: ignore
finally:
sys.stdout = old_stdout
out(string_io.getvalue())
@ -189,6 +255,7 @@ class SphinxDocTestRunner(doctest.DocTestRunner):
def _DocTestRunner__patched_linecache_getlines(self, filename,
module_globals=None):
# type: (unicode, Any) -> Any
# this is overridden from DocTestRunner adding the try-except below
m = self._DocTestRunner__LINECACHE_FILENAME_RE.match(filename)
if m and m.group('name') == self.test.name:
@ -213,6 +280,7 @@ class DocTestBuilder(Builder):
name = 'doctest'
def init(self):
# type: () -> None
# default options
self.opt = self.config.doctest_default_flags
@ -221,7 +289,7 @@ class DocTestBuilder(Builder):
# for doctest examples but unusable for multi-statement code such
# as setup code -- to be able to use doctest error reporting with
# that code nevertheless, we monkey-patch the "compile" it uses.
doctest.compile = self.compile
doctest.compile = self.compile # type: ignore
sys.path[0:0] = self.config.doctest_path
@ -236,7 +304,8 @@ class DocTestBuilder(Builder):
date = time.strftime('%Y-%m-%d %H:%M:%S')
self.outfile = codecs.open(path.join(self.outdir, 'output.txt'),
self.outfile = None # type: IO
self.outfile = codecs.open(path.join(self.outdir, 'output.txt'), # type: ignore
'w', encoding='utf-8')
self.outfile.write('''\
Results of doctest builder run on %s
@ -244,27 +313,33 @@ Results of doctest builder run on %s
''' % (date, '=' * len(date)))
def _out(self, text):
self.info(text, nonl=True)
# type: (unicode) -> None
logger.info(text, nonl=True)
self.outfile.write(text)
def _warn_out(self, text):
# type: (unicode) -> None
if self.app.quiet or self.app.warningiserror:
self.warn(text)
logger.warning(text)
else:
self.info(text, nonl=True)
logger.info(text, nonl=True)
if isinstance(text, binary_type):
text = force_decode(text, None)
self.outfile.write(text)
def get_target_uri(self, docname, typ=None):
# type: (unicode, unicode) -> unicode
return ''
def get_outdated_docs(self):
# type: () -> Set[unicode]
return self.env.found_docs
def finish(self):
# type: () -> None
# write executive summary
def s(v):
# type: (int) -> unicode
return v != 1 and 's' or ''
repl = (self.total_tries, s(self.total_tries),
self.total_failures, s(self.total_failures),
@ -284,17 +359,19 @@ Doctest summary
self.app.statuscode = 1
def write(self, build_docnames, updated_docnames, method='update'):
# type: (Iterable[unicode], Sequence[unicode], unicode) -> None
if build_docnames is None:
build_docnames = sorted(self.env.all_docs)
self.info(bold('running tests...'))
logger.info(bold('running tests...'))
for docname in build_docnames:
# no need to resolve the doctree
doctree = self.env.get_doctree(docname)
self.test_doc(docname, doctree)
def test_doc(self, docname, doctree):
groups = {}
# type: (unicode, nodes.Node) -> None
groups = {} # type: Dict[unicode, TestGroup]
add_to_all_groups = []
self.setup_runner = SphinxDocTestRunner(verbose=False,
optionflags=self.opt)
@ -308,19 +385,21 @@ Doctest summary
if self.config.doctest_test_doctest_blocks:
def condition(node):
# type: (nodes.Node) -> bool
return (isinstance(node, (nodes.literal_block, nodes.comment)) and
'testnodetype' in node) or \
isinstance(node, nodes.doctest_block)
else:
def condition(node):
# type: (nodes.Node) -> bool
return isinstance(node, (nodes.literal_block, nodes.comment)) \
and 'testnodetype' in node
for node in doctree.traverse(condition):
source = 'test' in node and node['test'] or node.astext()
if not source:
self.warn('no code/output in %s block at %s:%s' %
(node.get('testnodetype', 'doctest'),
self.env.doc2path(docname), node.line))
logger.warning('no code/output in %s block at %s:%s',
node.get('testnodetype', 'doctest'),
self.env.doc2path(docname), node.line)
code = TestCode(source, type=node.get('testnodetype', 'doctest'),
lineno=node.line, options=node.get('options'))
node_groups = node.get('groups', ['default'])
@ -366,26 +445,29 @@ Doctest summary
self.cleanup_tries += res_t
def compile(self, code, name, type, flags, dont_inherit):
# type: (unicode, unicode, unicode, Any, bool) -> Any
return compile(code, name, self.type, flags, dont_inherit)
def test_group(self, group, filename):
# type: (TestGroup, unicode) -> None
if PY2:
filename_str = filename.encode(fs_encoding)
else:
filename_str = filename
ns = {}
ns = {} # type: Dict
def run_setup_cleanup(runner, testcodes, what):
# type: (Any, List[TestCode], Any) -> bool
examples = []
for testcode in testcodes:
examples.append(doctest.Example(
doctest_encode(testcode.code, self.env.config.source_encoding), '',
examples.append(doctest.Example( # type: ignore
doctest_encode(testcode.code, self.env.config.source_encoding), '', # type: ignore # NOQA
lineno=testcode.lineno))
if not examples:
return True
# simulate a doctest with the code
sim_doctest = doctest.DocTest(examples, {},
sim_doctest = doctest.DocTest(examples, {}, # type: ignore
'%s (%s code)' % (group.name, what),
filename_str, 0, None)
sim_doctest.globs = ns
@ -407,12 +489,11 @@ Doctest summary
# ordinary doctests (code/output interleaved)
try:
test = parser.get_doctest(
doctest_encode(code[0].code, self.env.config.source_encoding), {},
doctest_encode(code[0].code, self.env.config.source_encoding), {}, # type: ignore # NOQA
group.name, filename_str, code[0].lineno)
except Exception:
self.warn('ignoring invalid doctest code: %r' %
code[0].code,
'%s:%s' % (filename, code[0].lineno))
logger.warning('ignoring invalid doctest code: %r', code[0].code,
location=(filename, code[0].lineno))
continue
if not test.examples:
continue
@ -427,19 +508,19 @@ Doctest summary
output = code[1] and code[1].code or ''
options = code[1] and code[1].options or {}
# disable <BLANKLINE> processing as it is not needed
options[doctest.DONT_ACCEPT_BLANKLINE] = True
options[doctest.DONT_ACCEPT_BLANKLINE] = True # type: ignore
# find out if we're testing an exception
m = parser._EXCEPTION_RE.match(output)
if m:
exc_msg = m.group('msg')
else:
exc_msg = None
example = doctest.Example(
doctest_encode(code[0].code, self.env.config.source_encoding), output,
example = doctest.Example( # type: ignore
doctest_encode(code[0].code, self.env.config.source_encoding), output, # type: ignore # NOQA
exc_msg=exc_msg,
lineno=code[0].lineno,
options=options)
test = doctest.DocTest([example], {}, group.name,
test = doctest.DocTest([example], {}, group.name, # type: ignore
filename_str, code[0].lineno, None)
self.type = 'exec' # multiple statements again
# DocTest.__init__ copies the globs namespace, which we don't want
@ -452,6 +533,7 @@ Doctest summary
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_directive('testsetup', TestsetupDirective)
app.add_directive('testcleanup', TestcleanupDirective)
app.add_directive('doctest', DoctestDirective)
@ -465,6 +547,6 @@ def setup(app):
app.add_config_value('doctest_global_cleanup', '', False)
app.add_config_value(
'doctest_default_flags',
doctest.DONT_ACCEPT_TRUE_FOR_1 | doctest.ELLIPSIS | doctest.IGNORE_EXCEPTION_DETAIL,
doctest.DONT_ACCEPT_TRUE_FOR_1 | doctest.ELLIPSIS | doctest.IGNORE_EXCEPTION_DETAIL, # type: ignore # NOQA
False)
return {'version': sphinx.__display_version__, 'parallel_read_safe': True}

64
sphinx/ext/graphviz.py
View File

@ -18,16 +18,24 @@ from subprocess import Popen, PIPE
from hashlib import sha1
from six import text_type
from docutils import nodes
from docutils.parsers.rst import directives
from docutils.parsers.rst import Directive, directives
from docutils.statemachine import ViewList
import sphinx
from sphinx.errors import SphinxError
from sphinx.locale import _
from sphinx.util import logging
from sphinx.util.i18n import search_image_for_language
from sphinx.util.osutil import ensuredir, ENOENT, EPIPE, EINVAL
from sphinx.util.compat import Directive
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
logger = logging.getLogger(__name__)
mapname_re = re.compile(r'<map id="(.*?)"')
@ -42,6 +50,7 @@ class graphviz(nodes.General, nodes.Inline, nodes.Element):
def figure_wrapper(directive, node, caption):
# type: (Directive, nodes.Node, unicode) -> nodes.figure
figure_node = nodes.figure('', node)
if 'align' in node:
figure_node['align'] = node.attributes.pop('align')
@ -58,6 +67,7 @@ def figure_wrapper(directive, node, caption):
def align_spec(argument):
# type: (Any) -> bool
return directives.choice(argument, ('left', 'center', 'right'))
@ -72,12 +82,13 @@ class Graphviz(Directive):
option_spec = {
'alt': directives.unchanged,
'align': align_spec,
'inline': directives.flag,
'caption': directives.unchanged,
'graphviz_dot': directives.unchanged,
'name': directives.unchanged,
}
def run(self):
# type: () -> List[nodes.Node]
if self.arguments:
document = self.state.document
if self.content:
@ -110,13 +121,12 @@ class Graphviz(Directive):
node['alt'] = self.options['alt']
if 'align' in self.options:
node['align'] = self.options['align']
if 'inline' in self.options:
node['inline'] = True
caption = self.options.get('caption')
if caption:
node = figure_wrapper(self, node, caption)
self.add_name(node)
return [node]
@ -131,12 +141,13 @@ class GraphvizSimple(Directive):
option_spec = {
'alt': directives.unchanged,
'align': align_spec,
'inline': directives.flag,
'caption': directives.unchanged,
'graphviz_dot': directives.unchanged,
'name': directives.unchanged,
}
def run(self):
# type: () -> List[nodes.Node]
node = graphviz()
node['code'] = '%s %s {\n%s\n}\n' % \
(self.name, self.arguments[0], '\n'.join(self.content))
@ -147,17 +158,17 @@ class GraphvizSimple(Directive):
node['alt'] = self.options['alt']
if 'align' in self.options:
node['align'] = self.options['align']
if 'inline' in self.options:
node['inline'] = True
caption = self.options.get('caption')
if caption:
node = figure_wrapper(self, node, caption)
self.add_name(node)
return [node]
def render_dot(self, code, options, format, prefix='graphviz'):
# type: (nodes.NodeVisitor, unicode, Dict, unicode, unicode) -> Tuple[unicode, unicode]
"""Render graphviz code into a PNG or PDF output file."""
graphviz_dot = options.get('graphviz_dot', self.builder.config.graphviz_dot)
hashkey = (code + str(options) + str(graphviz_dot) +
@ -190,8 +201,8 @@ def render_dot(self, code, options, format, prefix='graphviz'):
except OSError as err:
if err.errno != ENOENT: # No such file or directory
raise
self.builder.warn('dot command %r cannot be run (needed for graphviz '
'output), check the graphviz_dot setting' % graphviz_dot)
logger.warning('dot command %r cannot be run (needed for graphviz '
'output), check the graphviz_dot setting', graphviz_dot)
if not hasattr(self.builder, '_graphviz_warned_dot'):
self.builder._graphviz_warned_dot = {}
self.builder._graphviz_warned_dot[graphviz_dot] = True
@ -216,17 +227,9 @@ def render_dot(self, code, options, format, prefix='graphviz'):
return relfn, outfn
def warn_for_deprecated_option(self, node):
if hasattr(self.builder, '_graphviz_warned_inline'):
return
if 'inline' in node:
self.builder.warn(':inline: option for graphviz is deprecated since version 1.4.0.')
self.builder._graphviz_warned_inline = True
def render_dot_html(self, node, code, options, prefix='graphviz',
imgcls=None, alt=None):
# type: (nodes.NodeVisitor, graphviz, unicode, Dict, unicode, unicode, unicode) -> Tuple[unicode, unicode] # NOQA
format = self.builder.config.graphviz_output_format
try:
if format not in ('png', 'svg'):
@ -234,7 +237,7 @@ def render_dot_html(self, node, code, options, prefix='graphviz',
"'svg', but is %r" % format)
fname, outfn = render_dot(self, code, options, format, prefix)
except GraphvizError as exc:
self.builder.warn('dot code %r: ' % code + str(exc))
logger.warning('dot code %r: ' % code + str(exc))
raise nodes.SkipNode
if fname is None:
@ -259,7 +262,7 @@ def render_dot_html(self, node, code, options, prefix='graphviz',
(fname, alt, imgcss))
else:
# has a map: get the name of the map and connect the parts
mapname = mapname_re.match(imgmap[0].decode('utf-8')).group(1)
mapname = mapname_re.match(imgmap[0].decode('utf-8')).group(1) # type: ignore
self.body.append('<img src="%s" alt="%s" usemap="#%s" %s/>\n' %
(fname, alt, mapname, imgcss))
self.body.extend([item.decode('utf-8') for item in imgmap])
@ -270,15 +273,16 @@ def render_dot_html(self, node, code, options, prefix='graphviz',
def html_visit_graphviz(self, node):
warn_for_deprecated_option(self, node)
# type: (nodes.NodeVisitor, graphviz) -> None
render_dot_html(self, node, node['code'], node['options'])
def render_dot_latex(self, node, code, options, prefix='graphviz'):
# type: (nodes.NodeVisitor, graphviz, unicode, Dict, unicode) -> None
try:
fname, outfn = render_dot(self, code, options, 'pdf', prefix)
except GraphvizError as exc:
self.builder.warn('dot code %r: ' % code + str(exc))
logger.warning('dot code %r: ' % code + str(exc))
raise nodes.SkipNode
is_inline = self.is_inline(node)
@ -288,7 +292,7 @@ def render_dot_latex(self, node, code, options, prefix='graphviz'):
para_separator = '\n'
if fname is not None:
post = None
post = None # type: unicode
if not is_inline and 'align' in node:
if node['align'] == 'left':
self.body.append('{')
@ -305,15 +309,16 @@ def render_dot_latex(self, node, code, options, prefix='graphviz'):
def latex_visit_graphviz(self, node):
warn_for_deprecated_option(self, node)
# type: (nodes.NodeVisitor, graphviz) -> None
render_dot_latex(self, node, node['code'], node['options'])
def render_dot_texinfo(self, node, code, options, prefix='graphviz'):
# type: (nodes.NodeVisitor, graphviz, unicode, Dict, unicode) -> None
try:
fname, outfn = render_dot(self, code, options, 'png', prefix)
except GraphvizError as exc:
self.builder.warn('dot code %r: ' % code + str(exc))
logger.warning('dot code %r: ' % code + str(exc))
raise nodes.SkipNode
if fname is not None:
self.body.append('@image{%s,,,[graphviz],png}\n' % fname[:-4])
@ -321,12 +326,12 @@ def render_dot_texinfo(self, node, code, options, prefix='graphviz'):
def texinfo_visit_graphviz(self, node):
warn_for_deprecated_option(self, node)
# type: (nodes.NodeVisitor, graphviz) -> None
render_dot_texinfo(self, node, node['code'], node['options'])
def text_visit_graphviz(self, node):
warn_for_deprecated_option(self, node)
# type: (nodes.NodeVisitor, graphviz) -> None
if 'alt' in node.attributes:
self.add_text(_('[graph: %s]') % node['alt'])
else:
@ -335,7 +340,7 @@ def text_visit_graphviz(self, node):
def man_visit_graphviz(self, node):
warn_for_deprecated_option(self, node)
# type: (nodes.NodeVisitor, graphviz) -> None
if 'alt' in node.attributes:
self.body.append(_('[graph: %s]') % node['alt'])
else:
@ -344,6 +349,7 @@ def man_visit_graphviz(self, node):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_node(graphviz,
html=(html_visit_graphviz, None),
latex=(latex_visit_graphviz, None),

16
sphinx/ext/ifconfig.py
View File

@ -21,10 +21,15 @@
"""
from docutils import nodes
from docutils.parsers.rst import Directive
import sphinx
from sphinx.util.nodes import set_source_info
from sphinx.util.compat import Directive
if False:
# For type annotation
from typing import Any # NOQA
from sphinx.application import Sphinx # NOQA
class ifconfig(nodes.Element):
@ -37,9 +42,10 @@ class IfConfig(Directive):
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = True
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[nodes.Node]
node = ifconfig()
node.document = self.state.document
set_source_info(self, node)
@ -50,7 +56,8 @@ class IfConfig(Directive):
def process_ifconfig_nodes(app, doctree, docname):
ns = dict((k, app.config[k]) for k in app.config.values)
# type: (Sphinx, nodes.Node, unicode) -> None
ns = dict((confval.name, confval.value) for confval in app.config) # type: ignore
ns.update(app.config.__dict__.copy())
ns['builder'] = app.builder.name
for node in doctree.traverse(ifconfig):
@ -59,7 +66,7 @@ def process_ifconfig_nodes(app, doctree, docname):
except Exception as err:
# handle exceptions in a clean fashion
from traceback import format_exception_only
msg = ''.join(format_exception_only(err.__class__, err))
msg = ''.join(format_exception_only(err.__class__, err)) # type: ignore
newnode = doctree.reporter.error('Exception occured in '
'ifconfig expression: \n%s' %
msg, base_node=node)
@ -72,6 +79,7 @@ def process_ifconfig_nodes(app, doctree, docname):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_node(ifconfig)
app.add_directive('ifconfig', IfConfig)
app.connect('doctree-resolved', process_ifconfig_nodes)

42
sphinx/ext/imgmath.py
View File

@ -19,21 +19,32 @@ from subprocess import Popen, PIPE
from hashlib import sha1
from six import text_type
from docutils import nodes
import sphinx
from sphinx.locale import _
from sphinx.errors import SphinxError, ExtensionError
from sphinx.util import logging
from sphinx.util.png import read_png_depth, write_png_depth
from sphinx.util.osutil import ensuredir, ENOENT, cd
from sphinx.util.pycompat import sys_encoding
from sphinx.ext.mathbase import setup_math as mathbase_setup, wrap_displaymath
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.ext.mathbase import math as math_node, displaymath # NOQA
logger = logging.getLogger(__name__)
class MathExtError(SphinxError):
category = 'Math extension error'
def __init__(self, msg, stderr=None, stdout=None):
# type: (unicode, unicode, unicode) -> None
if stderr:
msg += '\n[stderr]\n' + stderr.decode(sys_encoding, 'replace')
if stdout:
@ -72,6 +83,7 @@ depth_re = re.compile(br'\[\d+ depth=(-?\d+)\]')
def render_math(self, math):
# type: (nodes.NodeVisitor, unicode) -> Tuple[unicode, int]
"""Render the LaTeX math expression *math* using latex and dvipng or
dvisvgm.
@ -116,9 +128,8 @@ def render_math(self, math):
else:
tempdir = self.builder._imgmath_tempdir
tf = codecs.open(path.join(tempdir, 'math.tex'), 'w', 'utf-8')
tf.write(latex)
tf.close()
with codecs.open(path.join(tempdir, 'math.tex'), 'w', 'utf-8') as tf:
tf.write(latex)
# build latex command; old versions of latex don't have the
# --output-directory option, so we have to manually chdir to the
@ -134,9 +145,9 @@ def render_math(self, math):
except OSError as err:
if err.errno != ENOENT: # No such file or directory
raise
self.builder.warn('LaTeX command %r cannot be run (needed for math '
'display), check the imgmath_latex setting' %
self.builder.config.imgmath_latex)
logger.warning('LaTeX command %r cannot be run (needed for math '
'display), check the imgmath_latex setting',
self.builder.config.imgmath_latex)
self.builder._imgmath_warned_latex = True
return None, None
@ -175,10 +186,10 @@ def render_math(self, math):
except OSError as err:
if err.errno != ENOENT: # No such file or directory
raise
self.builder.warn('%s command %r cannot be run (needed for math '
'display), check the imgmath_%s setting' %
(image_translator, image_translator_executable,
image_translator))
logger.warning('%s command %r cannot be run (needed for math '
'display), check the imgmath_%s setting',
image_translator, image_translator_executable,
image_translator)
self.builder._imgmath_warned_image_translator = True
return None, None
@ -199,23 +210,26 @@ def render_math(self, math):
def cleanup_tempdir(app, exc):
# type: (Sphinx, Exception) -> None
if exc:
return
if not hasattr(app.builder, '_imgmath_tempdir'):
return
try:
shutil.rmtree(app.builder._mathpng_tempdir)
shutil.rmtree(app.builder._mathpng_tempdir) # type: ignore
except Exception:
pass
def get_tooltip(self, node):
# type: (nodes.NodeVisitor, math_node) -> unicode
if self.builder.config.imgmath_add_tooltips:
return ' alt="%s"' % self.encode(node['latex']).strip()
return ''
def html_visit_math(self, node):
# type: (nodes.NodeVisitor, math_node) -> None
try:
fname, depth = render_math(self, '$' + node['latex'] + '$')
except MathExtError as exc:
@ -223,7 +237,7 @@ def html_visit_math(self, node):
sm = nodes.system_message(msg, type='WARNING', level=2,
backrefs=[], source=node['latex'])
sm.walkabout(self)
self.builder.warn('display latex %r: ' % node['latex'] + msg)
logger.warning('display latex %r: %s', node['latex'], msg)
raise nodes.SkipNode
if fname is None:
# something failed -- use text-only as a bad substitute
@ -238,6 +252,7 @@ def html_visit_math(self, node):
def html_visit_displaymath(self, node):
# type: (nodes.NodeVisitor, displaymath) -> None
if node['nowrap']:
latex = node['latex']
else:
@ -250,7 +265,7 @@ def html_visit_displaymath(self, node):
sm = nodes.system_message(msg, type='WARNING', level=2,
backrefs=[], source=node['latex'])
sm.walkabout(self)
self.builder.warn('inline latex %r: ' % node['latex'] + msg)
logger.warning('inline latex %r: %s', node['latex'], msg)
raise nodes.SkipNode
self.body.append(self.starttag(node, 'div', CLASS='math'))
self.body.append('<p>')
@ -269,6 +284,7 @@ def html_visit_displaymath(self, node):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
try:
mathbase_setup(app, (html_visit_math, None), (html_visit_displaymath, None))
except ExtensionError:

43
sphinx/ext/inheritance_diagram.py
View File

@ -42,20 +42,25 @@ import inspect
try:
from hashlib import md5
except ImportError:
from md5 import md5
from md5 import md5 # type: ignore
from six import text_type
from six.moves import builtins
from six.moves import builtins # type: ignore
from docutils import nodes
from docutils.parsers.rst import directives
from docutils.parsers.rst import Directive, directives
import sphinx
from sphinx.ext.graphviz import render_dot_html, render_dot_latex, \
render_dot_texinfo, figure_wrapper
from sphinx.pycode import ModuleAnalyzer
from sphinx.util import force_decode
from sphinx.util.compat import Directive
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
module_sig_re = re.compile(r'''^(?:([\w.]*)\.)? # module names
@ -64,6 +69,7 @@ module_sig_re = re.compile(r'''^(?:([\w.]*)\.)? # module names
def try_import(objname):
# type: (unicode) -> Any
"""Import a object or module using *name* and *currentmodule*.
*name* should be a relative name from *currentmodule* or
a fully-qualified name.
@ -72,9 +78,9 @@ def try_import(objname):
"""
try:
__import__(objname)
return sys.modules.get(objname)
return sys.modules.get(objname) # type: ignore
except ImportError:
modname, attrname = module_sig_re.match(objname).groups()
modname, attrname = module_sig_re.match(objname).groups() # type: ignore
if modname is None:
return None
try:
@ -85,6 +91,7 @@ def try_import(objname):
def import_classes(name, currmodule):
# type: (unicode, unicode) -> Any
"""Import a class using its fully-qualified *name*."""
target = None
@ -127,6 +134,7 @@ class InheritanceGraph(object):
"""
def __init__(self, class_names, currmodule, show_builtins=False,
private_bases=False, parts=0):
# type: (unicode, str, bool, bool, int) -> None
"""*class_names* is a list of child classes to show bases from.
If *show_builtins* is True, then Python builtins will be shown
@ -141,13 +149,15 @@ class InheritanceGraph(object):
'inheritance diagram')
def _import_classes(self, class_names, currmodule):
# type: (unicode, str) -> List[Any]
"""Import a list of classes."""
classes = []
classes = [] # type: List[Any]
for name in class_names:
classes.extend(import_classes(name, currmodule))
return classes
def _class_info(self, classes, show_builtins, private_bases, parts):
# type: (List[Any], bool, bool, int) -> List[Tuple[unicode, unicode, List[unicode], unicode]] # NOQA
"""Return name and bases for all classes that are ancestors of
*classes*.
@ -158,6 +168,7 @@ class InheritanceGraph(object):
py_builtins = vars(builtins).values()
def recurse(cls):
# type: (Any) -> None
if not show_builtins and cls in py_builtins:
return
if not private_bases and cls.__name__.startswith('_'):
@ -179,7 +190,7 @@ class InheritanceGraph(object):
except Exception: # might raise AttributeError for strange classes
pass
baselist = []
baselist = [] # type: List[unicode]
all_classes[cls] = (nodename, fullname, baselist, tooltip)
for base in cls.__bases__:
if not show_builtins and base in py_builtins:
@ -196,6 +207,7 @@ class InheritanceGraph(object):
return list(all_classes.values())
def class_name(self, cls, parts=0):
# type: (Any, int) -> unicode
"""Given a class object, return a fully-qualified name.
This works for things I've tested in matplotlib so far, but may not be
@ -212,8 +224,9 @@ class InheritanceGraph(object):
return '.'.join(name_parts[-parts:])
def get_all_class_names(self):
# type: () -> List[unicode]
"""Get all of the class names involved in the graph."""
return [fullname for (_, fullname, _, _) in self.class_info]
return [fullname for (_, fullname, _, _) in self.class_info] # type: ignore
# These are the default attrs for graphviz
default_graph_attrs = {
@ -234,13 +247,16 @@ class InheritanceGraph(object):
}
def _format_node_attrs(self, attrs):
# type: (Dict) -> unicode
return ','.join(['%s=%s' % x for x in sorted(attrs.items())])
def _format_graph_attrs(self, attrs):
# type: (Dict) -> unicode
return ''.join(['%s=%s;\n' % x for x in sorted(attrs.items())])
def generate_dot(self, name, urls={}, env=None,
graph_attrs={}, node_attrs={}, edge_attrs={}):
# type: (unicode, Dict, BuildEnvironment, Dict, Dict, Dict) -> unicode
"""Generate a graphviz dot graph from the classes that were passed in
to __init__.
@ -262,7 +278,7 @@ class InheritanceGraph(object):
n_attrs.update(env.config.inheritance_node_attrs)
e_attrs.update(env.config.inheritance_edge_attrs)
res = []
res = [] # type: List[unicode]
res.append('digraph %s {\n' % name)
res.append(self._format_graph_attrs(g_attrs))
@ -308,6 +324,7 @@ class InheritanceDiagram(Directive):
}
def run(self):
# type: () -> List[nodes.Node]
node = inheritance_diagram()
node.document = self.state.document
env = self.state.document.settings.env
@ -347,11 +364,13 @@ class InheritanceDiagram(Directive):
def get_graph_hash(node):
# type: (inheritance_diagram) -> unicode
encoded = (node['content'] + str(node['parts'])).encode('utf-8')
return md5(encoded).hexdigest()[-10:]
def html_visit_inheritance_diagram(self, node):
# type: (nodes.NodeVisitor, inheritance_diagram) -> None
"""
Output the graph for HTML. This will insert a PNG with clickable
image map.
@ -384,6 +403,7 @@ def html_visit_inheritance_diagram(self, node):
def latex_visit_inheritance_diagram(self, node):
# type: (nodes.NodeVisitor, inheritance_diagram) -> None
"""
Output the graph for LaTeX. This will insert a PDF.
"""
@ -399,6 +419,7 @@ def latex_visit_inheritance_diagram(self, node):
def texinfo_visit_inheritance_diagram(self, node):
# type: (nodes.NodeVisitor, inheritance_diagram) -> None
"""
Output the graph for Texinfo. This will insert a PNG.
"""
@ -414,10 +435,12 @@ def texinfo_visit_inheritance_diagram(self, node):
def skip(self, node):
# type: (nodes.NodeVisitor, inheritance_diagram) -> None
raise nodes.SkipNode
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.setup_extension('sphinx.ext.graphviz')
app.add_node(
inheritance_diagram,

131
sphinx/ext/intersphinx.py
View File

@ -34,23 +34,69 @@ import posixpath
from os import path
import re
from six import iteritems, string_types
from six import PY3, iteritems, string_types
from six.moves.urllib.parse import urlsplit, urlunsplit
from docutils import nodes
from docutils.utils import relative_path
import sphinx
from sphinx.locale import _
from sphinx.builders.html import INVENTORY_FILENAME
from sphinx.util import requests
from sphinx.util import requests, logging
if False:
# For type annotation
from typing import Any, Callable, Dict, IO, Iterator, Tuple, Union # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.config import Config # NOQA
from sphinx.environment import BuildEnvironment # NOQA
if PY3:
unicode = str
Inventory = Dict[unicode, Dict[unicode, Tuple[unicode, unicode, unicode, unicode]]]
logger = logging.getLogger(__name__)
UTF8StreamReader = codecs.lookup('utf-8')[2]
class InventoryAdapter(object):
"""Inventory adapter for environment"""
def __init__(self, env):
self.env = env
if not hasattr(env, 'intersphinx_cache'):
self.env.intersphinx_cache = {}
self.env.intersphinx_inventory = {}
self.env.intersphinx_named_inventory = {}
@property
def cache(self):
# type: () -> Dict[unicode, Tuple[unicode, int, Inventory]]
return self.env.intersphinx_cache
@property
def main_inventory(self):
# type: () -> Inventory
return self.env.intersphinx_inventory
@property
def named_inventory(self):
# type: () -> Dict[unicode, Inventory]
return self.env.intersphinx_named_inventory
def clear(self):
self.env.intersphinx_inventory.clear()
self.env.intersphinx_named_inventory.clear()
def read_inventory_v1(f, uri, join):
# type: (IO, unicode, Callable) -> Inventory
f = UTF8StreamReader(f)
invdata = {}
invdata = {} # type: Inventory
line = next(f)
projname = line.rstrip()[11:]
line = next(f)
@ -70,7 +116,8 @@ def read_inventory_v1(f, uri, join):
def read_inventory_v2(f, uri, join, bufsize=16 * 1024):
invdata = {}
# type: (IO, unicode, Callable, int) -> Inventory
invdata = {} # type: Inventory
line = f.readline()
projname = line.rstrip()[11:].decode('utf-8')
line = f.readline()
@ -80,12 +127,14 @@ def read_inventory_v2(f, uri, join, bufsize=16 * 1024):
raise ValueError
def read_chunks():
# type: () -> Iterator[bytes]
decompressor = zlib.decompressobj()
for chunk in iter(lambda: f.read(bufsize), b''):
yield decompressor.decompress(chunk)
yield decompressor.flush()
def split_lines(iter):
# type: (Iterator[bytes]) -> Iterator[unicode]
buf = b''
for chunk in iter:
buf += chunk
@ -118,6 +167,7 @@ def read_inventory_v2(f, uri, join, bufsize=16 * 1024):
def read_inventory(f, uri, join, bufsize=16 * 1024):
# type: (IO, unicode, Callable, int) -> Inventory
line = f.readline().rstrip().decode('utf-8')
if line == '# Sphinx inventory version 1':
return read_inventory_v1(f, uri, join)
@ -126,6 +176,7 @@ def read_inventory(f, uri, join, bufsize=16 * 1024):
def _strip_basic_auth(url):
# type: (unicode) -> unicode
"""Returns *url* with basic auth credentials removed. Also returns the
basic auth username and password if they're present in *url*.
@ -147,6 +198,7 @@ def _strip_basic_auth(url):
def _read_from_url(url, config=None):
# type: (unicode, Config) -> IO
"""Reads data from *url* with an HTTP *GET*.
This function supports fetching from resources which use basic HTTP auth as
@ -172,6 +224,7 @@ def _read_from_url(url, config=None):
def _get_safe_url(url):
# type: (unicode) -> unicode
"""Gets version of *url* with basic auth passwords obscured. This function
returns results suitable for printing and logging.
@ -197,6 +250,7 @@ def _get_safe_url(url):
def fetch_inventory(app, uri, inv):
# type: (Sphinx, unicode, Any) -> Any
"""Fetch, parse and return an intersphinx inventory file."""
# both *uri* (base URI of the links to generate) and *inv* (actual
# location of the inventory file) can be local or remote URIs
@ -210,14 +264,14 @@ def fetch_inventory(app, uri, inv):
else:
f = open(path.join(app.srcdir, inv), 'rb')
except Exception as err:
app.warn('intersphinx inventory %r not fetchable due to '
'%s: %s' % (inv, err.__class__, err))
logger.warning('intersphinx inventory %r not fetchable due to %s: %s',
inv, err.__class__, err)
return
try:
if hasattr(f, 'url'):
newinv = f.url
newinv = f.url # type: ignore
if inv != newinv:
app.info('intersphinx inventory has moved: %s -> %s' % (inv, newinv))
logger.info('intersphinx inventory has moved: %s -> %s', inv, newinv)
if uri in (inv, path.dirname(inv), path.dirname(inv) + '/'):
uri = path.dirname(newinv)
@ -228,29 +282,29 @@ def fetch_inventory(app, uri, inv):
except ValueError as exc:
raise ValueError('unknown or unsupported inventory version: %r' % exc)
except Exception as err:
app.warn('intersphinx inventory %r not readable due to '
'%s: %s' % (inv, err.__class__.__name__, err))
logger.warning('intersphinx inventory %r not readable due to %s: %s',
inv, err.__class__.__name__, err)
else:
return invdata
def load_mappings(app):
# type: (Sphinx) -> None
"""Load all intersphinx mappings into the environment."""
now = int(time.time())
cache_time = now - app.config.intersphinx_cache_limit * 86400
env = app.builder.env
if not hasattr(env, 'intersphinx_cache'):
env.intersphinx_cache = {}
env.intersphinx_inventory = {}
env.intersphinx_named_inventory = {}
cache = env.intersphinx_cache
inventories = InventoryAdapter(app.builder.env)
update = False
for key, value in iteritems(app.config.intersphinx_mapping):
name = None # type: unicode
uri = None # type: unicode
inv = None # type: Union[unicode, Tuple[unicode, ...]]
if isinstance(value, (list, tuple)):
# new format
name, (uri, inv) = key, value
name, (uri, inv) = key, value # type: ignore
if not isinstance(name, string_types):
app.warn('intersphinx identifier %r is not string. Ignored' % name)
logger.warning('intersphinx identifier %r is not string. Ignored', name)
continue
else:
# old format, no name
@ -261,27 +315,26 @@ def load_mappings(app):
if not isinstance(inv, tuple):
invs = (inv, )
else:
invs = inv
invs = inv # type: ignore
for inv in invs:
if not inv:
inv = posixpath.join(uri, INVENTORY_FILENAME)
# decide whether the inventory must be read: always read local
# files; remote ones only if the cache time is expired
if '://' not in inv or uri not in cache \
or cache[uri][1] < cache_time:
safe_inv_url = _get_safe_url(inv)
app.info(
'loading intersphinx inventory from %s...' % safe_inv_url)
if '://' not in inv or uri not in inventories.cache \
or inventories.cache[uri][1] < cache_time:
safe_inv_url = _get_safe_url(inv) # type: ignore
logger.info('loading intersphinx inventory from %s...', safe_inv_url)
invdata = fetch_inventory(app, uri, inv)
if invdata:
cache[uri] = (name, now, invdata)
inventories.cache[uri] = (name, now, invdata)
update = True
break
if update:
env.intersphinx_inventory = {}
env.intersphinx_named_inventory = {}
inventories.clear()
# Duplicate values in different inventories will shadow each
# other; which one will override which can vary between builds
# since they are specified using an unordered dict. To make
@ -289,46 +342,45 @@ def load_mappings(app):
# add the unnamed inventories last. This means that the
# unnamed inventories will shadow the named ones but the named
# ones can still be accessed when the name is specified.
cached_vals = list(cache.values())
cached_vals = list(inventories.cache.values())
named_vals = sorted(v for v in cached_vals if v[0])
unnamed_vals = [v for v in cached_vals if not v[0]]
for name, _x, invdata in named_vals + unnamed_vals:
if name:
env.intersphinx_named_inventory[name] = invdata
inventories.named_inventory[name] = invdata
for type, objects in iteritems(invdata):
env.intersphinx_inventory.setdefault(
type, {}).update(objects)
inventories.main_inventory.setdefault(type, {}).update(objects)
def missing_reference(app, env, node, contnode):
# type: (Sphinx, BuildEnvironment, nodes.Node, nodes.Node) -> None
"""Attempt to resolve a missing reference via intersphinx references."""
target = node['reftarget']
inventories = InventoryAdapter(env)
objtypes = None # type: List[unicode]
if node['reftype'] == 'any':
# we search anything!
objtypes = ['%s:%s' % (domain.name, objtype)
for domain in env.domains.values()
for objtype in domain.object_types]
domain = None
elif node['reftype'] == 'doc':
domain = 'std' # special case
objtypes = ['std:doc']
else:
domain = node.get('refdomain')
if not domain:
# only objects in domains are in the inventory
return
objtypes = env.domains[domain].objtypes_for_role(node['reftype'])
objtypes = env.get_domain(domain).objtypes_for_role(node['reftype'])
if not objtypes:
return
objtypes = ['%s:%s' % (domain, objtype) for objtype in objtypes]
to_try = [(env.intersphinx_inventory, target)]
to_try = [(inventories.main_inventory, target)]
in_set = None
if ':' in target:
# first part may be the foreign doc set name
setname, newtarget = target.split(':', 1)
if setname in env.intersphinx_named_inventory:
if setname in inventories.named_inventory:
in_set = setname
to_try.append((env.intersphinx_named_inventory[setname], newtarget))
to_try.append((inventories.named_inventory[setname], newtarget))
for inventory, target in to_try:
for objtype in objtypes:
if objtype not in inventory or target not in inventory[objtype]:
@ -362,6 +414,7 @@ def missing_reference(app, env, node, contnode):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_config_value('intersphinx_mapping', {}, True)
app.add_config_value('intersphinx_cache_limit', 5, False)
app.add_config_value('intersphinx_timeout', None, False)
@ -381,7 +434,7 @@ if __name__ == '__main__':
print(msg, file=sys.stderr)
filename = sys.argv[1]
invdata = fetch_inventory(MockApp(), '', filename)
invdata = fetch_inventory(MockApp(), '', filename) # type: ignore
for key in sorted(invdata or {}):
print(key)
for entry, einfo in sorted(invdata[key].items()):

9
sphinx/ext/linkcode.py
View File

@ -16,12 +16,18 @@ from sphinx import addnodes
from sphinx.locale import _
from sphinx.errors import SphinxError
if False:
# For type annotation
from typing import Any # NOQA
from sphinx.application import Sphinx # NOQA
class LinkcodeError(SphinxError):
category = "linkcode error"
def doctree_read(app, doctree):
# type: (Sphinx, nodes.Node) -> None
env = app.builder.env
resolve_target = getattr(env.config, 'linkcode_resolve', None)
@ -38,7 +44,7 @@ def doctree_read(app, doctree):
for objnode in doctree.traverse(addnodes.desc):
domain = objnode.get('domain')
uris = set()
uris = set() # type: Set[unicode]
for signode in objnode:
if not isinstance(signode, addnodes.desc_signature):
continue
@ -72,6 +78,7 @@ def doctree_read(app, doctree):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.connect('doctree-read', doctree_read)
app.add_config_value('linkcode_resolve', None, '')
return {'version': sphinx.__display_version__, 'parallel_read_safe': True}

39
sphinx/ext/mathbase.py
View File

@ -10,13 +10,20 @@
"""
from docutils import nodes, utils
from docutils.parsers.rst import directives
from docutils.parsers.rst import Directive, directives
from sphinx.roles import XRefRole
from sphinx.locale import _
from sphinx.domains import Domain
from sphinx.util.nodes import make_refnode, set_source_info
from sphinx.util.compat import Directive
if False:
# For type annotation
from typing import Any, Callable, Iterable, Tuple # NOQA
from docutils.parsers.rst.states import Inliner # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
class math(nodes.Inline, nodes.TextElement):
@ -33,6 +40,7 @@ class eqref(nodes.Inline, nodes.TextElement):
class EqXRefRole(XRefRole):
def result_nodes(self, document, env, node, is_ref):
# type: (nodes.Node, BuildEnvironment, nodes.Node, bool) -> Tuple[List[nodes.Node], List[nodes.Node]] # NOQA
node['refdomain'] = 'math'
return [node], []
@ -44,22 +52,25 @@ class MathDomain(Domain):
initial_data = {
'objects': {}, # labelid -> (docname, eqno)
}
} # type: Dict[unicode, Dict[unicode, Tuple[unicode, int]]]
dangling_warnings = {
'eq': 'equation not found: %(target)s',
}
def clear_doc(self, docname):
# type: (unicode) -> None
for labelid, (doc, eqno) in list(self.data['objects'].items()):
if doc == docname:
del self.data['objects'][labelid]
def merge_domaindata(self, docnames, otherdata):
# type: (Iterable[unicode], Dict) -> None
for labelid, (doc, eqno) in otherdata['objects'].items():
if doc in docnames:
self.data['objects'][labelid] = doc
def resolve_xref(self, env, fromdocname, builder, typ, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, unicode, nodes.Node, nodes.Node) -> nodes.Node # NOQA
assert typ == 'eq'
docname, number = self.data['objects'].get(target, (None, None))
if docname:
@ -76,6 +87,7 @@ class MathDomain(Domain):
return None
def resolve_any_xref(self, env, fromdocname, builder, target, node, contnode):
# type: (BuildEnvironment, unicode, Builder, unicode, nodes.Node, nodes.Node) -> List[nodes.Node] # NOQA
refnode = self.resolve_xref(env, fromdocname, builder, 'eq', target, node, contnode)
if refnode is None:
return []
@ -83,9 +95,11 @@ class MathDomain(Domain):
return [refnode]
def get_objects(self):
# type: () -> List
return []
def add_equation(self, env, docname, labelid):
# type: (BuildEnvironment, unicode, unicode) -> int
equations = self.data['objects']
if labelid in equations:
path = env.doc2path(equations[labelid][0])
@ -97,12 +111,15 @@ class MathDomain(Domain):
return eqno
def get_next_equation_number(self, docname):
# type: (unicode) -> int
targets = [eq for eq in self.data['objects'].values() if eq[0] == docname]
return len(targets) + 1
def wrap_displaymath(math, label, numbering):
# type: (unicode, unicode, bool) -> unicode
def is_equation(part):
# type: (unicode) -> unicode
return part.strip()
if label is None:
@ -137,11 +154,13 @@ def wrap_displaymath(math, label, numbering):
def math_role(role, rawtext, text, lineno, inliner, options={}, content=[]):
# type: (unicode, unicode, unicode, int, Inliner, Dict, List[unicode]) -> Tuple[List[nodes.Node], List[nodes.Node]] # NOQA
latex = utils.unescape(text, restore_backslashes=True)
return [math(latex=latex)], []
def is_in_section_title(node):
# type: (nodes.Node) -> bool
"""Determine whether the node is in a section title"""
from sphinx.util.nodes import traverse_parent
@ -165,6 +184,7 @@ class MathDirective(Directive):
}
def run(self):
# type: () -> List[nodes.Node]
latex = '\n'.join(self.content)
if self.arguments and self.arguments[0]:
latex = self.arguments[0] + '\n\n' + latex
@ -186,6 +206,7 @@ class MathDirective(Directive):
return ret
def add_target(self, ret):
# type: (List[nodes.Node]) -> None
node = ret[0]
env = self.state.document.settings.env
@ -213,6 +234,7 @@ class MathDirective(Directive):
def latex_visit_math(self, node):
# type: (nodes.NodeVisitor, math) -> None
if is_in_section_title(node):
protect = r'\protect'
else:
@ -223,6 +245,7 @@ def latex_visit_math(self, node):
def latex_visit_displaymath(self, node):
# type: (nodes.NodeVisitor, displaymath) -> None
if not node['label']:
label = None
else:
@ -239,17 +262,20 @@ def latex_visit_displaymath(self, node):
def latex_visit_eqref(self, node):
# type: (nodes.NodeVisitor, eqref) -> None
label = "equation:%s:%s" % (node['docname'], node['target'])
self.body.append('\\eqref{%s}' % label)
raise nodes.SkipNode
def text_visit_math(self, node):
# type: (nodes.NodeVisitor, math) -> None
self.add_text(node['latex'])
raise nodes.SkipNode
def text_visit_displaymath(self, node):
# type: (nodes.NodeVisitor, displaymath) -> None
self.new_state()
self.add_text(node['latex'])
self.end_state()
@ -257,24 +283,29 @@ def text_visit_displaymath(self, node):
def man_visit_math(self, node):
# type: (nodes.NodeVisitor, math) -> None
self.body.append(node['latex'])
raise nodes.SkipNode
def man_visit_displaymath(self, node):
# type: (nodes.NodeVisitor, displaymath) -> None
self.visit_centered(node)
def man_depart_displaymath(self, node):
# type: (nodes.NodeVisitor, displaymath) -> None
self.depart_centered(node)
def texinfo_visit_math(self, node):
# type: (nodes.NodeVisitor, math) -> None
self.body.append('@math{' + self.escape_arg(node['latex']) + '}')
raise nodes.SkipNode
def texinfo_visit_displaymath(self, node):
# type: (nodes.NodeVisitor, displaymath) -> None
if node.get('label'):
self.add_anchor(node['label'], node)
self.body.append('\n\n@example\n%s\n@end example\n\n' %
@ -282,10 +313,12 @@ def texinfo_visit_displaymath(self, node):
def texinfo_depart_displaymath(self, node):
# type: (nodes.NodeVisitor, displaymath) -> None
pass
def setup_math(app, htmlinlinevisitors, htmldisplayvisitors):
# type: (Sphinx, Tuple[Callable, Any], Tuple[Callable, Any]) -> None
app.add_config_value('math_number_all', False, 'env')
app.add_domain(MathDomain)
app.add_node(math, override=True,

14
sphinx/ext/napoleon/__init__.py
View File

@ -14,8 +14,13 @@ import sys
from six import PY2, iteritems
import sphinx
from sphinx.application import Sphinx
from sphinx.ext.napoleon.docstring import GoogleDocstring, NumpyDocstring
if False:
# For type annotation
from typing import Any # NOQA
class Config(object):
"""Sphinx napoleon extension settings in `conf.py`.
@ -254,6 +259,7 @@ class Config(object):
}
def __init__(self, **settings):
# type: (Any) -> None
for name, (default, rebuild) in iteritems(self._config_values):
setattr(self, name, default)
for name, value in iteritems(settings):
@ -261,6 +267,7 @@ class Config(object):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
"""Sphinx extension setup function.
When the extension is loaded, Sphinx imports this module and executes
@ -282,7 +289,6 @@ def setup(app):
`The Extension API <http://sphinx-doc.org/extdev/appapi.html>`_
"""
from sphinx.application import Sphinx
if not isinstance(app, Sphinx):
return # probably called by tests
@ -297,6 +303,7 @@ def setup(app):
def _patch_python_domain():
# type: () -> None
try:
from sphinx.domains.python import PyTypedField
except ImportError:
@ -317,6 +324,7 @@ def _patch_python_domain():
def _process_docstring(app, what, name, obj, options, lines):
# type: (Sphinx, unicode, unicode, Any, Any, List[unicode]) -> None
"""Process the docstring for a given python object.
Called when autodoc has read and processed a docstring. `lines` is a list
@ -353,6 +361,7 @@ def _process_docstring(app, what, name, obj, options, lines):
"""
result_lines = lines
docstring = None # type: GoogleDocstring
if app.config.napoleon_numpy_docstring:
docstring = NumpyDocstring(result_lines, app.config, app, what, name,
obj, options)
@ -365,6 +374,7 @@ def _process_docstring(app, what, name, obj, options, lines):
def _skip_member(app, what, name, obj, skip, options):
# type: (Sphinx, unicode, unicode, Any, bool, Any) -> bool
"""Determine if private and special class members are included in docs.
The following settings in conf.py determine if private and special class
@ -453,4 +463,4 @@ def _skip_member(app, what, name, obj, skip, options):
(is_private and inc_private) or
(is_init and inc_init)):
return False
return skip
return None

163
sphinx/ext/napoleon/docstring.py
View File

@ -21,6 +21,12 @@ from six.moves import range
from sphinx.ext.napoleon.iterators import modify_iter
from sphinx.util.pycompat import UnicodeMixin
if False:
# For type annotation
from typing import Any, Callable, Tuple, Union # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.config import Config as SphinxConfig # NOQA
_directive_regex = re.compile(r'\.\. \S+::')
_google_section_regex = re.compile(r'^(\s|\w)+:\s*$')
@ -99,19 +105,20 @@ class GoogleDocstring(UnicodeMixin):
"""
def __init__(self, docstring, config=None, app=None, what='', name='',
obj=None, options=None):
# type: (Union[unicode, List[unicode]], SphinxConfig, Sphinx, unicode, unicode, Any, Any) -> None # NOQA
self._config = config
self._app = app
if not self._config:
from sphinx.ext.napoleon import Config
self._config = self._app and self._app.config or Config()
self._config = self._app and self._app.config or Config() # type: ignore
if not what:
if inspect.isclass(obj):
what = 'class'
elif inspect.ismodule(obj):
what = 'module'
elif isinstance(obj, collections.Callable):
elif isinstance(obj, collections.Callable): # type: ignore
what = 'function'
else:
what = 'object'
@ -124,11 +131,11 @@ class GoogleDocstring(UnicodeMixin):
docstring = docstring.splitlines()
self._lines = docstring
self._line_iter = modify_iter(docstring, modifier=lambda s: s.rstrip())
self._parsed_lines = []
self._parsed_lines = [] # type: List[unicode]
self._is_in_section = False
self._section_indent = 0
if not hasattr(self, '_directive_sections'):
self._directive_sections = []
self._directive_sections = [] # type: List[unicode]
if not hasattr(self, '_sections'):
self._sections = {
'args': self._parse_parameters_section,
@ -154,10 +161,11 @@ class GoogleDocstring(UnicodeMixin):
'warns': self._parse_warns_section,
'yield': self._parse_yields_section,
'yields': self._parse_yields_section,
}
} # type: Dict[unicode, Callable]
self._parse()
def __unicode__(self):
# type: () -> unicode
"""Return the parsed docstring in reStructuredText format.
Returns
@ -169,6 +177,7 @@ class GoogleDocstring(UnicodeMixin):
return u('\n').join(self.lines())
def lines(self):
# type: () -> List[unicode]
"""Return the parsed lines of the docstring in reStructuredText format.
Returns
@ -180,38 +189,42 @@ class GoogleDocstring(UnicodeMixin):
return self._parsed_lines
def _consume_indented_block(self, indent=1):
# type: (int) -> List[unicode]
lines = []
line = self._line_iter.peek()
while(not self._is_section_break() and
(not line or self._is_indented(line, indent))):
lines.append(next(self._line_iter))
lines.append(next(self._line_iter)) # type: ignore
line = self._line_iter.peek()
return lines
def _consume_contiguous(self):
# type: () -> List[unicode]
lines = []
while (self._line_iter.has_next() and
self._line_iter.peek() and
not self._is_section_header()):
lines.append(next(self._line_iter))
lines.append(next(self._line_iter)) # type: ignore
return lines
def _consume_empty(self):
# type: () -> List[unicode]
lines = []
line = self._line_iter.peek()
while self._line_iter.has_next() and not line:
lines.append(next(self._line_iter))
lines.append(next(self._line_iter)) # type: ignore
line = self._line_iter.peek()
return lines
def _consume_field(self, parse_type=True, prefer_type=False):
line = next(self._line_iter)
# type: (bool, bool) -> Tuple[unicode, unicode, List[unicode]]
line = next(self._line_iter) # type: ignore
before, colon, after = self._partition_field_on_colon(line)
_name, _type, _desc = before, '', after
_name, _type, _desc = before, '', after # type: unicode, unicode, unicode
if parse_type:
match = _google_typed_arg_regex.match(before)
match = _google_typed_arg_regex.match(before) # type: ignore
if match:
_name = match.group(1)
_type = match.group(2)
@ -221,11 +234,12 @@ class GoogleDocstring(UnicodeMixin):
if prefer_type and not _type:
_type, _name = _name, _type
indent = self._get_indent(line) + 1
_desc = [_desc] + self._dedent(self._consume_indented_block(indent))
_desc = [_desc] + self._dedent(self._consume_indented_block(indent)) # type: ignore
_desc = self.__class__(_desc, self._config).lines()
return _name, _type, _desc
return _name, _type, _desc # type: ignore
def _consume_fields(self, parse_type=True, prefer_type=False):
# type: (bool, bool) -> List[Tuple[unicode, unicode, List[unicode]]]
self._consume_empty()
fields = []
while not self._is_section_break():
@ -235,19 +249,21 @@ class GoogleDocstring(UnicodeMixin):
return fields
def _consume_inline_attribute(self):
line = next(self._line_iter)
# type: () -> Tuple[unicode, List[unicode]]
line = next(self._line_iter) # type: ignore
_type, colon, _desc = self._partition_field_on_colon(line)
if not colon:
_type, _desc = _desc, _type
_desc = [_desc] + self._dedent(self._consume_to_end())
_desc = [_desc] + self._dedent(self._consume_to_end()) # type: ignore
_desc = self.__class__(_desc, self._config).lines()
return _type, _desc
return _type, _desc # type: ignore
def _consume_returns_section(self):
# type: () -> List[Tuple[unicode, unicode, List[unicode]]]
lines = self._dedent(self._consume_to_next_section())
if lines:
before, colon, after = self._partition_field_on_colon(lines[0])
_name, _type, _desc = '', '', lines
_name, _type, _desc = '', '', lines # type: unicode, unicode, List[unicode]
if colon:
if after:
@ -263,30 +279,35 @@ class GoogleDocstring(UnicodeMixin):
return []
def _consume_usage_section(self):
# type: () -> List[unicode]
lines = self._dedent(self._consume_to_next_section())
return lines
def _consume_section_header(self):
section = next(self._line_iter)
# type: () -> unicode
section = next(self._line_iter) # type: ignore
stripped_section = section.strip(':')
if stripped_section.lower() in self._sections:
section = stripped_section
return section
def _consume_to_end(self):
# type: () -> List[unicode]
lines = []
while self._line_iter.has_next():
lines.append(next(self._line_iter))
lines.append(next(self._line_iter)) # type: ignore
return lines
def _consume_to_next_section(self):
# type: () -> List[unicode]
self._consume_empty()
lines = []
while not self._is_section_break():
lines.append(next(self._line_iter))
lines.append(next(self._line_iter)) # type: ignore
return lines + self._consume_empty()
def _dedent(self, lines, full=False):
# type: (List[unicode], bool) -> List[unicode]
if full:
return [line.lstrip() for line in lines]
else:
@ -294,6 +315,7 @@ class GoogleDocstring(UnicodeMixin):
return [line[min_indent:] for line in lines]
def _escape_args_and_kwargs(self, name):
# type: (unicode) -> unicode
if name[:2] == '**':
return r'\*\*' + name[2:]
elif name[:1] == '*':
@ -302,29 +324,32 @@ class GoogleDocstring(UnicodeMixin):
return name
def _fix_field_desc(self, desc):
# type: (List[unicode]) -> List[unicode]
if self._is_list(desc):
desc = [''] + desc
desc = [''] + desc # type: ignore
elif desc[0].endswith('::'):
desc_block = desc[1:]
indent = self._get_indent(desc[0])
block_indent = self._get_initial_indent(desc_block)
if block_indent > indent:
desc = [''] + desc
desc = [''] + desc # type: ignore
else:
desc = ['', desc[0]] + self._indent(desc_block, 4)
return desc
def _format_admonition(self, admonition, lines):
# type: (unicode, List[unicode]) -> List[unicode]
lines = self._strip_empty(lines)
if len(lines) == 1:
return ['.. %s:: %s' % (admonition, lines[0].strip()), '']
elif lines:
lines = self._indent(self._dedent(lines), 3)
return ['.. %s::' % admonition, ''] + lines + ['']
return ['.. %s::' % admonition, ''] + lines + [''] # type: ignore
else:
return ['.. %s::' % admonition, '']
def _format_block(self, prefix, lines, padding=None):
# type: (unicode, List[unicode], unicode) -> List[unicode]
if lines:
if padding is None:
padding = ' ' * len(prefix)
@ -342,6 +367,7 @@ class GoogleDocstring(UnicodeMixin):
def _format_docutils_params(self, fields, field_role='param',
type_role='type'):
# type: (List[Tuple[unicode, unicode, List[unicode]]], unicode, unicode) -> List[unicode] # NOQA
lines = []
for _name, _type, _desc in fields:
_desc = self._strip_empty(_desc)
@ -357,13 +383,14 @@ class GoogleDocstring(UnicodeMixin):
return lines + ['']
def _format_field(self, _name, _type, _desc):
# type: (unicode, unicode, List[unicode]) -> List[unicode]
_desc = self._strip_empty(_desc)
has_desc = any(_desc)
separator = has_desc and ' -- ' or ''
if _name:
if _type:
if '`' in _type:
field = '**%s** (%s)%s' % (_name, _type, separator)
field = '**%s** (%s)%s' % (_name, _type, separator) # type: unicode
else:
field = '**%s** (*%s*)%s' % (_name, _type, separator)
else:
@ -386,10 +413,11 @@ class GoogleDocstring(UnicodeMixin):
return [field]
def _format_fields(self, field_type, fields):
# type: (unicode, List[Tuple[unicode, unicode, List[unicode]]]) -> List[unicode]
field_type = ':%s:' % field_type.strip()
padding = ' ' * len(field_type)
multi = len(fields) > 1
lines = []
lines = [] # type: List[unicode]
for _name, _type, _desc in fields:
field = self._format_field(_name, _type, _desc)
if multi:
@ -404,6 +432,7 @@ class GoogleDocstring(UnicodeMixin):
return lines
def _get_current_indent(self, peek_ahead=0):
# type: (int) -> int
line = self._line_iter.peek(peek_ahead + 1)[peek_ahead]
while line != self._line_iter.sentinel:
if line:
@ -413,18 +442,21 @@ class GoogleDocstring(UnicodeMixin):
return 0
def _get_indent(self, line):
# type: (unicode) -> int
for i, s in enumerate(line):
if not s.isspace():
return i
return len(line)
def _get_initial_indent(self, lines):
# type: (List[unicode]) -> int
for line in lines:
if line:
return self._get_indent(line)
return 0
def _get_min_indent(self, lines):
# type: (List[unicode]) -> int
min_indent = None
for line in lines:
if line:
@ -436,9 +468,11 @@ class GoogleDocstring(UnicodeMixin):
return min_indent or 0
def _indent(self, lines, n=4):
# type: (List[unicode], int) -> List[unicode]
return [(' ' * n) + line for line in lines]
def _is_indented(self, line, indent=1):
# type: (unicode, int) -> bool
for i, s in enumerate(line):
if i >= indent:
return True
@ -447,11 +481,12 @@ class GoogleDocstring(UnicodeMixin):
return False
def _is_list(self, lines):
# type: (List[unicode]) -> bool
if not lines:
return False
if _bullet_list_regex.match(lines[0]):
if _bullet_list_regex.match(lines[0]): # type: ignore
return True
if _enumerated_list_regex.match(lines[0]):
if _enumerated_list_regex.match(lines[0]): # type: ignore
return True
if len(lines) < 2 or lines[0].endswith('::'):
return False
@ -464,6 +499,7 @@ class GoogleDocstring(UnicodeMixin):
return next_indent > indent
def _is_section_header(self):
# type: () -> bool
section = self._line_iter.peek().lower()
match = _google_section_regex.match(section)
if match and section.strip(':') in self._sections:
@ -478,6 +514,7 @@ class GoogleDocstring(UnicodeMixin):
return False
def _is_section_break(self):
# type: () -> bool
line = self._line_iter.peek()
return (not self._line_iter.has_next() or
self._is_section_header() or
@ -486,6 +523,7 @@ class GoogleDocstring(UnicodeMixin):
not self._is_indented(line, self._section_indent)))
def _parse(self):
# type: () -> None
self._parsed_lines = self._consume_empty()
if self._name and (self._what == 'attribute' or self._what == 'data'):
@ -498,7 +536,7 @@ class GoogleDocstring(UnicodeMixin):
section = self._consume_section_header()
self._is_in_section = True
self._section_indent = self._get_current_indent()
if _directive_regex.match(section):
if _directive_regex.match(section): # type: ignore
lines = [section] + self._consume_to_next_section()
else:
lines = self._sections[section.lower()](section)
@ -513,42 +551,47 @@ class GoogleDocstring(UnicodeMixin):
self._parsed_lines.extend(lines)
def _parse_attribute_docstring(self):
# type: () -> List[unicode]
_type, _desc = self._consume_inline_attribute()
return self._format_field('', _type, _desc)
def _parse_attributes_section(self, section):
# type: (unicode) -> List[unicode]
lines = []
for _name, _type, _desc in self._consume_fields():
if self._config.napoleon_use_ivar:
field = ':ivar %s: ' % _name
field = ':ivar %s: ' % _name # type: unicode
lines.extend(self._format_block(field, _desc))
if _type:
lines.append(':vartype %s: %s' % (_name, _type))
else:
lines.extend(['.. attribute:: ' + _name, ''])
field = self._format_field('', _type, _desc)
lines.extend(self._indent(field, 3))
field = self._format_field('', _type, _desc) # type: ignore
lines.extend(self._indent(field, 3)) # type: ignore
lines.append('')
if self._config.napoleon_use_ivar:
lines.append('')
return lines
def _parse_examples_section(self, section):
# type: (unicode) -> List[unicode]
use_admonition = self._config.napoleon_use_admonition_for_examples
return self._parse_generic_section(section, use_admonition)
def _parse_usage_section(self, section):
header = ['.. rubric:: Usage:', '']
block = ['.. code-block:: python', '']
# type: (unicode) -> List[unicode]
header = ['.. rubric:: Usage:', ''] # type: List[unicode]
block = ['.. code-block:: python', ''] # type: List[unicode]
lines = self._consume_usage_section()
lines = self._indent(lines, 3)
return header + block + lines + ['']
def _parse_generic_section(self, section, use_admonition):
# type: (unicode, bool) -> List[unicode]
lines = self._strip_empty(self._consume_to_next_section())
lines = self._dedent(lines)
if use_admonition:
header = '.. admonition:: %s' % section
header = '.. admonition:: %s' % section # type: unicode
lines = self._indent(lines, 3)
else:
header = '.. rubric:: %s' % section
@ -558,6 +601,7 @@ class GoogleDocstring(UnicodeMixin):
return [header, '']
def _parse_keyword_arguments_section(self, section):
# type: (unicode) -> List[unicode]
fields = self._consume_fields()
if self._config.napoleon_use_keyword:
return self._format_docutils_params(
@ -568,26 +612,31 @@ class GoogleDocstring(UnicodeMixin):
return self._format_fields('Keyword Arguments', fields)
def _parse_methods_section(self, section):
lines = []
# type: (unicode) -> List[unicode]
lines = [] # type: List[unicode]
for _name, _, _desc in self._consume_fields(parse_type=False):
lines.append('.. method:: %s' % _name)
if _desc:
lines.extend([''] + self._indent(_desc, 3))
lines.extend([''] + self._indent(_desc, 3)) # type: ignore
lines.append('')
return lines
def _parse_note_section(self, section):
# type: (unicode) -> List[unicode]
lines = self._consume_to_next_section()
return self._format_admonition('note', lines)
def _parse_notes_section(self, section):
# type: (unicode) -> List[unicode]
use_admonition = self._config.napoleon_use_admonition_for_notes
return self._parse_generic_section('Notes', use_admonition)
def _parse_other_parameters_section(self, section):
# type: (unicode) -> List[unicode]
return self._format_fields('Other Parameters', self._consume_fields())
def _parse_parameters_section(self, section):
# type: (unicode) -> List[unicode]
fields = self._consume_fields()
if self._config.napoleon_use_param:
return self._format_docutils_params(fields)
@ -595,11 +644,12 @@ class GoogleDocstring(UnicodeMixin):
return self._format_fields('Parameters', fields)
def _parse_raises_section(self, section):
# type: (unicode) -> List[unicode]
fields = self._consume_fields(parse_type=False, prefer_type=True)
field_type = ':raises:'
padding = ' ' * len(field_type)
multi = len(fields) > 1
lines = []
lines = [] # type: List[unicode]
for _, _type, _desc in fields:
_desc = self._strip_empty(_desc)
has_desc = any(_desc)
@ -633,10 +683,12 @@ class GoogleDocstring(UnicodeMixin):
return lines
def _parse_references_section(self, section):
# type: (unicode) -> List[unicode]
use_admonition = self._config.napoleon_use_admonition_for_references
return self._parse_generic_section('References', use_admonition)
def _parse_returns_section(self, section):
# type: (unicode) -> List[unicode]
fields = self._consume_returns_section()
multi = len(fields) > 1
if multi:
@ -644,7 +696,7 @@ class GoogleDocstring(UnicodeMixin):
else:
use_rtype = self._config.napoleon_use_rtype
lines = []
lines = [] # type: List[unicode]
for _name, _type, _desc in fields:
if use_rtype:
field = self._format_field(_name, '', _desc)
@ -665,30 +717,36 @@ class GoogleDocstring(UnicodeMixin):
return lines
def _parse_see_also_section(self, section):
# type: (unicode) -> List[unicode]
lines = self._consume_to_next_section()
return self._format_admonition('seealso', lines)
def _parse_todo_section(self, section):
# type: (unicode) -> List[unicode]
lines = self._consume_to_next_section()
return self._format_admonition('todo', lines)
def _parse_warning_section(self, section):
# type: (unicode) -> List[unicode]
lines = self._consume_to_next_section()
return self._format_admonition('warning', lines)
def _parse_warns_section(self, section):
# type: (unicode) -> List[unicode]
return self._format_fields('Warns', self._consume_fields())
def _parse_yields_section(self, section):
# type: (unicode) -> List[unicode]
fields = self._consume_returns_section()
return self._format_fields('Yields', fields)
def _partition_field_on_colon(self, line):
# type: (unicode) -> Tuple[unicode, unicode, unicode]
before_colon = []
after_colon = []
colon = ''
found_colon = False
for i, source in enumerate(_xref_regex.split(line)):
for i, source in enumerate(_xref_regex.split(line)): # type: ignore
if found_colon:
after_colon.append(source)
else:
@ -706,6 +764,7 @@ class GoogleDocstring(UnicodeMixin):
"".join(after_colon).strip())
def _strip_empty(self, lines):
# type: (List[unicode]) -> List[unicode]
if lines:
start = -1
for i, line in enumerate(lines):
@ -820,12 +879,14 @@ class NumpyDocstring(GoogleDocstring):
"""
def __init__(self, docstring, config=None, app=None, what='', name='',
obj=None, options=None):
# type: (Union[unicode, List[unicode]], SphinxConfig, Sphinx, unicode, unicode, Any, Any) -> None # NOQA
self._directive_sections = ['.. index::']
super(NumpyDocstring, self).__init__(docstring, config, app, what,
name, obj, options)
def _consume_field(self, parse_type=True, prefer_type=False):
line = next(self._line_iter)
# type: (bool, bool) -> Tuple[unicode, unicode, List[unicode]]
line = next(self._line_iter) # type: ignore
if parse_type:
_name, _, _type = self._partition_field_on_colon(line)
else:
@ -841,16 +902,19 @@ class NumpyDocstring(GoogleDocstring):
return _name, _type, _desc
def _consume_returns_section(self):
# type: () -> List[Tuple[unicode, unicode, List[unicode]]]
return self._consume_fields(prefer_type=True)
def _consume_section_header(self):
section = next(self._line_iter)
# type: () -> unicode
section = next(self._line_iter) # type: ignore
if not _directive_regex.match(section):
# Consume the header underline
next(self._line_iter)
next(self._line_iter) # type: ignore
return section
def _is_section_break(self):
# type: () -> bool
line1, line2 = self._line_iter.peek(2)
return (not self._line_iter.has_next() or
self._is_section_header() or
@ -860,10 +924,11 @@ class NumpyDocstring(GoogleDocstring):
not self._is_indented(line1, self._section_indent)))
def _is_section_header(self):
# type: () -> bool
section, underline = self._line_iter.peek(2)
section = section.lower()
if section in self._sections and isinstance(underline, string_types):
return bool(_numpy_section_regex.match(underline))
return bool(_numpy_section_regex.match(underline)) # type: ignore
elif self._directive_sections:
if _directive_regex.match(section):
for directive_section in self._directive_sections:
@ -875,6 +940,7 @@ class NumpyDocstring(GoogleDocstring):
r" (?P<name2>[a-zA-Z0-9_.-]+))\s*", re.X)
def _parse_see_also_section(self, section):
# type: (unicode) -> List[unicode]
lines = self._consume_to_next_section()
try:
return self._parse_numpydoc_see_also_section(lines)
@ -882,6 +948,7 @@ class NumpyDocstring(GoogleDocstring):
return self._format_admonition('seealso', lines)
def _parse_numpydoc_see_also_section(self, content):
# type: (List[unicode]) -> List[unicode]
"""
Derived from the NumpyDoc implementation of _parse_see_also.
@ -914,13 +981,13 @@ class NumpyDocstring(GoogleDocstring):
del rest[:]
current_func = None
rest = []
rest = [] # type: List[unicode]
for line in content:
if not line.strip():
continue
m = self._name_rgx.match(line)
m = self._name_rgx.match(line) # type: ignore
if m and line[m.end():].strip().startswith(':'):
push_item(current_func, rest)
current_func, line = line[:m.end()], line[m.end():]
@ -960,12 +1027,12 @@ class NumpyDocstring(GoogleDocstring):
'const': 'const',
'attribute': 'attr',
'attr': 'attr'
}
} # type: Dict[unicode, unicode]
if self._what is None:
func_role = 'obj'
func_role = 'obj' # type: unicode
else:
func_role = roles.get(self._what, '')
lines = []
lines = [] # type: List[unicode]
last_had_desc = True
for func, desc, role in items:
if role:

21
sphinx/ext/napoleon/iterators.py
View File

@ -13,6 +13,10 @@
import collections
if False:
# For type annotation
from typing import Any, Iterable # NOQA
class peek_iter(object):
"""An iterator object that supports peeking ahead.
@ -48,34 +52,39 @@ class peek_iter(object):
"""
def __init__(self, *args):
# type: (Any) -> None
"""__init__(o, sentinel=None)"""
self._iterable = iter(*args)
self._cache = collections.deque()
self._iterable = iter(*args) # type: Iterable
self._cache = collections.deque() # type: collections.deque
if len(args) == 2:
self.sentinel = args[1]
else:
self.sentinel = object()
def __iter__(self):
# type: () -> peek_iter
return self
def __next__(self, n=None):
# type: (int) -> Any
# note: prevent 2to3 to transform self.next() in next(self) which
# causes an infinite loop !
return getattr(self, 'next')(n)
def _fillcache(self, n):
# type: (int) -> None
"""Cache `n` items. If `n` is 0 or None, then 1 item is cached."""
if not n:
n = 1
try:
while len(self._cache) < n:
self._cache.append(next(self._iterable))
self._cache.append(next(self._iterable)) # type: ignore
except StopIteration:
while len(self._cache) < n:
self._cache.append(self.sentinel)
def has_next(self):
# type: () -> bool
"""Determine if iterator is exhausted.
Returns
@ -91,6 +100,7 @@ class peek_iter(object):
return self.peek() != self.sentinel
def next(self, n=None):
# type: (int) -> Any
"""Get the next item or `n` items of the iterator.
Parameters
@ -126,6 +136,7 @@ class peek_iter(object):
return result
def peek(self, n=None):
# type: (int) -> Any
"""Preview the next item or `n` items of the iterator.
The iterator is not advanced when peek is called.
@ -209,6 +220,7 @@ class modify_iter(peek_iter):
"""
def __init__(self, *args, **kwargs):
# type: (Any, Any) -> None
"""__init__(o, sentinel=None, modifier=lambda x: x)"""
if 'modifier' in kwargs:
self.modifier = kwargs['modifier']
@ -223,6 +235,7 @@ class modify_iter(peek_iter):
super(modify_iter, self).__init__(*args)
def _fillcache(self, n):
# type: (int) -> None
"""Cache `n` modified items. If `n` is 0 or None, 1 item is cached.
Each item returned by the iterator is passed through the
@ -233,7 +246,7 @@ class modify_iter(peek_iter):
n = 1
try:
while len(self._cache) < n:
self._cache.append(self.modifier(next(self._iterable)))
self._cache.append(self.modifier(next(self._iterable))) # type: ignore
except StopIteration:
while len(self._cache) < n:
self._cache.append(self.sentinel)

43
sphinx/ext/pngmath.py
View File

@ -20,20 +20,31 @@ from subprocess import Popen, PIPE
from hashlib import sha1
from six import text_type
from docutils import nodes
import sphinx
from sphinx.errors import SphinxError, ExtensionError
from sphinx.util import logging
from sphinx.util.png import read_png_depth, write_png_depth
from sphinx.util.osutil import ensuredir, ENOENT, cd
from sphinx.util.pycompat import sys_encoding
from sphinx.ext.mathbase import setup_math as mathbase_setup, wrap_displaymath
if False:
# For type annotation
from typing import Any, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.ext.mathbase import math as math_node, displaymath # NOQA
logger = logging.getLogger(__name__)
class MathExtError(SphinxError):
category = 'Math extension error'
def __init__(self, msg, stderr=None, stdout=None):
# type: (unicode, unicode, unicode) -> None
if stderr:
msg += '\n[stderr]\n' + stderr.decode(sys_encoding, 'replace')
if stdout:
@ -71,6 +82,7 @@ depth_re = re.compile(br'\[\d+ depth=(-?\d+)\]')
def render_math(self, math):
# type: (nodes.NodeVisitor, unicode) -> Tuple[unicode, int]
"""Render the LaTeX math expression *math* using latex and dvipng.
Return the filename relative to the built document and the "depth",
@ -107,9 +119,8 @@ def render_math(self, math):
else:
tempdir = self.builder._mathpng_tempdir
tf = codecs.open(path.join(tempdir, 'math.tex'), 'w', 'utf-8')
tf.write(latex)
tf.close()
with codecs.open(path.join(tempdir, 'math.tex'), 'w', 'utf-8') as tf:
tf.write(latex)
# build latex command; old versions of latex don't have the
# --output-directory option, so we have to manually chdir to the
@ -125,9 +136,9 @@ def render_math(self, math):
except OSError as err:
if err.errno != ENOENT: # No such file or directory
raise
self.builder.warn('LaTeX command %r cannot be run (needed for math '
'display), check the pngmath_latex setting' %
self.builder.config.pngmath_latex)
logger.warning('LaTeX command %r cannot be run (needed for math '
'display), check the pngmath_latex setting',
self.builder.config.pngmath_latex)
self.builder._mathpng_warned_latex = True
return None, None
@ -150,9 +161,9 @@ def render_math(self, math):
except OSError as err:
if err.errno != ENOENT: # No such file or directory
raise
self.builder.warn('dvipng command %r cannot be run (needed for math '
'display), check the pngmath_dvipng setting' %
self.builder.config.pngmath_dvipng)
logger.warning('dvipng command %r cannot be run (needed for math '
'display), check the pngmath_dvipng setting',
self.builder.config.pngmath_dvipng)
self.builder._mathpng_warned_dvipng = True
return None, None
stdout, stderr = p.communicate()
@ -171,23 +182,26 @@ def render_math(self, math):
def cleanup_tempdir(app, exc):
# type: (Sphinx, Exception) -> None
if exc:
return
if not hasattr(app.builder, '_mathpng_tempdir'):
return
try:
shutil.rmtree(app.builder._mathpng_tempdir)
shutil.rmtree(app.builder._mathpng_tempdir) # type: ignore
except Exception:
pass
def get_tooltip(self, node):
# type: (nodes.NodeVisitor, math_node) -> unicode
if self.builder.config.pngmath_add_tooltips:
return ' alt="%s"' % self.encode(node['latex']).strip()
return ''
def html_visit_math(self, node):
# type: (nodes.NodeVisitor, math_node) -> None
try:
fname, depth = render_math(self, '$' + node['latex'] + '$')
except MathExtError as exc:
@ -195,7 +209,7 @@ def html_visit_math(self, node):
sm = nodes.system_message(msg, type='WARNING', level=2,
backrefs=[], source=node['latex'])
sm.walkabout(self)
self.builder.warn('display latex %r: ' % node['latex'] + msg)
logger.warning('display latex %r: %s', node['latex'], msg)
raise nodes.SkipNode
if fname is None:
# something failed -- use text-only as a bad substitute
@ -210,6 +224,7 @@ def html_visit_math(self, node):
def html_visit_displaymath(self, node):
# type: (nodes.NodeVisitor, displaymath) -> None
if node['nowrap']:
latex = node['latex']
else:
@ -222,7 +237,7 @@ def html_visit_displaymath(self, node):
sm = nodes.system_message(msg, type='WARNING', level=2,
backrefs=[], source=node['latex'])
sm.walkabout(self)
self.builder.warn('inline latex %r: ' % node['latex'] + msg)
logger.warning('inline latex %r: %s', node['latex'], msg)
raise nodes.SkipNode
self.body.append(self.starttag(node, 'div', CLASS='math'))
self.body.append('<p>')
@ -239,7 +254,9 @@ def html_visit_displaymath(self, node):
def setup(app):
app.warn('sphinx.ext.pngmath has been deprecated. Please use sphinx.ext.imgmath instead.')
# type: (Sphinx) -> Dict[unicode, Any]
logger.warning('sphinx.ext.pngmath has been deprecated. '
'Please use sphinx.ext.imgmath instead.')
try:
mathbase_setup(app, (html_visit_math, None), (html_visit_displaymath, None))
except ExtensionError:

37
sphinx/ext/todo.py
View File

@ -18,10 +18,19 @@ from docutils.parsers.rst import directives
import sphinx
from sphinx.locale import _
from sphinx.environment import NoUri
from sphinx.util import logging
from sphinx.util.nodes import set_source_info
from docutils.parsers.rst import Directive
from docutils.parsers.rst.directives.admonitions import BaseAdmonition
if False:
# For type annotation
from typing import Any, Iterable # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
logger = logging.getLogger(__name__)
class todo_node(nodes.Admonition, nodes.Element):
pass
@ -46,6 +55,7 @@ class Todo(BaseAdmonition):
}
def run(self):
# type: () -> List[nodes.Node]
if not self.options.get('class'):
self.options['class'] = ['admonition-todo']
@ -63,12 +73,13 @@ class Todo(BaseAdmonition):
def process_todos(app, doctree):
# type: (Sphinx, nodes.Node) -> None
# collect all todos in the environment
# this is not done in the directive itself because it some transformations
# must have already been run, e.g. substitutions
env = app.builder.env
if not hasattr(env, 'todo_all_todos'):
env.todo_all_todos = []
env.todo_all_todos = [] # type: ignore
for node in doctree.traverse(todo_node):
app.emit('todo-defined', node)
@ -80,7 +91,7 @@ def process_todos(app, doctree):
targetnode = None
newnode = node.deepcopy()
del newnode['ids']
env.todo_all_todos.append({
env.todo_all_todos.append({ # type: ignore
'docname': env.docname,
'source': node.source or env.doc2path(env.docname),
'lineno': node.line,
@ -89,7 +100,8 @@ def process_todos(app, doctree):
})
if env.config.todo_emit_warnings:
env.warn_node("TODO entry found: %s" % node[1].astext(), node)
logger.warning("TODO entry found: %s", node[1].astext(),
location=node)
class TodoList(Directive):
@ -101,15 +113,17 @@ class TodoList(Directive):
required_arguments = 0
optional_arguments = 0
final_argument_whitespace = False
option_spec = {}
option_spec = {} # type: Dict
def run(self):
# type: () -> List[todolist]
# Simply insert an empty todolist node which will be replaced later
# when process_todo_nodes is called
return [todolist('')]
def process_todo_nodes(app, doctree, fromdocname):
# type: (Sphinx, nodes.Node, unicode) -> None
if not app.config['todo_include_todos']:
for node in doctree.traverse(todo_node):
node.parent.remove(node)
@ -119,7 +133,7 @@ def process_todo_nodes(app, doctree, fromdocname):
env = app.builder.env
if not hasattr(env, 'todo_all_todos'):
env.todo_all_todos = []
env.todo_all_todos = [] # type: ignore
for node in doctree.traverse(todolist):
if not app.config['todo_include_todos']:
@ -128,7 +142,7 @@ def process_todo_nodes(app, doctree, fromdocname):
content = []
for todo_info in env.todo_all_todos:
for todo_info in env.todo_all_todos: # type: ignore
para = nodes.paragraph(classes=['todo-source'])
if app.config['todo_link_only']:
description = _('<<original entry>>')
@ -168,30 +182,35 @@ def process_todo_nodes(app, doctree, fromdocname):
def purge_todos(app, env, docname):
# type: (Sphinx, BuildEnvironment, unicode) -> None
if not hasattr(env, 'todo_all_todos'):
return
env.todo_all_todos = [todo for todo in env.todo_all_todos
env.todo_all_todos = [todo for todo in env.todo_all_todos # type: ignore
if todo['docname'] != docname]
def merge_info(app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Iterable[unicode], BuildEnvironment) -> None
if not hasattr(other, 'todo_all_todos'):
return
if not hasattr(env, 'todo_all_todos'):
env.todo_all_todos = []
env.todo_all_todos.extend(other.todo_all_todos)
env.todo_all_todos = [] # type: ignore
env.todo_all_todos.extend(other.todo_all_todos) # type: ignore
def visit_todo_node(self, node):
# type: (nodes.NodeVisitor, todo_node) -> None
self.visit_admonition(node)
# self.visit_admonition(node, 'todo')
def depart_todo_node(self, node):
# type: (nodes.NodeVisitor, todo_node) -> None
self.depart_admonition(node)
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_event('todo-defined')
app.add_config_value('todo_include_todos', False, 'html')
app.add_config_value('todo_link_only', False, 'html')

52
sphinx/ext/viewcode.py
View File

@ -12,51 +12,60 @@
import traceback
from six import iteritems, text_type
from docutils import nodes
import sphinx
from sphinx import addnodes
from sphinx.locale import _
from sphinx.pycode import ModuleAnalyzer
from sphinx.util import get_full_modname
from sphinx.util import get_full_modname, logging, status_iterator
from sphinx.util.nodes import make_refnode
from sphinx.util.console import blue
if False:
# For type annotation
from typing import Any, Iterable, Iterator, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.environment import BuildEnvironment # NOQA
logger = logging.getLogger(__name__)
def _get_full_modname(app, modname, attribute):
# type: (Sphinx, str, unicode) -> unicode
try:
return get_full_modname(modname, attribute)
except AttributeError:
# sphinx.ext.viewcode can't follow class instance attribute
# then AttributeError logging output only verbose mode.
app.verbose('Didn\'t find %s in %s' % (attribute, modname))
logger.verbose('Didn\'t find %s in %s', attribute, modname)
return None
except Exception as e:
# sphinx.ext.viewcode follow python domain directives.
# because of that, if there are no real modules exists that specified
# by py:function or other directives, viewcode emits a lot of warnings.
# It should be displayed only verbose mode.
app.verbose(traceback.format_exc().rstrip())
app.verbose('viewcode can\'t import %s, failed with error "%s"' %
(modname, e))
logger.verbose(traceback.format_exc().rstrip())
logger.verbose('viewcode can\'t import %s, failed with error "%s"', modname, e)
return None
def doctree_read(app, doctree):
# type: (Sphinx, nodes.Node) -> None
env = app.builder.env
if not hasattr(env, '_viewcode_modules'):
env._viewcode_modules = {}
env._viewcode_modules = {} # type: ignore
if app.builder.name == "singlehtml":
return
if app.builder.name.startswith("epub") and not env.config.viewcode_enable_epub:
return
def has_tag(modname, fullname, docname, refname):
entry = env._viewcode_modules.get(modname, None)
entry = env._viewcode_modules.get(modname, None) # type: ignore
try:
analyzer = ModuleAnalyzer.for_module(modname)
except Exception:
env._viewcode_modules[modname] = False
env._viewcode_modules[modname] = False # type: ignore
return
if not isinstance(analyzer.code, text_type):
code = analyzer.code.decode(analyzer.encoding)
@ -65,7 +74,7 @@ def doctree_read(app, doctree):
if entry is None or entry[0] != code:
analyzer.find_tags()
entry = code, analyzer.tags, {}, refname
env._viewcode_modules[modname] = entry
env._viewcode_modules[modname] = entry # type: ignore
elif entry is False:
return
_, tags, used, _ = entry
@ -76,7 +85,7 @@ def doctree_read(app, doctree):
for objnode in doctree.traverse(addnodes.desc):
if objnode.get('domain') != 'py':
continue
names = set()
names = set() # type: Set[unicode]
for signode in objnode:
if not isinstance(signode, addnodes.desc_signature):
continue
@ -106,16 +115,18 @@ def doctree_read(app, doctree):
def env_merge_info(app, env, docnames, other):
# type: (Sphinx, BuildEnvironment, Iterable[unicode], BuildEnvironment) -> None
if not hasattr(other, '_viewcode_modules'):
return
# create a _viewcode_modules dict on the main environment
if not hasattr(env, '_viewcode_modules'):
env._viewcode_modules = {}
env._viewcode_modules = {} # type: ignore
# now merge in the information from the subprocess
env._viewcode_modules.update(other._viewcode_modules)
env._viewcode_modules.update(other._viewcode_modules) # type: ignore
def missing_reference(app, env, node, contnode):
# type: (Sphinx, BuildEnvironment, nodes.Node, nodes.Node) -> nodes.Node
# resolve our "viewcode" reference nodes -- they need special treatment
if node['reftype'] == 'viewcode':
return make_refnode(app.builder, node['refdoc'], node['reftarget'],
@ -123,20 +134,22 @@ def missing_reference(app, env, node, contnode):
def collect_pages(app):
# type: (Sphinx) -> Iterator[Tuple[unicode, Dict[unicode, Any], unicode]]
env = app.builder.env
if not hasattr(env, '_viewcode_modules'):
return
highlighter = app.builder.highlighter
highlighter = app.builder.highlighter # type: ignore
urito = app.builder.get_relative_uri
modnames = set(env._viewcode_modules)
modnames = set(env._viewcode_modules) # type: ignore
# app.builder.info(' (%d module code pages)' %
# len(env._viewcode_modules), nonl=1)
for modname, entry in app.status_iterator(
iteritems(env._viewcode_modules), 'highlighting module code... ',
blue, len(env._viewcode_modules), lambda x: x[0]):
for modname, entry in status_iterator(iteritems(env._viewcode_modules), # type: ignore
'highlighting module code... ', "blue",
len(env._viewcode_modules), # type: ignore
app.verbosity, lambda x: x[0]):
if not entry:
continue
code, tags, used, refname = entry
@ -185,7 +198,7 @@ def collect_pages(app):
'title': modname,
'body': (_('<h1>Source code for %s</h1>') % modname +
'\n'.join(lines)),
}
} # type: Dict[unicode, Any]
yield (pagename, context, 'page.html')
if not modnames:
@ -218,6 +231,7 @@ def collect_pages(app):
def setup(app):
# type: (Sphinx) -> Dict[unicode, Any]
app.add_config_value('viewcode_import', True, False)
app.add_config_value('viewcode_enable_epub', False, False)
app.connect('doctree-read', doctree_read)

49
sphinx/highlighting.py
View File

@ -11,11 +11,13 @@
from six import text_type
from sphinx.util import logging
from sphinx.util.pycompat import htmlescape
from sphinx.util.texescape import tex_hl_escape_map_new
from sphinx.ext import doctest
from pygments import highlight
from pygments.lexer import Lexer # NOQA
from pygments.lexers import PythonLexer, Python3Lexer, PythonConsoleLexer, \
CLexer, TextLexer, RstLexer
from pygments.lexers import get_lexer_by_name, guess_lexer
@ -25,6 +27,14 @@ from pygments.styles import get_style_by_name
from pygments.util import ClassNotFound
from sphinx.pygments_styles import SphinxStyle, NoneStyle
if False:
# For type annotation
from typing import Any # NOQA
from pygments.formatter import Formatter # NOQA
logger = logging.getLogger(__name__)
lexers = dict(
none = TextLexer(stripnl=False),
python = PythonLexer(stripnl=False),
@ -33,7 +43,7 @@ lexers = dict(
pycon3 = PythonConsoleLexer(python3=True, stripnl=False),
rest = RstLexer(stripnl=False),
c = CLexer(stripnl=False),
)
) # type: Dict[unicode, Lexer]
for _lexer in lexers.values():
_lexer.add_filter('raiseonerror')
@ -55,8 +65,8 @@ class PygmentsBridge(object):
html_formatter = HtmlFormatter
latex_formatter = LatexFormatter
def __init__(self, dest='html', stylename='sphinx',
trim_doctest_flags=False):
def __init__(self, dest='html', stylename='sphinx', trim_doctest_flags=False):
# type: (unicode, unicode, bool) -> None
self.dest = dest
if stylename is None or stylename == 'sphinx':
style = SphinxStyle
@ -69,7 +79,7 @@ class PygmentsBridge(object):
else:
style = get_style_by_name(stylename)
self.trim_doctest_flags = trim_doctest_flags
self.formatter_args = {'style': style}
self.formatter_args = {'style': style} # type: Dict[unicode, Any]
if dest == 'html':
self.formatter = self.html_formatter
else:
@ -77,10 +87,12 @@ class PygmentsBridge(object):
self.formatter_args['commandprefix'] = 'PYG'
def get_formatter(self, **kwargs):
kwargs.update(self.formatter_args)
# type: (Any) -> Formatter
kwargs.update(self.formatter_args) # type: ignore
return self.formatter(**kwargs)
def unhighlighted(self, source):
# type: (unicode) -> unicode
if self.dest == 'html':
return '<pre>' + htmlescape(source) + '</pre>\n'
else:
@ -91,7 +103,8 @@ class PygmentsBridge(object):
return '\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n' + \
source + '\\end{Verbatim}\n'
def highlight_block(self, source, lang, opts=None, warn=None, force=False, **kwargs):
def highlight_block(self, source, lang, opts=None, location=None, force=False, **kwargs):
# type: (unicode, unicode, Any, Any, bool, Any) -> unicode
if not isinstance(source, text_type):
source = source.decode()
@ -119,34 +132,31 @@ class PygmentsBridge(object):
try:
lexer = lexers[lang] = get_lexer_by_name(lang, **(opts or {}))
except ClassNotFound:
if warn:
warn('Pygments lexer name %r is not known' % lang)
lexer = lexers['none']
else:
raise
logger.warning('Pygments lexer name %r is not known', lang,
location=location)
lexer = lexers['none']
else:
lexer.add_filter('raiseonerror')
# trim doctest options if wanted
if isinstance(lexer, PythonConsoleLexer) and self.trim_doctest_flags:
source = doctest.blankline_re.sub('', source)
source = doctest.doctestopt_re.sub('', source)
source = doctest.blankline_re.sub('', source) # type: ignore
source = doctest.doctestopt_re.sub('', source) # type: ignore
# highlight via Pygments
formatter = self.get_formatter(**kwargs)
try:
hlsource = highlight(source, lexer, formatter)
except ErrorToken as exc:
except ErrorToken:
# this is most probably not the selected language,
# so let it pass unhighlighted
if lang == 'default':
pass # automatic highlighting failed.
elif warn:
warn('Could not lex literal_block as "%s". '
'Highlighting skipped.' % lang,
type='misc', subtype='highlighting_failure')
else:
raise exc
logger.warning('Could not lex literal_block as "%s". '
'Highlighting skipped.', lang,
type='misc', subtype='highlighting_failure',
location=location)
hlsource = highlight(source, lexers['none'], formatter)
if self.dest == 'html':
return hlsource
@ -156,6 +166,7 @@ class PygmentsBridge(object):
return hlsource.translate(tex_hl_escape_map_new)
def get_stylesheet(self):
# type: () -> unicode
formatter = self.get_formatter()
if self.dest == 'html':
return formatter.get_style_defs('.highlight')

42
sphinx/io.py
View File

@ -12,6 +12,7 @@ from docutils.io import FileInput
from docutils.readers import standalone
from docutils.writers import UnfilteredWriter
from six import string_types, text_type
from typing import Any, Union # NOQA
from sphinx.transforms import (
ApplySourceWorkaround, ExtraTranslatableNodes, CitationReferences,
@ -23,6 +24,18 @@ from sphinx.transforms.i18n import (
PreserveTranslatableMessages, Locale, RemoveTranslatableInline,
)
from sphinx.util import import_object, split_docinfo
from sphinx.util.docutils import LoggingReporter
if False:
# For type annotation
from typing import Any, Tuple, Union # NOQA
from docutils import nodes # NOQA
from docutils.io import Input # NOQA
from docutils.parsers import Parser # NOQA
from docutils.transforms import Transform # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
class SphinxBaseReader(standalone.Reader):
@ -30,17 +43,19 @@ class SphinxBaseReader(standalone.Reader):
Add our source parsers
"""
def __init__(self, app, parsers={}, *args, **kwargs):
# type: (Sphinx, Dict[unicode, Parser], Any, Any) -> None
standalone.Reader.__init__(self, *args, **kwargs)
self.parser_map = {}
self.parser_map = {} # type: Dict[unicode, Parser]
for suffix, parser_class in parsers.items():
if isinstance(parser_class, string_types):
parser_class = import_object(parser_class, 'source parser')
parser_class = import_object(parser_class, 'source parser') # type: ignore
parser = parser_class()
if hasattr(parser, 'set_application'):
parser.set_application(app)
self.parser_map[suffix] = parser
def read(self, source, parser, settings):
# type: (Input, Parser, Dict) -> nodes.document
self.source = source
for suffix in self.parser_map:
@ -56,8 +71,18 @@ class SphinxBaseReader(standalone.Reader):
return self.document
def get_transforms(self):
# type: () -> List[Transform]
return standalone.Reader.get_transforms(self) + self.transforms
def new_document(self):
# type: () -> nodes.document
document = standalone.Reader.new_document(self)
reporter = document.reporter
document.reporter = LoggingReporter(reporter.source, reporter.report_level,
reporter.halt_level, reporter.debug_flag,
reporter.error_handler)
return document
class SphinxStandaloneReader(SphinxBaseReader):
"""
@ -84,17 +109,21 @@ class SphinxI18nReader(SphinxBaseReader):
FilterSystemMessages, RefOnlyBulletListTransform]
def __init__(self, *args, **kwargs):
# type: (Any, Any) -> None
SphinxBaseReader.__init__(self, *args, **kwargs)
self.lineno = None
self.lineno = None # type: int
def set_lineno_for_reporter(self, lineno):
# type: (int) -> None
self.lineno = lineno
def new_document(self):
# type: () -> nodes.document
document = SphinxBaseReader.new_document(self)
reporter = document.reporter
def get_source_and_line(lineno=None):
# type: (int) -> Tuple[unicode, int]
return reporter.source, self.lineno
reporter.get_source_and_line = get_source_and_line
@ -105,28 +134,33 @@ class SphinxDummyWriter(UnfilteredWriter):
supported = ('html',) # needed to keep "meta" nodes
def translate(self):
# type: () -> None
pass
class SphinxFileInput(FileInput):
def __init__(self, app, env, *args, **kwds):
# type: (Sphinx, BuildEnvironment, Any, Any) -> None
self.app = app
self.env = env
kwds['error_handler'] = 'sphinx' # py3: handle error on open.
FileInput.__init__(self, *args, **kwds)
def decode(self, data):
# type: (Union[unicode, bytes]) -> unicode
if isinstance(data, text_type): # py3: `data` already decoded.
return data
return data.decode(self.encoding, 'sphinx') # py2: decoding
def read(self):
# type: () -> unicode
def get_parser_type(source_path):
# type: (unicode) -> Tuple[unicode]
for suffix in self.env.config.source_parsers:
if source_path.endswith(suffix):
parser_class = self.env.config.source_parsers[suffix]
if isinstance(parser_class, string_types):
parser_class = import_object(parser_class, 'source parser')
parser_class = import_object(parser_class, 'source parser') # type: ignore # NOQA
return parser_class.supported
else:
return ('restructuredtext',)

22
sphinx/jinja2glue.py
View File

@ -17,18 +17,28 @@ from jinja2 import FileSystemLoader, BaseLoader, TemplateNotFound, \
contextfunction
from jinja2.utils import open_if_exists
from jinja2.sandbox import SandboxedEnvironment
from typing import Any, Callable, Iterator, Tuple # NOQA
from sphinx.application import TemplateBridge
from sphinx.util.osutil import mtimes_of_files
if False:
# For type annotation
from typing import Any, Callable, Iterator, Tuple # NOQA
from sphinx.builders import Builder # NOQA
from sphinx.environment import BuildEnvironment # NOQA
from sphinx.themes import Theme # NOQA
def _tobool(val):
# type: (unicode) -> bool
if isinstance(val, string_types):
return val.lower() in ('true', '1', 'yes', 'on')
return bool(val)
def _toint(val):
# type: (unicode) -> int
try:
return int(val)
except ValueError:
@ -36,6 +46,7 @@ def _toint(val):
def _slice_index(values, slices):
# type: (List, int) -> Iterator[List]
seq = list(values)
length = 0
for value in values:
@ -57,6 +68,7 @@ def _slice_index(values, slices):
def accesskey(context, key):
# type: (Any, unicode) -> unicode
"""Helper to output each access key only once."""
if '_accesskeys' not in context:
context.vars['_accesskeys'] = {}
@ -68,12 +80,15 @@ def accesskey(context, key):
class idgen(object):
def __init__(self):
# type: () -> None
self.id = 0
def current(self):
# type: () -> int
return self.id
def __next__(self):
# type: () -> int
self.id += 1
return self.id
next = __next__ # Python 2/Jinja compatibility
@ -86,6 +101,7 @@ class SphinxFileSystemLoader(FileSystemLoader):
"""
def get_source(self, environment, template):
# type: (BuildEnvironment, unicode) -> Tuple[unicode, unicode, Callable]
for searchpath in self.searchpath:
filename = path.join(searchpath, template)
f = open_if_exists(filename)
@ -97,6 +113,7 @@ class SphinxFileSystemLoader(FileSystemLoader):
mtime = path.getmtime(filename)
def uptodate():
# type: () -> bool
try:
return path.getmtime(filename) == mtime
except OSError:
@ -113,6 +130,7 @@ class BuiltinTemplateLoader(TemplateBridge, BaseLoader):
# TemplateBridge interface
def init(self, builder, theme=None, dirs=None):
# type: (Builder, Theme, List[unicode]) -> None
# create a chain of paths to search
if theme:
# the theme's own dir and its bases' dirs
@ -155,17 +173,21 @@ class BuiltinTemplateLoader(TemplateBridge, BaseLoader):
builder.app.translator)
def render(self, template, context):
# type: (unicode, Dict) -> None
return self.environment.get_template(template).render(context)
def render_string(self, source, context):
# type: (unicode, Dict) -> unicode
return self.environment.from_string(source).render(context)
def newest_template_mtime(self):
# type: () -> float
return max(mtimes_of_files(self.pathchain, '.html'))
# Loader interface
def get_source(self, environment, template):
# type: (BuildEnvironment, unicode) -> Tuple[unicode, unicode, Callable]
loaders = self.loaders
# exclamation mark starts search from theme
if template.startswith('!'):

Some files were not shown because too many files have changed in this diff Show More