Merge branch 'master' into 5660-version-modified-classes

This commit is contained in:
Timotheus Kampik 2018-12-24 14:57:42 +01:00
commit b779975a44
18 changed files with 457 additions and 145 deletions

View File

@ -197,6 +197,7 @@ Bugs fixed
possibility to use original meaning in place of Sphinx custom one
* #5834: apidoc: wrong help for ``--tocfile``
* #5800: todo: crashed if todo is defined in TextElement
* #5846: htmlhelp: convert hex escaping to decimal escaping in .hhc/.hhk files
Testing
--------

View File

@ -25,6 +25,7 @@ Sphinx documentation contents
templating
latex
extdev/index
development/tutorials/index
faq
glossary

View File

@ -100,7 +100,7 @@ This is the current list of contributed extensions in that repository:
- zopeext: provide an ``autointerface`` directive for using `Zope interfaces`_
See the :ref:`extension tutorial <exttut>` on getting started with writing your
See the :doc:`extension tutorials <../development/tutorials/index>` on getting started with writing your
own extensions.

View File

@ -0,0 +1,162 @@
Developing a "Hello world" directive
====================================
The objective of this tutorial is to create a very basic extension that adds a new
directive that outputs a paragraph containing `hello world`.
Only basic information is provided in this tutorial. For more information,
refer to the :doc:`other tutorials <index>` that go into more
details.
.. warning:: For this extension, you will need some basic understanding of docutils_
and Python.
Creating a new extension file
-----------------------------
Your extension file could be in any folder of your project. In our case,
let's do the following:
#. Create an :file:`_ext` folder in :file:`source`.
#. Create a new Python file in the :file:`_ext` folder called
:file:`helloworld.py`.
Here is an example of the folder structure you might obtain:
.. code-block:: text
└── source
   ├── _ext
  └── helloworld.py
   ├── _static
   ├── _themes
   ├── conf.py
   ├── somefolder
   ├── somefile.rst
   └── someotherfile.rst
Writing the extension
---------------------
Open :file:`helloworld.py` and paste the following code in it:
.. code-block:: python
from docutils import nodes
from docutils.parsers.rst import Directive
class HelloWorld(Directive):
def run(self):
paragraph_node = nodes.paragraph(text='Hello World!')
return [paragraph_node]
def setup(app):
app.add_directive("helloworld", HelloWorld)
Some essential things are happening in this example, and you will see them
in all directives:
.. rubric:: Directive declaration
Our new directive is declared in the ``HelloWorld`` class, it extends
docutils_' ``Directive`` class. All extensions that create directives
should extend this class.
.. rubric:: ``run`` method
This method is a requirement and it is part of every directive. It contains
the main logic of the directive and it returns a list of docutils nodes to
be processed by Sphinx.
.. seealso::
:doc:`todo`
.. rubric:: docutils nodes
The ``run`` method returns a list of nodes. Nodes are docutils' way of
representing the content of a document. There are many types of nodes
available: text, paragraph, reference, table, etc.
.. seealso::
`The docutils documentation on nodes <docutils nodes>`_
The ``nodes.paragraph`` class creates a new paragraph node. A paragraph
node typically contains some text that we can set during instantiation using
the ``text`` parameter.
.. rubric:: ``setup`` function
This function is a requirement. We use it to plug our new directive into
Sphinx.
The simplest thing you can do it call the ``app.add_directive`` method.
.. note::
The first argument is the name of the directive itself as used in an rST file.
In our case, we would use ``helloworld``:
.. code-block:: rst
Some intro text here...
.. helloworld::
Some more text here...
Updating the conf.py file
-------------------------
The extension file has to be declared in your :file:`conf.py` file to make
Sphinx aware of it:
#. Open :file:`conf.py`. It is in the :file:`source` folder by default.
#. Add ``sys.path.append(os.path.abspath("./_ext"))`` before
the ``extensions`` variable declaration (if it exists).
#. Update or create the ``extensions`` list and add the
extension file name to the list:
.. code-block:: python
extensions.append('helloworld')
You can now use the extension.
.. admonition:: Example
.. code-block:: rst
Some intro text here...
.. helloworld::
Some more text here...
The sample above would generate:
.. code-block:: text
Some intro text here...
Hello World!
Some more text here...
This is the very basic principle of an extension that creates a new directive.
For a more advanced example, refer to :doc:`todo`.
Further reading
---------------
You can create your own nodes if needed, refer to the
:doc:`todo` for more information.
.. _docutils: http://docutils.sourceforge.net/
.. _`docutils nodes`: http://docutils.sourceforge.net/docs/ref/doctree.html

View File

@ -0,0 +1,11 @@
Extension tutorials
===================
Refer to the following tutorials to get started with extension development.
.. toctree::
:caption: Directive tutorials
:maxdepth: 1
helloworld
todo

View File

@ -1,7 +1,5 @@
.. _exttut:
Tutorial: Writing a simple extension
====================================
Developing a "TODO" extension
=============================
This section is intended as a walkthrough for the creation of custom extensions.
It covers the basics of writing and activating an extension, as well as
@ -12,112 +10,12 @@ include todo entries in the documentation, and to collect these in a central
place. (A similar "todo" extension is distributed with Sphinx.)
Important objects
-----------------
There are several key objects whose API you will use while writing an
extension. These are:
**Application**
The application object (usually called ``app``) is an instance of
:class:`.Sphinx`. It controls most high-level functionality, such as the
setup of extensions, event dispatching and producing output (logging).
If you have the environment object, the application is available as
``env.app``.
**Environment**
The build environment object (usually called ``env``) is an instance of
:class:`.BuildEnvironment`. It is responsible for parsing the source
documents, stores all metadata about the document collection and is
serialized to disk after each build.
Its API provides methods to do with access to metadata, resolving references,
etc. It can also be used by extensions to cache information that should
persist for incremental rebuilds.
If you have the application or builder object, the environment is available
as ``app.env`` or ``builder.env``.
**Builder**
The builder object (usually called ``builder``) is an instance of a specific
subclass of :class:`.Builder`. Each builder class knows how to convert the
parsed documents into an output format, or otherwise process them (e.g. check
external links).
If you have the application object, the builder is available as
``app.builder``.
**Config**
The config object (usually called ``config``) provides the values of
configuration values set in :file:`conf.py` as attributes. It is an instance
of :class:`.Config`.
The config is available as ``app.config`` or ``env.config``.
Build Phases
------------
One thing that is vital in order to understand extension mechanisms is the way
in which a Sphinx project is built: this works in several phases.
**Phase 0: Initialization**
In this phase, almost nothing of interest to us happens. The source
directory is searched for source files, and extensions are initialized.
Should a stored build environment exist, it is loaded, otherwise a new one is
created.
**Phase 1: Reading**
In Phase 1, all source files (and on subsequent builds, those that are new or
changed) are read and parsed. This is the phase where directives and roles
are encountered by docutils, and the corresponding code is executed. The
output of this phase is a *doctree* for each source file; that is a tree of
docutils nodes. For document elements that aren't fully known until all
existing files are read, temporary nodes are created.
There are nodes provided by docutils, which are documented `in the docutils
documentation <http://docutils.sourceforge.net/docs/ref/doctree.html>`__.
Additional nodes are provided by Sphinx and :ref:`documented here <nodes>`.
During reading, the build environment is updated with all meta- and cross
reference data of the read documents, such as labels, the names of headings,
described Python objects and index entries. This will later be used to
replace the temporary nodes.
The parsed doctrees are stored on the disk, because it is not possible to
hold all of them in memory.
**Phase 2: Consistency checks**
Some checking is done to ensure no surprises in the built documents.
**Phase 3: Resolving**
Now that the metadata and cross-reference data of all existing documents is
known, all temporary nodes are replaced by nodes that can be converted into
output using components called tranform. For example, links are created for
object references that exist, and simple literal nodes are created for those
that don't.
**Phase 4: Writing**
This phase converts the resolved doctrees to the desired output format, such
as HTML or LaTeX. This happens via a so-called docutils writer that visits
the individual nodes of each doctree and produces some output in the process.
.. note::
Some builders deviate from this general build plan, for example, the builder
that checks external links does not need anything more than the parsed
doctrees and therefore does not have phases 2--4.
Extension Design
----------------
.. note:: To understand the design this extension, refer to
:ref:`important-objects` and :ref:`build-phases`.
We want the extension to add the following to Sphinx:
* A "todo" directive, containing some content that is marked with "TODO", and
@ -174,12 +72,13 @@ the individual calls do is the following:
If the third argument was ``'html'``, HTML documents would be full rebuild if the
config value changed its value. This is needed for config values that
influence reading (build phase 1).
influence reading (build :ref:`phase 1 <build-phases>`).
* :meth:`~Sphinx.add_node` adds a new *node class* to the build system. It also
can specify visitor functions for each supported output format. These visitor
functions are needed when the new nodes stay until phase 4 -- since the
``todolist`` node is always replaced in phase 3, it doesn't need any.
functions are needed when the new nodes stay until :ref:`phase 4 <build-phases>`
-- since the ``todolist`` node is always replaced in :ref:`phase 3 <build-phases>`,
it doesn't need any.
We need to create the two node classes ``todo`` and ``todolist`` later.
@ -276,7 +175,7 @@ The ``todo`` directive function looks like this::
return [targetnode, todo_node]
Several important things are covered here. First, as you can see, you can refer
to the build environment instance using ``self.state.document.settings.env``.
to the :ref:`build environment instance <important-objects>` using ``self.state.document.settings.env``.
Then, to act as a link target (from the todolist), the todo directive needs to
return a target node in addition to the todo node. The target ID (in HTML, this
@ -340,7 +239,8 @@ Here we clear out all todos whose docname matches the given one from the
added again during parsing.
The other handler belongs to the :event:`doctree-resolved` event. This event is
emitted at the end of phase 3 and allows custom resolving to be done::
emitted at the end of :ref:`phase 3 <build-phases>` and allows custom resolving
to be done::
def process_todo_nodes(app, doctree, fromdocname):
if not app.config.todo_include_todos:

View File

@ -52,6 +52,115 @@ Note that it is still necessary to register the builder using
.. _entry points: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins
.. _important-objects:
Important objects
-----------------
There are several key objects whose API you will use while writing an
extension. These are:
**Application**
The application object (usually called ``app``) is an instance of
:class:`.Sphinx`. It controls most high-level functionality, such as the
setup of extensions, event dispatching and producing output (logging).
If you have the environment object, the application is available as
``env.app``.
**Environment**
The build environment object (usually called ``env``) is an instance of
:class:`.BuildEnvironment`. It is responsible for parsing the source
documents, stores all metadata about the document collection and is
serialized to disk after each build.
Its API provides methods to do with access to metadata, resolving references,
etc. It can also be used by extensions to cache information that should
persist for incremental rebuilds.
If you have the application or builder object, the environment is available
as ``app.env`` or ``builder.env``.
**Builder**
The builder object (usually called ``builder``) is an instance of a specific
subclass of :class:`.Builder`. Each builder class knows how to convert the
parsed documents into an output format, or otherwise process them (e.g. check
external links).
If you have the application object, the builder is available as
``app.builder``.
**Config**
The config object (usually called ``config``) provides the values of
configuration values set in :file:`conf.py` as attributes. It is an instance
of :class:`.Config`.
The config is available as ``app.config`` or ``env.config``.
To see an example of use of these objects, refer to :doc:`../development/tutorials/index`.
.. _build-phases:
Build Phases
------------
One thing that is vital in order to understand extension mechanisms is the way
in which a Sphinx project is built: this works in several phases.
**Phase 0: Initialization**
In this phase, almost nothing of interest to us happens. The source
directory is searched for source files, and extensions are initialized.
Should a stored build environment exist, it is loaded, otherwise a new one is
created.
**Phase 1: Reading**
In Phase 1, all source files (and on subsequent builds, those that are new or
changed) are read and parsed. This is the phase where directives and roles
are encountered by docutils, and the corresponding code is executed. The
output of this phase is a *doctree* for each source file; that is a tree of
docutils nodes. For document elements that aren't fully known until all
existing files are read, temporary nodes are created.
There are nodes provided by docutils, which are documented `in the docutils
documentation <http://docutils.sourceforge.net/docs/ref/doctree.html>`__.
Additional nodes are provided by Sphinx and :ref:`documented here <nodes>`.
During reading, the build environment is updated with all meta- and cross
reference data of the read documents, such as labels, the names of headings,
described Python objects and index entries. This will later be used to
replace the temporary nodes.
The parsed doctrees are stored on the disk, because it is not possible to
hold all of them in memory.
**Phase 2: Consistency checks**
Some checking is done to ensure no surprises in the built documents.
**Phase 3: Resolving**
Now that the metadata and cross-reference data of all existing documents is
known, all temporary nodes are replaced by nodes that can be converted into
output using components called tranform. For example, links are created for
object references that exist, and simple literal nodes are created for those
that don't.
**Phase 4: Writing**
This phase converts the resolved doctrees to the desired output format, such
as HTML or LaTeX. This happens via a so-called docutils writer that visits
the individual nodes of each doctree and produces some output in the process.
.. note::
Some builders deviate from this general build plan, for example, the builder
that checks external links does not need anything more than the parsed
doctrees and therefore does not have phases 2--4.
To see an example of application, refer to :doc:`../development/tutorials/todo`.
.. _ext-metadata:
Extension metadata
@ -82,8 +191,8 @@ APIs used for writing extensions
--------------------------------
.. toctree::
:maxdepth: 2
tutorial
appapi
projectapi
envapi

View File

@ -30,7 +30,7 @@ How do I...
``sidebartoc`` block.
... write my own extension?
See the :ref:`extension tutorial <exttut>`.
See the :doc:`/development/tutorials/index`.
... convert from my existing docs using MoinMoin markup?
The easiest way is to convert to xhtml, then convert `xhtml to reST`_.

View File

@ -113,8 +113,7 @@ Prompt* (:kbd:`⊞Win-r` and type :command:`cmd`). Once the command prompt is
open, type :command:`python --version` and press Enter. If Python is
available, you will see the version of Python printed to the screen. If you do
not have Python installed, refer to the `Hitchhikers Guide to Python's`__
Python on Windows installation guides. You can install either `Python 3`__ or
`Python 2.7`__. Python 3 is recommended.
Python on Windows installation guides. You must install `Python 3`__.
Once Python is installed, you can install Sphinx using :command:`pip`. Refer
to the :ref:`pip installation instructions <install-pypi>` below for more
@ -122,7 +121,6 @@ information.
__ https://docs.python-guide.org/
__ https://docs.python-guide.org/starting/install3/win/
__ https://docs.python-guide.org/starting/install/win/
.. _install-pypi:

View File

@ -22,26 +22,41 @@ Configuration
To configure your Sphinx project for Markdown support, proceed as follows:
#. Install *recommonmark*::
#. Install the Markdown parser *recommonmark* from its source on GitHub::
pip install recommonmark
pip install git+https://github.com/rtfd/recommonmark
#. Add the Markdown parser to the ``source_parsers`` configuration variable in
your Sphinx configuration file::
.. note::
source_parsers = {
'.md': 'recommonmark.parser.CommonMarkParser',
The configuration as explained here requires recommonmark version
0.5.0.dev or higher, which is at the time of writing not available on
PyPI. If you want to use a released recommonmark version, follow the
instructions in the `Sphinx 1.8 documentation`__.
__ https://www.sphinx-doc.org/en/1.8/usage/markdown.html
#. Add *recommonmark* to the
:confval:`list of configured extensions <extensions>`::
extensions = ['recommonmark']
.. versionchanged:: 1.8
Version 1.8 deprecates and version 3.0 removes the ``source_parsers``
configuration variable that was used by older *recommonmark* versions.
#. If you want to use Markdown files with extensions other than ``.md``, adjust
the :confval:`source_suffix` variable. The following example configures
Sphinx to parse all files with the extensions ``.md`` and ``.txt`` as
Markdown::
source_suffix = {
'.rst': 'restructuredtext',
'.txt': 'markdown',
'.md': 'markdown',
}
You can replace ``.md`` with a filename extension of your choice.
#. Add the Markdown filename extension to the ``source_suffix`` configuration
variable::
source_suffix = ['.rst', '.md']
#. You can further configure *recommonmark* to allow custom syntax that
standard *CommonMark* doesn't support. Read more in the `recommonmark
standard *CommonMark* doesn't support. Read more in the `recommonmark
documentation`__.
__ https://recommonmark.readthedocs.io/en/latest/auto_structify.html

View File

@ -11,6 +11,7 @@
import html
import os
import re
from os import path
from docutils import nodes
@ -25,7 +26,7 @@ from sphinx.util.osutil import make_filename_from_project
if False:
# For type annotation
from typing import Any, Dict, IO, List, Tuple # NOQA
from typing import Any, Dict, IO, List, Match, Tuple # NOQA
from sphinx.application import Sphinx # NOQA
from sphinx.config import Config # NOQA
@ -169,6 +170,24 @@ chm_locales = {
}
def chm_htmlescape(*args, **kwargs):
# type: (*Any, **Any) -> str
"""
chm_htmlescape() is a wrapper of htmlescape().
.hhc/.hhk files don't recognize hex escaping, we need convert
hex escaping to decimal escaping. for example: `&#x27;` -> `&#39;`
htmlescape() may generates a hex escaping `&#x27;` for single
quote `'`, this wrapper fixes this.
"""
def convert(matchobj):
# type: (Match[str]) -> str
codepoint = int(matchobj.group(1), 16)
return '&#%d;' % codepoint
return re.sub(r'&#[xX]([0-9a-fA-F]+);',
convert,
html.escape(*args, **kwargs))
class HTMLHelpBuilder(StandaloneHTMLBuilder):
"""
Builder that also outputs Windows HTML help project, contents and
@ -278,7 +297,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
write_toc(subnode, ullevel)
elif isinstance(node, nodes.reference):
link = node['refuri']
title = html.escape(node.astext()).replace('"', '&quot;')
title = chm_htmlescape(node.astext()).replace('"', '&quot;')
f.write(object_sitemap % (title, link))
elif isinstance(node, nodes.bullet_list):
if ullevel != 0:
@ -308,7 +327,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder):
item = ' <param name="%s" value="%s">\n' % \
(name, value)
f.write(item)
title = html.escape(title)
title = chm_htmlescape(title)
f.write('<LI> <OBJECT type="text/sitemap">\n')
write_param('Keyword', title)
if len(refs) == 0:

View File

@ -1,7 +1,7 @@
# Configuration file for the Sphinx documentation builder.
#
# This file does only contain a selection of the most common options. For a
# full list see the documentation:
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# http://www.sphinx-doc.org/en/master/config
# -- Path setup --------------------------------------------------------------

View File

@ -696,15 +696,12 @@ def rfc1123_to_epoch(rfc1123):
def xmlname_checker():
# type: () -> Pattern
# https://www.w3.org/TR/REC-xml/#NT-Name
# Only Python 3.3 or newer support character code in regular expression
name_start_chars = [
':', ['A', 'Z'], '_', ['a', 'z'], ['\u00C0', '\u00D6'],
['\u00D8', '\u00F6'], ['\u00F8', '\u02FF'], ['\u0370', '\u037D'],
['\u037F', '\u1FFF'], ['\u200C', '\u200D'], ['\u2070', '\u218F'],
['\u2C00', '\u2FEF'], ['\u3001', '\uD7FF'], ['\uF900', '\uFDCF'],
['\uFDF0', '\uFFFD']]
name_start_chars.append(['\U00010000', '\U000EFFFF'])
['\uFDF0', '\uFFFD'], ['\U00010000', '\U000EFFFF']]
name_chars = [
"\\-", "\\.", ['0', '9'], '\u00B7', ['\u0300', '\u036F'],

View File

@ -103,16 +103,12 @@ def getargspec(func):
def isenumclass(x):
# type: (Type) -> bool
"""Check if the object is subclass of enum."""
if enum is None:
return False
return inspect.isclass(x) and issubclass(x, enum.Enum)
def isenumattribute(x):
# type: (Any) -> bool
"""Check if the object is attribute of enum."""
if enum is None:
return False
return isinstance(x, enum.Enum)

View File

@ -0,0 +1,4 @@
# -*- coding: utf-8 -*-
project = 'test'
master_doc = 'index'

View File

@ -0,0 +1,19 @@
Index markup
------------
.. index::
single: entry
pair: entry; pair
double: entry; double
triple: index; entry; triple
keyword: with
see: from; to
seealso: fromalso; toalso
.. index::
!Main, !Other
!single: entry; pair
.. index:: triple-quoted string, Unicode Consortium, raw string
single: """; string literal
single: '''; string literal

View File

@ -0,0 +1,64 @@
@echo off
setlocal
pushd %~dp0
set this=%~n0
if not defined PYTHON set PYTHON=py
if not defined SPHINXBUILD (
%PYTHON% -c "import sphinx" > nul 2> nul
if errorlevel 1 (
echo Installing sphinx with %PYTHON%
%PYTHON% -m pip install sphinx
if errorlevel 1 exit /B
)
set SPHINXBUILD=%PYTHON% -c "import sphinx.cmd.build, sys; sys.exit(sphinx.cmd.build.main())"
)
rem Search for HHC in likely places
set HTMLHELP=
where hhc /q && set HTMLHELP=hhc && goto :skiphhcsearch
where /R ..\externals hhc > "%TEMP%\hhc.loc" 2> nul && set /P HTMLHELP= < "%TEMP%\hhc.loc" & del "%TEMP%\hhc.loc"
if not exist "%HTMLHELP%" where /R "%ProgramFiles(x86)%" hhc > "%TEMP%\hhc.loc" 2> nul && set /P HTMLHELP= < "%TEMP%\hhc.loc" & del "%TEMP%\hhc.loc"
if not exist "%HTMLHELP%" where /R "%ProgramFiles%" hhc > "%TEMP%\hhc.loc" 2> nul && set /P HTMLHELP= < "%TEMP%\hhc.loc" & del "%TEMP%\hhc.loc"
if not exist "%HTMLHELP%" (
echo.
echo.The HTML Help Workshop was not found. Set the HTMLHELP variable
echo.to the path to hhc.exe or download and install it from
echo.http://msdn.microsoft.com/en-us/library/ms669985
exit /B 1
)
echo hhc.exe path: %HTMLHELP%
if "%BUILDDIR%" EQU "" set BUILDDIR=build
%SPHINXBUILD% >nul 2> nul
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
popd
exit /B 1
)
set SPHINXOPTS=-D html_theme_options.body_max_width=none %SPHINXOPTS%
cmd /S /C "%SPHINXBUILD% %SPHINXOPTS% -bhtmlhelp -dbuild\doctrees . "%BUILDDIR%\htmlhelp"
"%HTMLHELP%" "%BUILDDIR%\htmlhelp\test.hhp"
rem hhc.exe seems to always exit with code 1, reset to 0 for less than 2
if not errorlevel 2 cmd /C exit /b 0
echo.
if errorlevel 1 (
echo.Build failed (exit code %ERRORLEVEL%^), check for error messages
echo.above. Any output will be found in %BUILDDIR%\%1
) else (
echo.Build succeeded. All output should be in %BUILDDIR%\%1
)
popd

View File

@ -8,6 +8,8 @@
:license: BSD, see LICENSE for details.
"""
import re
import pytest
from sphinx.builders.htmlhelp import default_htmlhelp_basename
@ -29,3 +31,17 @@ def test_default_htmlhelp_basename():
config = Config({'project': 'Sphinx Documentation'})
config.init_values()
assert default_htmlhelp_basename(config) == 'sphinxdoc'
@pytest.mark.sphinx('htmlhelp', testroot='build-htmlhelp')
def test_chm(app):
app.build()
# check .hhk file
outname = app.builder.config.htmlhelp_basename
hhk_path = str(app.outdir / outname + '.hhk')
with open(hhk_path, 'rb') as f:
data = f.read()
m = re.search(br'&#[xX][0-9a-fA-F]+;', data)
assert m is None, 'Hex escaping exists in .hhk file: ' + str(m.group(0))