Merge pull request #5678 from ArtFlag/docs/extension-tutorials

Docs: Extension tutorial portal + helloworld example

doc/contents.rst

@@ -25,6 +25,7 @@ Sphinx documentation contents
    templating
    latex
    extdev/index
+   development/tutorials/index
 
    faq
    glossary

doc/develop.rst

@@ -100,7 +100,7 @@ This is the current list of contributed extensions in that repository:
 - zopeext: provide an ``autointerface`` directive for using `Zope interfaces`_
 
 
-See the :ref:`extension tutorial <exttut>` on getting started with writing your
+See the :doc:`extension tutorials <../development/tutorials/index>` on getting started with writing your
 own extensions.
 
 

doc/development/tutorials/helloworld.rst

@@ -0,0 +1,162 @@
+Developing a "Hello world" directive
+====================================
+
+The objective of this tutorial is to create a very basic extension that adds a new
+directive that outputs a paragraph containing `hello world`.
+
+Only basic information is provided in this tutorial. For more information,
+refer to the :doc:`other tutorials <index>` that go into more
+details.
+
+.. warning:: For this extension, you will need some basic understanding of docutils_
+   and Python.
+
+Creating a new extension file
+-----------------------------
+
+Your extension file could be in any folder of your project. In our case,
+let's do the following:
+
+#. Create an :file:`_ext` folder in :file:`source`.
+#. Create a new Python file in the :file:`_ext` folder called
+   :file:`helloworld.py`.
+
+   Here is an example of the folder structure you might obtain:
+
+   .. code-block:: text
+
+         └── source
+             ├── _ext
+             │   └── helloworld.py
+             ├── _static
+             ├── _themes
+             ├── conf.py
+             ├── somefolder
+             ├── somefile.rst
+             └── someotherfile.rst
+
+Writing the extension
+---------------------
+
+Open :file:`helloworld.py` and paste the following code in it:
+
+.. code-block:: python
+
+   from docutils import nodes
+   from docutils.parsers.rst import Directive
+
+
+   class HelloWorld(Directive):
+       def run(self):
+           paragraph_node = nodes.paragraph(text='Hello World!')
+           return [paragraph_node]
+
+
+   def setup(app):
+       app.add_directive("helloworld", HelloWorld)
+
+
+Some essential things are happening in this example, and you will see them
+in all directives:
+
+.. rubric:: Directive declaration
+
+Our new directive is declared in the ``HelloWorld`` class, it extends
+docutils_' ``Directive`` class. All extensions that create directives
+should extend this class.
+
+.. rubric:: ``run`` method
+
+This method is a requirement and it is part of every directive. It contains
+the main logic of the directive and it returns a list of docutils nodes to
+be processed by Sphinx.
+
+.. seealso::
+
+   :doc:`todo`
+
+.. rubric:: docutils nodes
+
+The ``run`` method returns a list of nodes. Nodes are docutils' way of
+representing the content of a document. There are many types of nodes
+available: text, paragraph, reference, table, etc.
+
+.. seealso::
+
+   `The docutils documentation on nodes <docutils nodes>`_
+
+The ``nodes.paragraph`` class creates a new paragraph node. A paragraph
+node typically contains some text that we can set during instantiation using
+the ``text`` parameter.
+
+.. rubric:: ``setup`` function
+
+This function is a requirement. We use it to plug our new directive into
+Sphinx.
+The simplest thing you can do it call the ``app.add_directive`` method.
+
+.. note::
+
+   The first argument is the name of the directive itself as used in an rST file.
+
+   In our case, we would use ``helloworld``:
+
+   .. code-block:: rst
+
+      Some intro text here...
+
+      .. helloworld::
+
+      Some more text here...
+
+
+Updating the conf.py file
+-------------------------
+
+The extension file has to be declared in your :file:`conf.py` file to make
+Sphinx aware of it:
+
+#. Open :file:`conf.py`. It is in the :file:`source` folder by default.
+#. Add ``sys.path.append(os.path.abspath("./_ext"))`` before
+   the ``extensions`` variable declaration (if it exists).
+#. Update or create the ``extensions`` list and add the
+   extension file name to the list:
+
+   .. code-block:: python
+
+      extensions.append('helloworld')
+
+You can now use the extension.
+
+.. admonition:: Example
+
+   .. code-block:: rst
+
+      Some intro text here...
+
+      .. helloworld::
+
+      Some more text here...
+
+   The sample above would generate:
+
+   .. code-block:: text
+
+      Some intro text here...
+
+      Hello World!
+
+      Some more text here...
+
+This is the very basic principle of an extension that creates a new directive.
+
+For a more advanced example, refer to :doc:`todo`.
+
+Further reading
+---------------
+
+You can create your own nodes if needed, refer to the
+:doc:`todo` for more information.
+
+.. _docutils: http://docutils.sourceforge.net/
+.. _`docutils nodes`: http://docutils.sourceforge.net/docs/ref/doctree.html

doc/development/tutorials/index.rst

@@ -0,0 +1,11 @@
+Extension tutorials
+===================
+
+Refer to the following tutorials to get started with extension development.
+
+.. toctree::
+   :caption: Directive tutorials
+   :maxdepth: 1
+
+   helloworld
+   todo

doc/development/tutorials/todo.rst

@@ -1,7 +1,5 @@
-.. _exttut:
+Developing a "TODO" extension
-
+=============================
-Tutorial: Writing a simple extension
-====================================
 
 This section is intended as a walkthrough for the creation of custom extensions.
 It covers the basics of writing and activating an extension, as well as
@@ -12,112 +10,12 @@ include todo entries in the documentation, and to collect these in a central
 place.  (A similar "todo" extension is distributed with Sphinx.)
 
 
-Important objects
------------------
-
-There are several key objects whose API you will use while writing an
-extension.  These are:
-
-**Application**
-   The application object (usually called ``app``) is an instance of
-   :class:`.Sphinx`.  It controls most high-level functionality, such as the
-   setup of extensions, event dispatching and producing output (logging).
-
-   If you have the environment object, the application is available as
-   ``env.app``.
-
-**Environment**
-   The build environment object (usually called ``env``) is an instance of
-   :class:`.BuildEnvironment`.  It is responsible for parsing the source
-   documents, stores all metadata about the document collection and is
-   serialized to disk after each build.
-
-   Its API provides methods to do with access to metadata, resolving references,
-   etc.  It can also be used by extensions to cache information that should
-   persist for incremental rebuilds.
-
-   If you have the application or builder object, the environment is available
-   as ``app.env`` or ``builder.env``.
-
-**Builder**
-   The builder object (usually called ``builder``) is an instance of a specific
-   subclass of :class:`.Builder`.  Each builder class knows how to convert the
-   parsed documents into an output format, or otherwise process them (e.g. check
-   external links).
-
-   If you have the application object, the builder is available as
-   ``app.builder``.
-
-**Config**
-   The config object (usually called ``config``) provides the values of
-   configuration values set in :file:`conf.py` as attributes.  It is an instance
-   of :class:`.Config`.
-
-   The config is available as ``app.config`` or ``env.config``.
-
-
-Build Phases
-------------
-
-One thing that is vital in order to understand extension mechanisms is the way
-in which a Sphinx project is built: this works in several phases.
-
-**Phase 0: Initialization**
-
-   In this phase, almost nothing of interest to us happens.  The source
-   directory is searched for source files, and extensions are initialized.
-   Should a stored build environment exist, it is loaded, otherwise a new one is
-   created.
-
-**Phase 1: Reading**
-
-   In Phase 1, all source files (and on subsequent builds, those that are new or
-   changed) are read and parsed.  This is the phase where directives and roles
-   are encountered by docutils, and the corresponding code is executed.  The
-   output of this phase is a *doctree* for each source file; that is a tree of
-   docutils nodes.  For document elements that aren't fully known until all
-   existing files are read, temporary nodes are created.
-
-   There are nodes provided by docutils, which are documented `in the docutils
-   documentation <http://docutils.sourceforge.net/docs/ref/doctree.html>`__.
-   Additional nodes are provided by Sphinx and :ref:`documented here <nodes>`.
-
-   During reading, the build environment is updated with all meta- and cross
-   reference data of the read documents, such as labels, the names of headings,
-   described Python objects and index entries.  This will later be used to
-   replace the temporary nodes.
-
-   The parsed doctrees are stored on the disk, because it is not possible to
-   hold all of them in memory.
-
-**Phase 2: Consistency checks**
-
-   Some checking is done to ensure no surprises in the built documents.
-
-**Phase 3: Resolving**
-
-   Now that the metadata and cross-reference data of all existing documents is
-   known, all temporary nodes are replaced by nodes that can be converted into
-   output using components called tranform.  For example, links are created for
-   object references that exist, and simple literal nodes are created for those
-   that don't.
-
-**Phase 4: Writing**
-
-   This phase converts the resolved doctrees to the desired output format, such
-   as HTML or LaTeX.  This happens via a so-called docutils writer that visits
-   the individual nodes of each doctree and produces some output in the process.
-
-.. note::
-
-   Some builders deviate from this general build plan, for example, the builder
-   that checks external links does not need anything more than the parsed
-   doctrees and therefore does not have phases 2--4.
-
-
 Extension Design
 ----------------
 
+.. note:: To understand the design this extension, refer to
+   :ref:`important-objects` and :ref:`build-phases`.
+
 We want the extension to add the following to Sphinx:
 
 * A "todo" directive, containing some content that is marked with "TODO", and
@@ -174,12 +72,13 @@ the individual calls do is the following:
 
   If the third argument was ``'html'``, HTML documents would be full rebuild if the
   config value changed its value.  This is needed for config values that
-  influence reading (build phase 1).
+  influence reading (build :ref:`phase 1 <build-phases>`).
 
 * :meth:`~Sphinx.add_node` adds a new *node class* to the build system.  It also
   can specify visitor functions for each supported output format.  These visitor
-  functions are needed when the new nodes stay until phase 4 -- since the
+  functions are needed when the new nodes stay until :ref:`phase 4 <build-phases>`
-  ``todolist`` node is always replaced in phase 3, it doesn't need any.
+  -- since the ``todolist`` node is always replaced in :ref:`phase 3 <build-phases>`,
+  it doesn't need any.
 
   We need to create the two node classes ``todo`` and ``todolist`` later.
 
@@ -276,7 +175,7 @@ The ``todo`` directive function looks like this::
            return [targetnode, todo_node]
 
 Several important things are covered here. First, as you can see, you can refer
-to the build environment instance using ``self.state.document.settings.env``.
+to the :ref:`build environment instance <important-objects>` using ``self.state.document.settings.env``.
 
 Then, to act as a link target (from the todolist), the todo directive needs to
 return a target node in addition to the todo node.  The target ID (in HTML, this
@@ -340,7 +239,8 @@ Here we clear out all todos whose docname matches the given one from the
 added again during parsing.
 
 The other handler belongs to the :event:`doctree-resolved` event.  This event is
-emitted at the end of phase 3 and allows custom resolving to be done::
+emitted at the end of :ref:`phase 3 <build-phases>` and allows custom resolving
+to be done::
 
    def process_todo_nodes(app, doctree, fromdocname):
        if not app.config.todo_include_todos:

doc/extdev/index.rst

@@ -52,6 +52,115 @@ Note that it is still necessary to register the builder using
 
 .. _entry points: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins
 
+.. _important-objects:
+
+Important objects
+-----------------
+
+There are several key objects whose API you will use while writing an
+extension. These are:
+
+**Application**
+   The application object (usually called ``app``) is an instance of
+   :class:`.Sphinx`.  It controls most high-level functionality, such as the
+   setup of extensions, event dispatching and producing output (logging).
+
+   If you have the environment object, the application is available as
+   ``env.app``.
+
+**Environment**
+   The build environment object (usually called ``env``) is an instance of
+   :class:`.BuildEnvironment`.  It is responsible for parsing the source
+   documents, stores all metadata about the document collection and is
+   serialized to disk after each build.
+
+   Its API provides methods to do with access to metadata, resolving references,
+   etc.  It can also be used by extensions to cache information that should
+   persist for incremental rebuilds.
+
+   If you have the application or builder object, the environment is available
+   as ``app.env`` or ``builder.env``.
+
+**Builder**
+   The builder object (usually called ``builder``) is an instance of a specific
+   subclass of :class:`.Builder`.  Each builder class knows how to convert the
+   parsed documents into an output format, or otherwise process them (e.g. check
+   external links).
+
+   If you have the application object, the builder is available as
+   ``app.builder``.
+
+**Config**
+   The config object (usually called ``config``) provides the values of
+   configuration values set in :file:`conf.py` as attributes.  It is an instance
+   of :class:`.Config`.
+
+   The config is available as ``app.config`` or ``env.config``.
+
+To see an example of use of these objects, refer to :doc:`../development/tutorials/index`.
+
+.. _build-phases:
+
+Build Phases
+------------
+
+One thing that is vital in order to understand extension mechanisms is the way
+in which a Sphinx project is built: this works in several phases.
+
+**Phase 0: Initialization**
+
+   In this phase, almost nothing of interest to us happens.  The source
+   directory is searched for source files, and extensions are initialized.
+   Should a stored build environment exist, it is loaded, otherwise a new one is
+   created.
+
+**Phase 1: Reading**
+
+   In Phase 1, all source files (and on subsequent builds, those that are new or
+   changed) are read and parsed.  This is the phase where directives and roles
+   are encountered by docutils, and the corresponding code is executed.  The
+   output of this phase is a *doctree* for each source file; that is a tree of
+   docutils nodes.  For document elements that aren't fully known until all
+   existing files are read, temporary nodes are created.
+
+   There are nodes provided by docutils, which are documented `in the docutils
+   documentation <http://docutils.sourceforge.net/docs/ref/doctree.html>`__.
+   Additional nodes are provided by Sphinx and :ref:`documented here <nodes>`.
+
+   During reading, the build environment is updated with all meta- and cross
+   reference data of the read documents, such as labels, the names of headings,
+   described Python objects and index entries.  This will later be used to
+   replace the temporary nodes.
+
+   The parsed doctrees are stored on the disk, because it is not possible to
+   hold all of them in memory.
+
+**Phase 2: Consistency checks**
+
+   Some checking is done to ensure no surprises in the built documents.
+
+**Phase 3: Resolving**
+
+   Now that the metadata and cross-reference data of all existing documents is
+   known, all temporary nodes are replaced by nodes that can be converted into
+   output using components called tranform.  For example, links are created for
+   object references that exist, and simple literal nodes are created for those
+   that don't.
+
+**Phase 4: Writing**
+
+   This phase converts the resolved doctrees to the desired output format, such
+   as HTML or LaTeX.  This happens via a so-called docutils writer that visits
+   the individual nodes of each doctree and produces some output in the process.
+
+.. note::
+
+   Some builders deviate from this general build plan, for example, the builder
+   that checks external links does not need anything more than the parsed
+   doctrees and therefore does not have phases 2--4.
+
+To see an example of application, refer to :doc:`../development/tutorials/todo`.
+
 .. _ext-metadata:
 
 Extension metadata
@@ -82,8 +191,8 @@ APIs used for writing extensions
 --------------------------------
 
 .. toctree::
+   :maxdepth: 2
 
-   tutorial
    appapi
    projectapi
    envapi

doc/faq.rst

@@ -30,7 +30,7 @@ How do I...
    ``sidebartoc`` block.
 
 ... write my own extension?
-   See the :ref:`extension tutorial <exttut>`.
+   See the :doc:`/development/tutorials/index`.
 
 ... convert from my existing docs using MoinMoin markup?
    The easiest way is to convert to xhtml, then convert `xhtml to reST`_.