Doc: move API docs in their own chapter and add more of it.

doc/contents.rst

@@ -18,6 +18,7 @@ Sphinx documentation contents
    theming
    templating
    extensions
+   extdev/index
    websupport
 
    faq

doc/extdev/appapi.rst

@@ -1,14 +1,27 @@
 .. highlight:: rest
 
-Extension API
-=============
+Application API
+===============
+
+.. module:: sphinx.application
+   :synopsis: Application class and extensibility interface.
 
-.. currentmodule:: sphinx.application
 
 Each Sphinx extension is a Python module with at least a :func:`setup` function.
 This function is called at initialization time with one argument, the
-application object representing the Sphinx process.  This application object has
-the following public API:
+application object representing the Sphinx process.
+
+.. class:: Sphinx
+
+   This application object has the public API described in the following.
+
+Extension setup
+---------------
+
+These methods are usually called in an extension's ``setup()`` function.
+
+Examples of using the Sphinx extension API can be seen in the :mod:`sphinx.ext`
+package.
 
 .. method:: Sphinx.setup_extension(name)
 
@@ -293,6 +306,14 @@ the following public API:
 
    .. versionadded:: 1.1
 
+.. method:: Sphinx.require_sphinx(version)
+
+   Compare *version* (which must be a ``major.minor`` version string,
+   e.g. ``'1.1'``) with the version of the running Sphinx, and abort the build
+   when it is too old.
+
+   .. versionadded:: 1.0
+
 .. method:: Sphinx.connect(event, callback)
 
    Register *callback* to be called when *event* is emitted.  For details on
@@ -306,6 +327,16 @@ the following public API:
 
    Unregister callback *listener_id*.
 
+
+.. exception:: ExtensionError
+
+   All these methods raise this exception if something went wrong with the
+   extension API.
+
+
+Emitting events
+---------------
+
 .. method:: Sphinx.emit(event, *arguments)
 
    Emit *event* and pass *arguments* to the callback functions.  Return the
@@ -319,22 +350,21 @@ the following public API:
 
    .. versionadded:: 0.5
 
-.. method:: Sphinx.require_sphinx(version)
 
-   Compare *version* (which must be a ``major.minor`` version string,
-   e.g. ``'1.1'``) with the version of the running Sphinx, and abort the build
-   when it is too old.
+Producing messages / logging
+----------------------------
 
-   .. versionadded:: 1.0
+The application object also provides support for emitting leveled messages.
 
+.. automethod:: Sphinx.warn
 
-.. exception:: ExtensionError
+.. automethod:: Sphinx.info
 
-   All these functions raise this exception if something went wrong with the
-   extension API.
+.. automethod:: Sphinx.verbose
 
-Examples of using the Sphinx extension API can be seen in the :mod:`sphinx.ext`
-package.
+.. automethod:: Sphinx.debug
+
+.. automethod:: Sphinx.debug2
 
 
 .. _events:
@@ -478,6 +508,21 @@ Use this to adapt your extension to API changes in Sphinx.
       Before version 1.2, check the string ``sphinx.__version__``.
 
 
+The Config object
+-----------------
+
+.. module:: sphinx.config
+
+.. class:: Config
+
+   The config object makes the values of all config values available as
+   attributes.
+
+   It is available as the ``config`` attribute on the application and
+   environment objects.  For example, to get the value of :confval:`language`,
+   use either ``app.config.language`` or ``env.config.language``.
+
+
 .. _template-bridge:
 
 The template bridge
@@ -487,19 +532,3 @@ The template bridge
 
 .. autoclass:: TemplateBridge
    :members:
-
-
-.. _domain-api:
-
-Domain API
-----------
-
-.. module:: sphinx.domains
-
-.. autoclass:: Domain
-   :members:
-
-.. autoclass:: ObjType
-
-.. autoclass:: Index
-   :members:

doc/extdev/builderapi.rst

@@ -1,7 +1,7 @@
 .. _writing-builders:
 
-Writing new builders
-====================
+Builder API
+===========
 
 .. todo:: Expand this.
 

doc/extdev/domainapi.rst

@@ -0,0 +1,14 @@
+.. _domain-api:
+
+Domain API
+----------
+
+.. module:: sphinx.domains
+
+.. autoclass:: Domain
+   :members:
+
+.. autoclass:: ObjType
+
+.. autoclass:: Index
+   :members:

doc/extdev/envapi.rst

@@ -0,0 +1,54 @@
+Build environment API
+=====================
+
+.. module:: sphinx.environment
+
+.. class:: BuildEnvironment
+
+   **Attributes**
+
+   .. attribute:: app
+
+      Reference to the application object.
+
+   .. attribute:: config
+
+      Reference to the :class:`.Config` object.
+
+   .. attribute:: srcdir
+
+      Source directory (the directory containing ``conf.py``).
+
+   .. attribute:: doctreedir
+
+      Directory for storing pickled doctrees.
+
+   .. attribute:: found_docs
+
+      A set of all existing docnames.
+
+   .. attribute:: metadata
+
+      Dictionary mapping docnames to "metadata" (see :ref:`metadata`).
+
+   .. attribute:: titles
+
+      Dictionary mapping docnames to the docutils node for their main title.
+
+   .. autoattribute:: docname
+
+   **Utility methods**
+
+   .. automethod:: warn
+
+   .. automethod:: warn_node
+
+   .. automethod:: doc2path
+
+   .. automethod:: relfn2path
+
+   .. automethod:: note_dependency
+
+   .. automethod:: new_serialno
+
+   .. automethod:: note_reread

doc/extdev/index.rst

@@ -0,0 +1,33 @@
+.. _dev-extensions:
+
+Developing extensions for Sphinx
+================================
+
+Since many projects will need special features in their documentation, Sphinx is
+designed to be extensible on several levels.
+
+This is what you can do in an extension: First, you can add new
+:term:`builder`\s to support new output formats or actions on the parsed
+documents.  Then, it is possible to register custom reStructuredText roles and
+directives, extending the markup.  And finally, there are so-called "hook
+points" at strategic places throughout the build process, where an extension can
+register a hook and run specialized code.
+
+An extension is simply a Python module.  When an extension is loaded, Sphinx
+imports this module and executes its ``setup()`` function, which in turn
+notifies Sphinx of everything the extension offers -- see the extension tutorial
+for examples.
+
+The configuration file itself can be treated as an extension if it contains a
+``setup()`` function.  All other extensions to load must be listed in the
+:confval:`extensions` configuration value.
+
+.. toctree::
+
+   tutorial
+   appapi
+   envapi
+   builderapi
+   markupapi
+   domainapi
+   nodes

doc/extdev/markupapi.rst

@@ -0,0 +1,11 @@
+Docutils markup API
+===================
+
+Roles
+-----
+
+
+Directives
+----------
+
+TODO.

doc/extdev/nodes.rst

@@ -0,0 +1,57 @@
+.. _nodes:
+
+Doctree node classes added by Sphinx
+====================================
+
+.. module:: sphinx.addnodes
+
+Nodes for domain-specific object descriptions
+---------------------------------------------
+
+.. autoclass:: desc
+.. autoclass:: desc_signature
+.. autoclass:: desc_addname
+.. autoclass:: desc_type
+.. autoclass:: desc_returns
+.. autoclass:: desc_name
+.. autoclass:: desc_parameterlist
+.. autoclass:: desc_parameter
+.. autoclass:: desc_optional
+.. autoclass:: desc_annotation
+.. autoclass:: desc_content
+
+New admonition-like constructs
+------------------------------
+
+.. autoclass:: versionmodified
+.. autoclass:: seealso
+
+Other paragraph-level nodes
+-------------------------------
+
+.. autoclass:: compact_paragraph
+
+New inline nodes
+----------------
+
+.. autoclass:: index
+.. autoclass:: pending_xref
+.. autoclass:: literal_emphasis
+.. autoclass:: abbreviation
+.. autoclass:: download_reference
+
+Special nodes
+-------------
+
+.. autoclass:: only
+.. autoclass:: meta
+.. autoclass:: highlightlang
+
+You should not need to generate the nodes below in extensions.
+
+.. autoclass:: glossary
+.. autoclass:: toctree
+.. autoclass:: start_of_file
+.. autoclass:: productionlist
+.. autoclass:: production
+.. autoclass:: termsep

doc/extdev/tutorial.rst

@@ -12,6 +12,50 @@ include todo entries in the documentation, and collecting these in a central
 place.  (A similar "todo" extension is distributed with Sphinx.)
 
 
+Important objects
+-----------------
+
+There are several key objects whose API you will use while writing an
+extension.  These are:
+
+**Application**
+   The application object (usually called ``app``) is an instance of
+   :class:`.Sphinx`.  It controls the most high-level functionality, such as the
+   setup of extensions, event dispatching and producing output (logging).
+
+   If you have the environment object, the application is available as
+   ``env.app``.
+
+**Environment**
+   The build environment object (usually called ``env``) is an instance of
+   :class:`.BuildEnvironment`.  It is responsible for parsing the source
+   documents stores all metadata about the document collection and is serialized
+   after each build.
+
+   Its API provides methods to do with access to metadata, resolving references,
+   etc.  It can also be used by extensions to cache information that should
+   persist for incremental rebuilds.
+
+   If you have the application or builder object, the environment is available
+   as ``app.env`` or ``builder.env``.
+
+**Builder**
+   The builder object (usually called ``builder``) is an instance of a specific
+   subclass of :class:`.Builder`.  Each builder class knows how to convert the
+   parsed documents into an output format, or otherwise process them (e.g. check
+   external links).
+
+   If you have the application object, the environment is available as
+   ``app.builder``.
+
+**Config**
+   The config object (usually called ``config``) provides the values of
+   configuration values set in :file:`conf.py` as attributes.  It is an instance
+   of :class:`.Config`.
+
+   The config is available as ``app.config`` or ``env.config``.
+
+
 Build Phases
 ------------
 
@@ -29,11 +73,15 @@ in which a Sphinx project is built: this works in several phases.
 
    In Phase 1, all source files (and on subsequent builds, those that are new or
    changed) are read and parsed.  This is the phase where directives and roles
-   are encountered by the docutils, and the corresponding functions are called.
-   The output of this phase is a *doctree* for each source files, that is a tree
-   of docutils nodes.  For document elements that aren't fully known until all
+   are encountered by the docutils, and the corresponding code is executed.  The
+   output of this phase is a *doctree* for each source files, that is a tree of
+   docutils nodes.  For document elements that aren't fully known until all
    existing files are read, temporary nodes are created.
 
+   There are nodes provided by docutils, which are documented `in the docutils
+   documentation <http://docutils.sourceforge.net/docs/ref/doctree.html>`__.
+   Additional nodes are provided by Sphinx and :ref:`documented here <nodes>`.
+
    During reading, the build environment is updated with all meta- and cross
    reference data of the read documents, such as labels, the names of headings,
    described Python objects and index entries.  This will later be used to
@@ -165,21 +213,26 @@ docutils classes defined in :mod:`docutils.nodes`.  ``todo`` inherits from
 ``Admonition`` because it should be handled like a note or warning, ``todolist``
 is just a "general" node.
 
+.. note::
+
+   Many extensions will not have to create their own node classes and work fine
+   with the nodes already provided by `docutils
+   <http://docutils.sourceforge.net/docs/ref/doctree.html>`__ and :ref:`Sphinx
+   <nodes>`.
+
 
 The Directive Classes
 ---------------------
 
 A directive class is a class deriving usually from
-``docutils.parsers.rst.Directive``.  Since the class-based directive interface
-doesn't exist yet in Docutils 0.4, Sphinx has another base class called
-``sphinx.util.compat.Directive`` that you can derive your directive from, and it
-will work with both Docutils 0.4 and 0.5 upwards.  The directive interface is
-covered in detail in the `docutils documentation`_; the important thing is that the
-class has a method ``run`` that returns a list of nodes.
+``docutils.parsers.rst.Directive``.  The directive interface is covered in
+detail in the `docutils documentation`_; the important thing is that the class
+has attributes that configure the allowed markup and a method ``run`` that
+returns a list of nodes.
 
 The ``todolist`` directive is quite simple::
 
-   from sphinx.util.compat import Directive
+   from docutils.parsers.rst import Directive
 
    class TodolistDirective(Directive):
 
@@ -224,9 +277,9 @@ to the build environment instance using ``self.state.document.settings.env``.
 
 Then, to act as a link target (from the todolist), the todo directive needs to
 return a target node in addition to the todo node.  The target ID (in HTML, this
-will be the anchor name) is generated by using ``env.new_serialno`` which is
-returns a new integer directive on each call and therefore leads to unique
-target names.  The target node is instantiated without any text (the first two
+will be the anchor name) is generated by using ``env.new_serialno`` which
+returns a new unique integer on each call and therefore leads to unique target
+names.  The target node is instantiated without any text (the first two
 arguments).
 
 An admonition is created using a standard docutils function (wrapped in Sphinx

doc/extensions.rst

@@ -3,34 +3,12 @@
 Sphinx Extensions
 =================
 
-.. module:: sphinx.application
-   :synopsis: Application class and extensibility interface.
-
-Since many projects will need special features in their documentation, Sphinx is
-designed to be extensible on several levels.
-
-This is what you can do in an extension: First, you can add new
-:term:`builder`\s to support new output formats or actions on the parsed
-documents.  Then, it is possible to register custom reStructuredText roles and
-directives, extending the markup.  And finally, there are so-called "hook
-points" at strategic places throughout the build process, where an extension can
-register a hook and run specialized code.
-
-An extension is simply a Python module.  When an extension is loaded, Sphinx
-imports this module and executes its ``setup()`` function, which in turn
-notifies Sphinx of everything the extension offers -- see the extension tutorial
-for examples.
-
-The configuration file itself can be treated as an extension if it contains a
-``setup()`` function.  All other extensions to load must be listed in the
-:confval:`extensions` configuration value.
-
-.. toctree::
-
-   ext/tutorial
-   ext/appapi
-   ext/builderapi
+Since many projects will need special features in their documentation, Sphinx
+allows to add "extensions" to the build process, each of which can modify almost
+any aspect of document processing.
 
+This chapter describes the extensions bundled with Sphinx.  For the API
+documentation on writing your own extension, see :ref:`dev-extensions`.
 
 Builtin Sphinx extensions
 -------------------------

sphinx/application.py

@@ -235,6 +235,17 @@ class Sphinx(object):
             wfile.flush()
 
     def warn(self, message, location=None, prefix='WARNING: '):
+        """Emit a warning.
+
+        If *location* is given, it should either be a tuple of (docname, lineno)
+        or a string describing the location of the warning as well as possible.
+
+        *prefix* usually should not be changed.
+
+        .. note:: For warnings emitted during parsing, you should use
+                  :meth:`.BuildEnvironment.warn` since that will collect all
+                  warnings during parsing for later output.
+        """
         if isinstance(location, tuple):
             docname, lineno = location
             if docname:
@@ -249,9 +260,22 @@ class Sphinx(object):
         self._log(warntext, self._warning, True)
 
     def info(self, message='', nonl=False):
+        """Emit an informational message.
+
+        If *nonl* is true, don't emit a newline at the end (which implies that
+        more info output will follow soon.)
+        """
         self._log(message, self._status, nonl)
 
     def verbose(self, message, *args, **kwargs):
+        """Emit a verbose informational message.
+
+        The message will only be emitted for verbosity levels >= 1 (i.e. at
+        least one ``-v`` option was given).
+
+        The message can contain %-style interpolation placeholders, which is
+        formatted with either the ``*args`` or ``**kwargs`` when output.
+        """
         if self.verbosity < 1:
             return
         if args or kwargs:
@@ -259,6 +283,14 @@ class Sphinx(object):
         self._log(message, self._status)
 
     def debug(self, message, *args, **kwargs):
+        """Emit a debug-level informational message.
+
+        The message will only be emitted for verbosity levels >= 2 (i.e. at
+        least two ``-v`` options were given).
+
+        The message can contain %-style interpolation placeholders, which is
+        formatted with either the ``*args`` or ``**kwargs`` when output.
+        """
         if self.verbosity < 2:
             return
         if args or kwargs:
@@ -266,6 +298,14 @@ class Sphinx(object):
         self._log(darkgray(message), self._status)
 
     def debug2(self, message, *args, **kwargs):
+        """Emit a lowlevel debug-level informational message.
+
+        The message will only be emitted for verbosity level 3 (i.e. three
+        ``-v`` options were given).
+
+        The message can contain %-style interpolation placeholders, which is
+        formatted with either the ``*args`` or ``**kwargs`` when output.
+        """
         if self.verbosity < 3:
             return
         if args or kwargs:

sphinx/environment.py

@@ -246,10 +246,17 @@ class BuildEnvironment:
         self.versioning_condition = condition
 
     def warn(self, docname, msg, lineno=None):
+        """Emit a warning.
+
+        This differs from using ``app.warn()`` in that the warning may not
+        be emitted instantly, but collected for emitting all warnings after
+        the update of the environment.
+        """
         # strange argument order is due to backwards compatibility
         self._warnfunc(msg, (docname, lineno))
 
     def warn_node(self, msg, node):
+        """Like :meth:`warn`, but with source information taken from *node*."""
         self._warnfunc(msg, '%s:%s' % get_source_line(node))
 
     def clear_doc(self, docname):
@@ -693,7 +700,7 @@ class BuildEnvironment:
 
     @property
     def docname(self):
-        """Backwards compatible alias."""
+        """Returns the docname of the document currently being parsed."""
         return self.temp_data['docname']
 
     @property
@@ -707,16 +714,28 @@ class BuildEnvironment:
         return self.temp_data.get('py:class')
 
     def new_serialno(self, category=''):
-        """Return a serial number, e.g. for index entry targets."""
+        """Return a serial number, e.g. for index entry targets.
+
+        The number is guaranteed to be unique in the current document.
+        """
         key = category + 'serialno'
         cur = self.temp_data.get(key, 0)
         self.temp_data[key] = cur + 1
         return cur
 
     def note_dependency(self, filename):
+        """Add *filename* as a dependency of the current document.
+
+        This means that the document will be rebuilt if this file changes.
+
+        *filename* should be absolute or relative to the source directory.
+        """
         self.dependencies.setdefault(self.docname, set()).add(filename)
 
     def note_reread(self):
+        """Add the current document to the list of documents that will
+        automatically be re-read at the next build.
+        """
         self.reread_always.add(self.docname)
 
     def note_versionchange(self, type, version, node, lineno):
@@ -735,7 +754,6 @@ class BuildEnvironment:
                 self.app.debug('%s [filtered system message]', node.astext())
                 node.parent.remove(node)
 
-
     def process_dependencies(self, docname, doctree):
         """Process docutils-generated dependency info."""
         cwd = os.getcwd()