doc: Rework "showing code" section

This makes the section more consistent with the rest of the document.
This adds a new section to the basics guide for doctest blocks, which
are a docutils thing. It also update the default highlight language,
which is now 'default'.

Signed-off-by: Stephen Finucane <stephen@that.guru>

doc/usage/restructuredtext/basics.rst

@@ -112,6 +112,8 @@ There are also several more special blocks available:
 * doctest blocks (:duref:`ref <doctest-blocks>`)
 
 
+.. _rst-literal-blocks:
+
 Literal blocks
 --------------
 
@@ -147,6 +149,19 @@ using the :confval:`highlight_language` configuration option. The
 block-by-block basis. These directives are discussed later.
 
 
+.. _rst-doctest-blocks:
+
+Doctest blocks
+--------------
+
+Doctest blocks (:duref:`ref <doctest-blocks>`) are interactive Python sessions
+cut-and-pasted into docstrings. They do not require the
+:ref:`literal blocks <rst-literal-blocks>` syntax. The doctest block must end
+with a blank line and should *not* end with with an unused prompt::
+
+    >>> 1 + 1
+    2
+
 .. _rst-tables:
 
 Tables

doc/usage/restructuredtext/directives.rst

@@ -395,37 +395,51 @@ Showing code examples
 .. index:: pair: code; examples
            single: sourcecode
 
-.. todo:: Rework this to remove the bullet points
+There are multiple ways to show syntax-highlighted literal code blocks in
+Sphinx: using :ref:`reST doctest blocks <rst-doctest-blocks>`; using :ref:`reST
+literal blocks <rst-literal-blocks>`, optionally in combination with the
+:rst:dir:`highlight` directive; using the :rst:dir:`code-block` directive; and
+using the :rst:dir:`literalinclude` directive. Doctest blocks can only be used
+to show interactive Python sessions, while the remaining three can be used for
+other languages. Of these three, literal blocks are useful when an entire
+document, or at least large sections of it, use code blocks with the same
+syntax and which should be styled in the same manner. On the other hand, the
+:rst:dir:`code-block` directive makes more sense when you want more fine-tuned
+control over the styling of each block or when you have a document containing
+code blocks using multiple varied syntaxes. Finally, the
+:rst:dir:`literalinclude` directive is useful for including entire code files
+in your documentation.
 
-Examples of Python source code or interactive sessions are represented using
+In all cases, Syntax highlighting is provided by `Pygments
-standard reST literal blocks.  They are started by a ``::`` at the end of the
+<http://pygments.org>`_. When using literal blocks, this is configured using
-preceding paragraph and delimited by indentation.
+any :rst:dir:`highlight` directives in the source file. When a ``highlight``
+directive is encountered, it is used until the next ``highlight`` directive is
+encountered. If there is no ``highlight`` directive in the file, the global
+highlighting language is used. This defaults to ``python`` but can be
+configured using the :confval:`highlight_language` config value. The following
+values are supported:
 
-Representing an interactive session requires including the prompts and output
+* ``none`` (no highlighting)
-along with the Python code.  No special markup is required for interactive
+* ``default`` (similar to ``python3`` but with a fallback to ``none`` without
-sessions.  After the last line of input or output presented, there should not
+  warning highlighting fails; the default when :confval:`highlight_language`
-be an "unused" primary prompt; this is an example of what *not* to do::
+  isn't set)
+* ``guess`` (let Pygments guess the lexer based on contents, only works with
+  certain well-recognizable languages)
+* ``python``
+* ``rest``
+* ``c``
+* ... and any other `lexer alias that Pygments supports`__
 
-   >>> 1 + 1
+If highlighting with the selected language fails (i.e. Pygments emits an
-   2
+"Error" token), the block is not highlighted in any way.
-   >>>
 
-Syntax highlighting is done with `Pygments <http://pygments.org>`_ and handled
+.. important::
-in a smart way:
 
-* There is a "highlighting language" for each source file.  Per default, this
+   The list of lexer aliases supported is tied to the Pygment version. If you
-  is ``'python'`` as the majority of files will have to highlight Python
+   want to ensure consistent highlighting, you should fix your version of
-  snippets, but the doc-wide default can be set with the
+   Pygments.
-  :confval:`highlight_language` config value.
 
-* Within Python highlighting mode, interactive sessions are recognized
+__ http://pygments.org/docs/lexers/
-  automatically and highlighted appropriately.  Normal Python code is only
-  highlighted if it is parseable (so you can use Python as the default, but
-  interspersed snippets of shell commands or other code blocks will not be
-  highlighted as Python).
-
-* The highlighting language can be changed using the ``highlight`` directive,
-  used as follows:
 
 .. rst:directive:: .. highlight:: language
 
@@ -433,52 +447,36 @@ in a smart way:
 
       .. highlight:: c
 
-     This language is used until the next ``highlight`` directive is
+   This language is used until the next ``highlight`` directive is encountered.
-     encountered.
+   As discussed previously, *language* can be any lexer alias supported by
+   Pygments.
 
-* For documents that have to show snippets in different languages, there's also
+   **Additional options**
-  a :rst:dir:`code-block` directive that is given the highlighting language
-  directly:
 
-  .. rst:directive:: .. code-block:: language
+   Pygments can generate line numbers for code blocks.  To enable this, use the
-
+   ``linenothreshold`` option. ::
-     Use it like this::
-
-        .. code-block:: ruby
-
-           Some Ruby code.
-
-     The directive's alias name :rst:dir:`sourcecode` works as well.
-
-* The valid values for the highlighting language are:
-
-  * ``none`` (no highlighting)
-  * ``python`` (the default when :confval:`highlight_language` isn't set)
-  * ``guess`` (let Pygments guess the lexer based on contents, only works with
-    certain well-recognizable languages)
-  * ``rest``
-  * ``c``
-  * ... and any other `lexer alias that Pygments supports
-    <http://pygments.org/docs/lexers/>`_.
-
-* If highlighting with the selected language fails (i.e. Pygments emits an
-  "Error" token), the block is not highlighted in any way.
-
-Line numbers
-^^^^^^^^^^^^
-
-Pygments can generate line numbers for code blocks.  For
-automatically-highlighted blocks (those started by ``::``), line numbers must
-be switched on in a :rst:dir:`highlight` directive, with the
-``linenothreshold`` option::
 
       .. highlight:: python
          :linenothreshold: 5
 
    This will produce line numbers for all code blocks longer than five lines.
 
-For :rst:dir:`code-block` blocks, a ``linenos`` flag option can be given to
+.. rst:directive:: .. code-block:: language
-switch on line numbers for the individual block::
+
+   Example::
+
+      .. code-block:: ruby
+
+         Some Ruby code.
+
+   The directive's alias name :rst:dir:`sourcecode` works as well.  As with
+   :rst:dir:`highlight`\ 's ``language`` option, ``language`` can be any lexer
+   alias supported by Pygments.
+
+   **Additional options**
+
+   Pygments can generate line numbers for code blocks. To enable this for, use
+   the ``linenos`` flag option. ::
 
       .. code-block:: ruby
          :linenos:
@@ -505,18 +503,34 @@ emphasize particular lines::
            print 'This one is not...'
            print '...but this one is.'
 
+   A ``caption`` option can be given to show that name before the code block.
+   A ``name`` option can be provided implicit target name that can be
+   referenced by using :rst:role:`ref`.  For example::
+
+     .. code-block:: python
+        :caption: this.py
+        :name: this-py
+
+        print 'Explicit is better than implicit.'
+
+   A ``dedent`` option can be given to strip indentation characters from the
+   code block. For example::
+
+      .. code-block:: ruby
+         :dedent: 4
+
+             some ruby code
+
   .. versionchanged:: 1.1
-   ``emphasize-lines`` has been added.
+     The ``emphasize-lines`` option has been added.
 
   .. versionchanged:: 1.3
-   ``lineno-start`` has been added.
+     The ``lineno-start``, ``caption``, ``name`` and ``dedent`` options have
+     been added.
 
   .. versionchanged:: 1.6.6
      LaTeX supports the ``emphasize-lines`` option.
 
-Includes
-^^^^^^^^
-
 .. rst:directive:: .. literalinclude:: filename
 
    Longer displays of verbatim text may be included by storing the example text
@@ -530,20 +544,26 @@ Includes
    it is absolute (starting with ``/``), it is relative to the top source
    directory.
 
-   Tabs in the input are expanded if you give a ``tab-width`` option with the
+   **Additional options**
-   desired tab width.
 
    Like :rst:dir:`code-block`, the directive supports the ``linenos`` flag
    option to switch on line numbers, the ``lineno-start`` option to select the
    first line number, the ``emphasize-lines`` option to emphasize particular
-   lines, and a ``language`` option to select a language different from the
+   lines, the ``name`` option to provide an implicit target name, the
-   current file's standard language.  Example with options::
+   ``dedent`` option to strip indentation characters for the code block, and a
+   ``language`` option to select a language different from the current file's
+   standard language. In addition, it supports the ``caption`` option; however,
+   this can be provided with no argument to use the filename as the caption.
+   Example with options::
 
       .. literalinclude:: example.rb
          :language: ruby
          :emphasize-lines: 12,15-18
          :linenos:
 
+   Tabs in the input are expanded if you give a ``tab-width`` option with the
+   desired tab width.
+
    Include files are assumed to be encoded in the :confval:`source_encoding`.
    If the file has a different encoding, you can specify it with the
    ``encoding`` option::
@@ -602,56 +622,24 @@ Includes
    This shows the diff between ``example.py`` and ``example.py.orig`` with
    unified diff format.
 
-   .. versionadded:: 0.4.3
+   .. versionchanged:: 0.4.3
-      The ``encoding`` option.
+      Added the ``encoding`` option.
-   .. versionadded:: 0.6
+
-      The ``pyobject``, ``lines``, ``start-after`` and ``end-before`` options,
+   .. versionchanged:: 0.6
-      as well as support for absolute filenames.
+      Added the ``pyobject``, ``lines``, ``start-after`` and ``end-before``
-   .. versionadded:: 1.0
+      options, as well as support for absolute filenames.
-      The ``prepend`` and ``append`` options, as well as ``tab-width``.
+
-   .. versionadded:: 1.3
+   .. versionchanged:: 1.0
-      The ``diff`` option.
+      Added the ``prepend``, ``append``, and ``tab-width`` options.
-      The ``lineno-match`` option.
+
+   .. versionchanged:: 1.3
+      Added the ``diff``, ``lineno-match``, ``caption``, ``name``, and
+      ``dedent`` options.
+
    .. versionchanged:: 1.6
       With both ``start-after`` and ``lines`` in use, the first line as per
       ``start-after`` is considered to be with line number ``1`` for ``lines``.
 
-Caption and name
-^^^^^^^^^^^^^^^^
-
-.. versionadded:: 1.3
-
-A ``caption`` option can be given to show that name before the code block.
-A ``name`` option can be provided implicit target name that can be referenced
-by using :rst:role:`ref`.
-For example::
-
-   .. code-block:: python
-      :caption: this.py
-      :name: this-py
-
-      print 'Explicit is better than implicit.'
-
-:rst:dir:`literalinclude` also supports the ``caption`` and ``name`` option.
-``caption`` has an additional feature that if you leave the value empty, the
-shown filename will be exactly the one given as an argument.
-
-Dedent
-^^^^^^
-
-.. versionadded:: 1.3
-
-A ``dedent`` option can be given to strip indentation characters from the code
-block. For example::
-
-   .. literalinclude:: example.rb
-      :language: ruby
-      :dedent: 4
-      :lines: 10-15
-
-:rst:dir:`code-block` also supports the ``dedent`` option.
-
-
 .. _glossary-directive:
 
 Glossary