Docutils Project Documentation Overview

Author: | David Goodger
Contact: | goodger@python.org
Date: | 2004-06-28
Revision: | 1.15
Copyright: | This document has been placed in the public domain.

The latest working documents may be accessed individually below, or from the docs directory of the development snapshots. They are also distributed with the latest project release package, but those may not be completely up to date.

Contents

  • Docutils Stakeholders
  • Project Fundamentals
  • user/: Introductory & Tutorial Material for End-Users
  • ref/: Reference Material for All Groups
  • peps/: Python Enhancement Proposals
  • api/: API Reference Material for Client-Developers
  • howto/: Instructions for Developers
  • dev/: Development Notes and Plans for Core-Developers

Docutils Stakeholders

Docutils stakeholders can be categorized in several groups:

  1. End-users: users of reStructuredText and the Docutils tools. Although some are developers (e.g. Python developers utilizing reStructuredText for docstrings in their source), many are not.
  2. Client-developers: developers using Docutils as a library, programmers developing with Docutils.
  3. Component-developers: those who implement application-specific components, directives, and/or roles, separately from Docutils.
  4. Core-developers: developers of the Docutils codebase and participants in the Docutils project community.
  5. Re-implementers: developers of alternate implementations of Docutils.

There's a lot of overlap between these groups. Most (perhaps all) core-developers, component-developers, client-developers, and re-implementers are also end-users. Core-developers are also client-developers, and may also be component-developers in other projects. Component-developers are also client-developers.

Project Fundamentals

These files are for all Docutils stakeholders. They are kept at the top level of the Docutils project directory.

README.txt: | Project overview: quick-start, requirements, installation, and usage.
COPYING.txt: | Conditions for Docutils redistribution, with links to licenses.
FAQ.txt: | Docutils Frequently Asked Questions. If you have a question or issue, there's a good chance it's already answered here.
BUGS.txt: | A list of known bugs, and how to report a bug.
HISTORY.txt: | Change history log.
THANKS.txt: | Acknowledgements.

user/: Introductory & Tutorial Material for End-Users

Docutils-general:

  • Docutils Front-End Tools
  • Docutils Configuration Files
  • Docutils LaTeX Writer

reStructuredText:

  • A ReStructuredText Primer (HTML) (or text source)
  • Quick reStructuredText (user reference)
  • reStructuredText Cheat Sheet (text only; 1 page for syntax, 1 page directive & role reference)

ref/: Reference Material for All Groups

Many of these files began as developer specifications, but now that they're mature and used by end-users and client-developers, they have become reference material. Successful specs evolve into refs.

Docutils-general:

  • The Docutils Document Tree (incomplete)
  • Docutils Transforms
  • Docutils Generic DTD
  • OASIS XML Exchange Table Model Declaration Module (CALS tables DTD module)

Although not in the "ref" directory, PEP 258 is a must-read reference for any Docutils developer.

reStructuredText:

  • An Introduction to reStructuredText (includes the Goals and History of reStructuredText)
  • reStructuredText Markup Specification
  • reStructuredText Directives
  • reStructuredText Interpreted Text Roles

Prehistoric:

  • Setext Documents Mirror

peps/: Python Enhancement Proposals

  • PEP 256: Docstring Processing System Framework is a high-level generic proposal. [PEP 256 in the master repository]
  • PEP 257: Docstring Conventions addresses docstring style and touches on content. [PEP 257 in the master repository]
  • PEP 258: Docutils Design Specification is an overview of the architecture of Docutils. It documents design issues and implementation details. [PEP 258 in the master repository]
  • PEP 287: reStructuredText Docstring Format proposes a standard markup syntax. [PEP 287 in the master repository]

Please note that PEPs in the master repository may not be current, whereas the local versions are.

api/: API Reference Material for Client-Developers

  • The Docutils Publisher
  • Inside A Docutils Command-Line Front-End Tool
  • Docutils Runtime Settings
  • (Docutils Transforms should be moved here)

PEP 258 is an overview of the architecture of Docutils.

howto/: Instructions for Developers

  • Docutils Internationalization
  • Creating reStructuredText Directives
  • Creating reStructuredText Interpreted Text Roles

dev/: Development Notes and Plans for Core-Developers

Docutils-general:

  • Docutils To Do List
  • Docutils Project Policies
  • Docutils Web Site
  • Docutils Release Procedure
  • Docutils Testing
  • Docstring Semantics (incomplete)
  • Python Source Reader (incomplete)
  • Docutils Python DTD (experimental)
  • Plan for Enthought API Documentation Tool
  • Enthought API Documentation Tool RFP

reStructuredText:

  • A Record of reStructuredText Syntax Alternatives
  • Problems With StructuredText
View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils Bugs

Author: | David Goodger; open to all Docutils developers
Contact: | goodger@python.org
Date: | 2004-06-29
Revision: | 1.12
Copyright: | This document has been placed in the public domain.

Bugs in Docutils?!? Yes, we do have a few. Some are old-timers that tend to stay in the shadows and don't bother anybody. Once in a while new bugs are born. From time to time some bugs (new and old) crawl out into the light and must be dealt with. Icky.

This document describes how to report a bug, and lists known bugs.

Contents

  • How To Report A Bug
  • Known Bugs

How To Report A Bug

If you think you've discovered a bug, please read through these guidelines before reporting it.

First, make sure it's a new bug:

  • Please check the list of known bugs below and the SourceForge Bug Tracker to see if it has already been reported.

  • Are you using the very latest version of Docutils? The bug may have already been fixed. Please get the latest version of Docutils from CVS or from the development snapshot and check again. Even if your bug has not been fixed, others probably have, and you're better off with the most up-to-date code.

    If you don't have time to check the latest snapshot, please report the bug anyway. We'd rather tell you that it's already fixed than miss reports of unfixed bugs.

  • If Docutils does not behave the way you expect, look in the documentation (don't forget the FAQ!) and mailing list archives for evidence that it should behave the way you expect.

If you're not sure, please ask on the docutils-users@lists.sourceforge.net 1 mailing list first.

If it's a new bug, the most important thing you can do is to write a simple description and a recipe that reproduces the bug. Try to create a minimal document that demonstrates the bug. The easier you make it to understand and track down the bug, the more likely a fix will be.

Now you're ready to write the bug report. Please include:

  • A clear description of the bug. Describe how you expected Docutils to behave, and contrast that with how it actually behaved. While the bug may seem obvious to you, it may not be so obvious to someone else, so it's best to avoid a guessing game.
  • A complete description of the environment in which you reproduced the bug:
    • Your operating system & version.
    • The version of Python (python -V).
    • The version of Docutils (use the "-V" option to most Docutils front-end tools).
    • Any private modifications you made to Docutils.
    • Anything else that could possibly be relevant. Err on the side of too much information, rather than too little.
  • A literal transcript of the exact command you ran, and the exact output. Use the "--traceback" option to get a complete picture.
  • The exact input and output files. Better to attach complete files to your bug report than to include just a summary or excerpt.
  • If you also want to include speculation as to the cause, and even a patch to fix the bug, that would be great!

The best place to send your bug report is to the SourceForge Bug Tracker. That way, it won't be misplaced or forgotten. In fact, an open bug report on SourceForge is a constant irritant that begs to be squashed.

Thank you!

(This section was inspired by the Subversion project's BUGS file.)

[1] Due to overwhelming amounts of spam, the docutils-users@lists.sourceforge.net mailing list has been set up for subscriber posting only. Non-subscribers who post to docutils-users will receive a message with "Subject: Your message to Docutils-users awaits moderator approval". Legitimate messages are accepted and posted as soon as possible (a list administrator must verify the message manually). If you'd like to subscribe to docutils-users, please visit <http://lists.sourceforge.net/lists/listinfo/docutils-users>.

Known Bugs

Also see the SourceForge Bug Tracker.

  • The parser doesn't know anything about double-width characters such as Chinese hanza & Japanese kanji/kana. Also, it's dependent on whitespace and punctuation as markup delimiters, which may not be applicable in these languages.

  • Line numbers in system messages are inconsistent in the parser.

    • In text inserted by the "include" directive, errors are often not reported with the correct "source" or "line" numbers. Perhaps all Reporter calls need "source" and "line" keyword arguments. Elements' .line assignments should be checked. (Assign to .source too? Add a set_info method? To what?) There's a test in test/test_parsers/test_rst/test_directives/test_include.py.
    • Some line numbers in elements are not being set properly (explicitly), just implicitly/automatically. See rev. 1.74 of docutils/parsers/rst/states.py for an example of how to set.

  • Quite a few nodes are getting a "None" source attribute as well. In particular, see the bodies of definition lists.

  • Explicit targets are sometimes mis-located. In particular, placing a target before a section header puts the target at the end of the previous section instead of the start of the next section. The code in docutils.transforms.misc.ClassAttribute could be used to fix this. (Reported by David Priest.)

  • David Abrahams pointed out that doubly-indirect substitutions have a bug, but only when there's multiple references:

    |substitute| my coke for gin
|substitute| you for my mum
at least I'll get my washing done

.. |substitute| replace:: |replace|
.. |replace| replace:: swap

    This is tricky. Substitutions have to propagate back completely.

  • Another bug from David Abrahams (run with rst2html.py --traceback):

    .. [#crtp] See |runtime|

foo [#tag_dispatching]_

.. [#tag_dispatching] See  |runtime|

.. |runtime| replace:: 7__
__ runtime.html

__ reference/__main.html

    Change the references.Substitutions tranform's priority from 220 to 680, so it happens after reference resolution? Then we have to deal with multiple IDs. Perhaps the Substitution transform should remove all IDs from definitions after the first substitution reference is processed.

View document source. Generated on: 2004-07-01 22:43 UTC. Generated by Docutils from reStructuredText source.


Copying Docutils

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Date: | 2004-06-22
Web site: | http://docutils.sourceforge.net/
Copyright: | This document has been placed in the public domain.

Most of the files included in this project have been placed in the public domain, and therefore have no license requirements and no restrictions on copying or usage; see the Public Domain Dedication below. There are a few exceptions, listed below.

One goal of the Docutils project is to be included in the Python standard library distribution, at which time it is expected that copyright will be asserted by the Python Software Foundation.

Public Domain Dedication

The persons who have associated their work with this project (the "Dedicator": David Goodger and the many contributors to the Docutils project) hereby dedicate the entire copyright, less the exceptions listed below, in the work of authorship known as "Docutils" identified below (the "Work") to the public domain.

The primary repository for the Work is the Internet World Wide Web site <http://docutils.sourceforge.net/>. The Work consists of the files within the "docutils" module of the Docutils project CVS repository (Internet host cvs.sourceforge.net, filesystem path /cvsroot/docutils), whose Internet web interface is located at <http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/docutils/docutils/>. Files dedicated to the public domain may be identified by the inclusion, near the beginning of each file, of a declaration of the form:

Copyright: This document/module/DTD/stylesheet/file/etc. has been
           placed in the public domain.

Dedicator makes this dedication for the benefit of the public at large and to the detriment of Dedicator's heirs and successors. Dedicator intends this dedication to be an overt act of relinquishment in perpetuity of all present and future rights under copyright law, whether vested or contingent, in the Work. Dedicator understands that such relinquishment of all rights includes the relinquishment of all rights to enforce (by lawsuit or otherwise) those copyrights in the Work.

Dedicator recognizes that, once placed in the public domain, the Work may be freely reproduced, distributed, transmitted, used, modified, built upon, or otherwise exploited by anyone for any purpose, commercial or non-commercial, and in any way, including by methods that have not yet been invented or conceived.

(This dedication is derived from the text of the Creative Commons Public Domain Dedication.)

Exceptions

The exceptions to the Public Domain Dedication above are:

  • extras/optparse.py, copyright by Gregory P. Ward, released under a BSD-style license (which can be found in the module's source code).
  • extras/textwrap.py, copyright by Gregory P. Ward and the Python Software Foundation, released under the Python 2.3 license (local copy).
  • extras/roman.py, copyright by Mark Pilgrim, released under the Python 2.1.1 license (local copy).
  • test/docutils_difflib.py, copyright by the Python Software Foundation, released under the Python 2.2 license (local copy). This file is included for compatibility with Python versions less than 2.2. (It's only used to report test failures anyhow; it isn't installed anywhere. The included file is a pre-generator version of the difflib.py module included in Python 2.2.)
  • tools/pep2html.py, copyright by the Python Software Foundation, released under the Python 2.2 license (local copy).
  • tools/editors/emacs/rst-html.el, copyright by Martin Blais, released under the GNU General Public License (local copy).
  • tools/editors/emacs/rst-mode.el, copyright by Stefan Merten, released under the GNU General Public License (local copy).

(Disclaimer: I am not a lawyer.) The BSD license and the Python licenses are OSI-approved and GPL-compatible. Although complicated by multiple owners and lots of legalese, the Python license basically lets you copy, use, modify, and redistribute files as long as you keep the copyright attribution intact, note any changes you make, and don't use the owner's name in vain. The BSD license is similar.

Plaintext versions of all the linked-to licenses are provided in the licenses directory.

View document source. Generated on: 2004-07-01 22:43 UTC. Generated by Docutils from reStructuredText source.


Docutils Frequently Asked Questions

Date: | 2004-06-29
Web site: | http://docutils.sourceforge.net/
Copyright: | This document has been placed in the public domain.

Contents

  • 1 Docutils
    • 1.1 What is Docutils?
    • 1.2 Why is it called "Docutils"?
    • 1.3 Is there a GUI authoring environment for Docutils?
    • 1.4 What is the status of the Docutils project?
    • 1.5 What is the Docutils project release policy?
  • 2 reStructuredText
    • 2.1 What is reStructuredText?
    • 2.2 Why is it called "reStructuredText"?
    • 2.3 What's the standard abbreviation for "reStructuredText"?
    • 2.4 What's the standard filename extension for a reStructuredText file?
    • 2.5 Are there any reStructuredText editor extensions?
    • 2.6 How can I indicate the document title? Subtitle?
    • 2.7 How can I represent esoteric characters (e.g. character entities) in a document?
    • 2.8 How can I generate backticks using a Scandinavian keyboard?
    • 2.9 Are there any tools for HTML/XML-to-reStructuredText? (Round-tripping)
    • 2.10 Are there any Wikis that use reStructuredText syntax?
    • 2.11 Are there any Weblog (Blog) projects that use reStructuredText syntax?
    • 2.12 Can lists be indented without generating block quotes?
    • 2.13 Could the requirement for blank lines around lists be relaxed?
    • 2.14 How can I include mathematical equations in documents?
    • 2.15 Is nested inline markup possible?
    • 2.16 How to indicate a line break or a significant newline?
    • 2.17 A URL containing asterisks doesn't work. What to do?
  • 3 HTML Writer
    • 3.1 What is the status of the HTML Writer?
    • 3.2 What kind of HTML does it produce?
    • 3.3 What browsers are supported?
    • 3.4 Unexpected results from tools/html.py: H1, H1 instead of H1, H2. Why?
    • 3.5 Why do enumerated lists only use numbers (no letters or roman numerals)?
  • 4 Python Source Reader
    • 4.1 Can I use Docutils for Python auto-documentation?
  • 5 Miscellaneous
    • 5.1 Is the Docutils document model based on any existing XML models?

This is a work in progress. Please feel free to ask questions and/or provide answers; send email to the Docutils-Users mailing list 1. Project members should feel free to edit the source text file directly.

[1] Due to overwhelming amounts of spam, the docutils-users@lists.sourceforge.net mailing list has been set up for subscriber posting only. Non-subscribers who post to docutils-users will receive a message with "Subject: Your message to Docutils-users awaits moderator approval". Legitimate messages are accepted and posted as soon as possible (a list administrator must verify the message manually). If you'd like to subscribe to docutils-users, please visit <http://lists.sourceforge.net/lists/listinfo/docutils-users>.

1 Docutils

1.1 What is Docutils?

Docutils is a system for processing plaintext documentation into useful formats, such as HTML, XML, and TeX. It supports multiple types of input, such as standalone files (implemented), inline documentation from Python modules and packages (under development), PEPs (Python Enhancement Proposals) (implemented), and others as discovered.

For an overview of the Docutils project implementation, see PEP 258, "Docutils Design Specification".

Docutils is implemented in Python.

1.2 Why is it called "Docutils"?

Docutils is short for "Python Documentation Utilities". The name "Docutils" was inspired by "Distutils", the Python Distribution Utilities architected by Greg Ward, a component of Python's standard library.

The earliest known use of the term "docutils" in a Python context was a fleeting reference in a message by Fred Drake on 1999-12-02 in the Python Doc-SIG mailing list. It was suggested as a project name on 2000-11-27 on Doc-SIG, again by Fred Drake, in response to a question from Tony "Tibs" Ibbs: "What do we want to call this thing?". This was shortly after David Goodger first announced reStructuredText on Doc-SIG.

Tibs used the name "Docutils" for his effort "to document what the Python docutils package should support, with a particular emphasis on documentation strings". Tibs joined the current project (and its predecessors) and graciously donated the name.

For more history of reStructuredText and the Docutils project, see An Introduction to reStructuredText.

Please note that the name is "Docutils", not "DocUtils" or "Doc-Utils" or any other variation.

1.3 Is there a GUI authoring environment for Docutils?

DocFactory is under development. It uses wxPython and looks very promising.

1.4 What is the status of the Docutils project?

Although useful and relatively stable, Docutils is experimental code, with APIs and architecture subject to change.

Our highest priority is to fix bugs as they are reported. So the latest code from CVS (or development snapshots) is almost always the most stable (bug-free) as well as the most featureful.

1.5 What is the Docutils project release policy?

It ought to be "release early & often", but official releases are a significant effort and aren't done that often. We have automatically-generated development snapshots which always contain the latest code from CVS. As the project matures, we may formalize on a stable/development-branch scheme, but we're not using anything like that yet.

If anyone would like to volunteer as a release coordinator, please contact the project coordinator.

2 reStructuredText

2.1 What is reStructuredText?

reStructuredText is an easy-to-read, what-you-see-is-what-you-get plaintext markup syntax and parser system. The reStructuredText parser is a component of Docutils. reStructuredText is a revision and reinterpretation of the StructuredText and Setext lightweight markup systems.

If you are reading this on the web, you can see for yourself. The source for this FAQ is written in reStructuredText; open it in another window and compare them side by side.

A ReStructuredText Primer and the Quick reStructuredText user reference are a good place to start. The reStructuredText Markup Specification is a detailed technical specification.

2.2 Why is it called "reStructuredText"?

The name came from a combination of "StructuredText", one of reStructuredText's predecessors, with "re": "revised", "reworked", and "reinterpreted", and as in the re.py regular expression module. For a detailed history of reStructuredText and the Docutils project, see An Introduction to reStructuredText.

2.3 What's the standard abbreviation for "reStructuredText"?

"RST" and "ReST" (or "reST") are both acceptable. Care should be taken with capitalization, to avoid confusion with "REST", an acronym for "Representational State Transfer".

The abbreviations "reSTX" and "rSTX"/"rstx" should not be used; they overemphasize reStructuredText's precedessor, Zope's StructuredText.

2.4 What's the standard filename extension for a reStructuredText file?

It's ".txt". Some people would like to use ".rest" or ".rst" or ".restx", but why bother? ReStructuredText source files are meant to be readable as plaintext, and most operating systems already associate ".txt" with text files. Using a specialized filename extension would require that users alter their OS settings, which is something that many users will not be willing or able to do.

2.5 Are there any reStructuredText editor extensions?

See Editor Support for reStructuredText.

2.6 How can I indicate the document title? Subtitle?

A uniquely-adorned section title at the beginning of a document is treated specially, as the document title. Similarly, a uniquely-adorned section title immediately after the document title becomes the document subtitle. For example:

This is the Document Title
==========================

This is the Document Subtitle
-----------------------------

Here's an ordinary paragraph.

Counterexample:

Here's an ordinary paragraph.

This is *not* a Document Title
==============================

The "ordinary paragraph" above the section title
prevents it from becoming the document title.

Another counterexample:

This is not the Document Title,  because...
===========================================

Here's an ordinary paragraph.

... the title adornment is not unique
=====================================

Another ordinary paragraph.

2.7 How can I represent esoteric characters (e.g. character entities) in a document?

For example, say you want an em-dash (XML character entity &mdash;, Unicode character \u2014) in your document: use a real em-dash. Insert concrete characters (e.g. type a real em-dash) into your input file, using whatever encoding suits your application, and tell Docutils the input encoding. Docutils uses Unicode internally, so the em-dash character is a real em-dash internally.

ReStructuredText has no character entity subsystem; it doesn't know anything about XML charents. To Docutils, "&mdash;" in input text is 7 discrete characters; no interpretation happens. When writing HTML, the "&" is converted to "&amp;", so in the raw output you'd see "&amp;mdash;". There's no difference in interpretation for text inside or outside inline literals or literal blocks -- there's no character entity interpretation in either case.

If you can't use a Unicode-compatible encoding and must rely on 7-bit ASCII, there is a workaround. Files containing character entity set substitution definitions using the "unicode" directive are available (tarball). A description and instructions for use are here. Thanks to David Priest for the original idea. Incorporating these files into Docutils is on the to-do list.

If you insist on using XML-style charents, you'll have to implement a pre-processing system to convert to UTF-8 or something. That introduces complications though; you can no longer write about charents naturally; instead of writing "&mdash;" you'd have to write "&amp;mdash;".

2.8 How can I generate backticks using a Scandinavian keyboard?

The use of backticks in reStructuredText is a bit awkward with Scandinavian keyboards, where the backtick is a "dead" key. To get one ` character one must press SHIFT-` + SPACE.

Unfortunately, with all the variations out there, there's no way to please everyone. For Scandinavian programmers and technical writers, this is not limited to reStructuredText but affects many languages and environments.

Possible solutions include

  • If you have to input a lot of backticks, simply type one in the normal/awkward way, select it, copy and then paste the rest (CTRL-V is a lot faster than SHIFT-` + SPACE).

  • Use keyboard macros.

  • Remap the keyboard. The Scandinavian keyboard layout is awkward for other programming/technical characters too; for example, []{} etc. are a bit awkward compared to US keyboards.

    According to Axel Kollmorgen,

    Under Windows, you can use the Microsoft Keyboard Layout Creator to easily map the backtick key to a real backtick (no dead key). took me five minutes to load my default (german) keyboard layout, untick "Dead Key?" from the backtick key properties ("in all shift states"), "build dll and setup package", install the generated .msi, and add my custom keyboard layout via Control Panel > Regional and Language Options > Languages > Details > Add Keyboard layout (and setting it as default "when you start your computer").

If anyone knows of other/better solutions, please let us know.

2.9 Are there any tools for HTML/XML-to-reStructuredText? (Round-tripping)

People have tossed the idea around, but little if any actual work has ever been done. There's no reason why reStructuredText should not be round-trippable to/from XML; any technicalities which prevent round-tripping would be considered bugs. Whitespace would not be identical, but paragraphs shouldn't suffer. The tricky parts would be the smaller details, like links and IDs and other bookkeeping.

For HTML, true round-tripping may not be possible. Even adding lots of extra "class" attributes may not be enough. A "simple HTML" to RST filter is possible -- for some definition of "simple HTML" -- but HTML is used as dumb formatting so much that such a filter may not be particularly useful. No general-purpose filter exists. An 80/20 approach should work though: build a tool that does 80% of the work automatically, leaving the other 20% for manual tweaks.

2.10 Are there any Wikis that use reStructuredText syntax?

There are several, with various degrees of completeness. With no implied endorsement or recommendation, and in no particular order:

  • Ian Bicking's experimental code
  • MoinMoin has some support; here's a sample
  • Zope-based Zwiki
  • Zope3-based Zwiki (in the Zope 3 source tree as zope.products.zwiki)
  • StikiWiki
  • Trac supports using reStructuredText as an alternative to wiki markup. This includes support for TracLinks from within RST text via a custom RST reference-directive or, even easier, an interpreted text role 'trac'

Please let us know of any other reStructuredText Wikis.

The example application for the Web Framework Shootout article is a Wiki using reStructuredText.

2.11 Are there any Weblog (Blog) projects that use reStructuredText syntax?

With no implied endorsement or recommendation, and in no particular order:

  • Python Desktop Server
  • PyBloxsom
  • rst2ht
  • htmlnav
  • restblog
  • ReSTWeb
  • Lino WebMan

Please let us know of any other reStructuredText Blogs.

2.12 Can lists be indented without generating block quotes?

Some people like to write lists with indentation, without intending a block quote context, like this:

paragraph

  * list item 1
  * list item 2

There has been a lot of discussion about this, but there are some issues that would need to be resolved before it could be implemented. There is a summary of the issues and pointers to the discussions in the to-do list.

2.13 Could the requirement for blank lines around lists be relaxed?

Short answer: no.

In reStructuredText, it would be impossible to unambigously mark up and parse lists without blank lines before and after. Deeply nested lists may look ugly with so many blank lines, but it's a price we pay for unambiguous markup. Some other plaintext markup systems do not require blank lines in nested lists, but they have to compromise somehow, either accepting ambiguity or requiring extra complexity. For example, Epytext does not require blank lines around lists, but it does require that lists be indented and that ambiguous cases be escaped.

2.14 How can I include mathematical equations in documents?

There is no elegant built-in way, yet. There are several ideas, but no obvious winner. This issue requires a champion to solve the technical and aesthetic issues and implement a generic solution. Here's the to-do list entry.

There are several quick & dirty ways to include equations in documents:

  • For HTML there is MathML but its rendering is still quite broken (or absent) on most browsers. An example of raw MathML is here.
  • If you use LaTeX output only, a simple way is to use the "raw" directive; examples are here.
  • Images of the equations can be used, typically generated by TeX. Beni Cherniavsky has written some preprocessing scripts for making inline TeX (for LaTeX and other [HTML etc.] output) math much easier; instructions here.
  • Here is an example of potential itex math markup.

2.15 Is nested inline markup possible?

Not currently, no. It's on the to-do list (details here), and hopefully will be part of the reStructuredText parser soon. At that time, markup like this will become possible:

Here is some *emphasized text containing a `hyperlink`_ and
``inline literals``*.

There are workarounds, but they are either convoluted or ugly or both. They are not recommended.

  • Inline markup can be combined with hyperlinks using substitution definitions and references with the "replace" directive. For example:

    Here is an |emphasized hyperlink|_.

.. |emphasized hyperlink| replace:: *emphasized hyperlink*
.. _emphasized hyperlink: http://example.org

    It is not possible for just a portion of the replacement text to be a hyperlink; it's the entire replacement text or nothing.

  • The "raw" directive can be used to insert raw HTML into HTML output:

    Here is some |stuff|.

.. |stuff| raw:: html

   <em>emphasized text containing a
   <a href="http://example.org">hyperlink</a> and
   <tt>inline literals</tt></em>

    Raw LaTeX is supported for LaTeX output, etc.

2.16 How to indicate a line break or a significant newline?

The "line-block" directive is designed for address blocks, verse, and other cases where line breaks are significant and must be preserved. Unlike literal blocks, the typeface is not changed, and inline markup is recognized. For example:

.. line-block::

   A one, two, a one two three four

   Half a bee, philosophically,
       must, *ipso facto*, half not be.
   But half the bee has got to be,
       *vis a vis* its entity.  D'you see?

   But can a bee be said to be
       or not to be an entire bee,
           when half the bee is not a bee,
               due to some ancient injury?

   Singing...

In the near future, we will be implementing new syntax for line blocks, like this:

| Lend us a couple of bob till Thursday.
| I'm absolutely skint.
| But I'm expecting a postal order and I can pay you back
  as soon as it comes.
| Love, Ewan.

Here's a workaround for manually inserting explicit line breaks in HTML output:

.. |br| raw:: html

   <br />

I want to break this line here: |br| this is after the break.

If the extra whitespace bothers you, |br|\ backslash-escape it.

2.17 A URL containing asterisks doesn't work. What to do?

Asterisks are valid URL characters (see RFC 2396), sometimes used in URLs. For example:

http://cvs.example.org/viewcvs.py/*checkout*/module/file

Unfortunately, the parser thinks the asterisks are indicating emphasis. The slashes serve as delineating punctuation, allowing the asterisks to be recognized as markup. The example above is separated by the parser into a truncated URL, an emphasized word, and some regular text:

http://cvs.example.org/viewcvs.py/
*checkout*
/module/file

To turn off markup recognition, use a backslash to escape at least the first asterisk, like this:

http://cvs.example.org/viewcvs.py/\*checkout*/module/file

Escaping the second asterisk doesn't hurt, but it isn't necessary.

3 HTML Writer

3.1 What is the status of the HTML Writer?

The HTML Writer module, docutils/writers/html4css1.py, is a proof-of-concept reference implementation. While it is a complete implementation, some aspects of the HTML it produces may be incompatible with older browsers or specialized applications (such as web templating). Alternate implementations are welcome.

3.2 What kind of HTML does it produce?

It produces XHTML compatible with the HTML 4.01 and XHTML 1.0 specifications. A cascading style sheet ("default.css" by default) is required for proper viewing with a modern graphical browser. Correct rendering of the HTML produced depends on the CSS support of the browser.

3.3 What browsers are supported?

No specific browser is targeted; all modern graphical browsers should work. Some older browsers, text-only browsers, and browsers without full CSS support are known to produce inferior results. Mozilla (version 1.0 and up) and MS Internet Explorer (version 5.0 and up) are known to give good results. Reports of experiences with other browsers are welcome.

3.4 Unexpected results from tools/html.py: H1, H1 instead of H1, H2. Why?

Here's the question in full:

I have this text:

Heading 1
=========

All my life, I wanted to be H1.

Heading 1.1
-----------

But along came H1, and so shouldn't I be H2?
No!  I'm H1!

Heading 1.1.1
*************

Yeah, imagine me, I'm stuck at H3!  No?!?

When I run it through tools/html.py, I get unexpected results (below). I was expecting H1, H2, then H3; instead, I get H1, H1, H2:

...
<html lang="en">
<head>
...
<title>Heading 1</title>
<link rel="stylesheet" href="default.css" type="text/css" />
</head>
<body>
<div class="document" id="heading-1">
<h1 class="title">Heading 1</h1>                <-- first H1
<p>All my life, I wanted to be H1.</p>
<div class="section" id="heading-1-1">
<h1><a name="heading-1-1">Heading 1.1</a></h1>        <-- H1
<p>But along came H1, and so now I must be H2.</p>
<div class="section" id="heading-1-1-1">
<h2><a name="heading-1-1-1">Heading 1.1.1</a></h2>
<p>Yeah, imagine me, I'm stuck at H3!</p>
...

What gives?

Check the "class" attribute on the H1 tags, and you will see a difference. The first H1 is actually <h1 class="title">; this is the document title, and the default stylesheet renders it centered. There can also be an <h2 class="subtitle"> for the document subtitle.

If there's only one highest-level section title at the beginning of a document, it is treated specially, as the document title. (Similarly, a lone second-highest-level section title may become the document subtitle.) See How can I indicate the document title? Subtitle? for details. Rather than use a plain H1 for the document title, we use <h1 class="title"> so that we can use H1 again within the document. Why do we do this? HTML only has H1-H6, so by making H1 do double duty, we effectively reserve these tags to provide 6 levels of heading beyond the single document title.

HTML is being used for dumb formatting for nothing but final display. A stylesheet is required, and one is provided: tools/stylesheets/default.css. Of course, you're welcome to roll your own. The default stylesheet provides rules to format <h1 class="title"> and <h2 class="subtitle"> differently from ordinary <h1> and <h2>:

h1.title {
  text-align: center }

h2.subtitle {
  text-align: center }

If you don't want the top section heading to be interpreted as a title at all, disable the doctitle_xform setting (--no-doc-title option). This will interpret your document differently from the standard settings, which might not be a good idea. If you don't like the reuse of the H1 in the HTML output, you can tweak the initial_header_level setting (--initial-header-level option) -- but unless you match its value to your specific document, you might end up with bad HTML (e.g. H3 without H2).

(Thanks to Mark McEahern for the question and much of the answer.)

3.5 Why do enumerated lists only use numbers (no letters or roman numerals)?

The rendering of enumerators (the numbers or letters acting as list markers) is completely governed by the stylesheet, so either the browser can't find the stylesheet (try using the "--embed-stylesheet" option), or the browser can't understand it (try a recent Firefox, Mozilla, Konqueror, Opera, Safari, or even MSIE).

4 Python Source Reader

4.1 Can I use Docutils for Python auto-documentation?

Yes, in conjunction with other projects.

Docstring extraction functionality from within Docutils is still under development. There is most of a source code parsing module in docutils/readers/python/moduleparser.py. We do plan to finish it eventually. Ian Bicking wrote an initial front end for the moduleparser.py module, in sandbox/ianb/extractor/extractor.py. Ian also did some work on the Python Source Reader (docutils.readers.python) component at PyCon DC 2004.

Version 2.0 of Ed Loper's Epydoc supports reStructuredText-format docstrings for HTML output. Docutils 0.3 or newer is required. Development of a Docutils-specific auto-documentation tool will continue. Epydoc works by importing Python modules to be documented, whereas the Docutils-specific tool, described above, will parse modules without importing them (as with HappyDoc, which doesn't support reStructuredText).

The advantages of parsing over importing are security and flexibility; the disadvantage is complexity/difficulty.

  • Security: untrusted code that shouldn't be executed can be parsed; importing a module executes its top-level code.
  • Flexibility: comments and unofficial docstrings (those not supported by Python syntax) can only be processed by parsing.
  • Complexity/difficulty: it's a lot harder to parse and analyze a module than it is to import and analyze one.

For more details, please see "Docstring Extraction Rules" in PEP 258, item 3 ("How").

5 Miscellaneous

5.1 Is the Docutils document model based on any existing XML models?

Not directly, no. It borrows bits from DocBook, HTML, and others. I (David Goodger) have designed several document models over the years, and have my own biases. The Docutils document model is designed for simplicity and extensibility, and has been influenced by the needs of the reStructuredText markup.

View document source. Generated on: 2004-07-01 22:43 UTC. Generated by Docutils from reStructuredText source.


Docutils History

Author: | David Goodger; open to all Docutils developers
Contact: | goodger@python.org
Date: | 2004-06-27
Web site: | http://docutils.sourceforge.net/
Copyright: | This document has been placed in the public domain.

Contents

  • Changes Since 0.3.3
  • Release 0.3.3 (2004-05-09)
  • Release 0.3 (2003-06-24)
  • Release 0.2 (2002-07-31)
  • Release 0.1 (2002-04-20)

Changes Since 0.3.3

General:

  • Documentation cleanup/reorganization.

    • Created new subdirectories of docs/:
      • docs/user/: introductory/tutorial material for end-users
      • docs/dev/: for core-developers (development notes, plans, etc.)
      • docs/api/: API reference material for client-developers
      • docs/ref/: reference material for all groups
      • docs/howto/: for component-developers and core-developers
      • docs/peps/: Python Enhancement Proposals
    • Moved docs/* to docs/user/.
    • Moved pysource.dtd, pysource.txt, semantics.txt from spec/ to docs/dev.
    • Moved doctree.txt, docutils.dtd, soextblx.dtd, transforms.txt from spec/ to docs/ref/.
    • Moved alternatives.txt, and problems.txt from spec/rst/ to docs/dev/rst/.
    • Moved reStructuredText.txt, directives.txt, interpreted.txt, and introduction.txt from spec/rst/ to docs/ref/rst/. Renamed interpreted.txt to roles.txt, reStructuredText.txt to restructuredtext.txt.
    • Moved spec/howto/ to docs/howto.

    In order to keep the CVS history of moved files, we supplied SourceForge with a script for modifying the Docutils CVS repository.

    After running the cleanup script:

    • Added docs/index.txt.
    • Added a .htaccess file to the web module, containing redirects for all old paths to new paths. They'll preserve fragments (the "#name" part of a URL), and won't clutter up the file system, and will correct the URL in the user's browser.
    • Added BUGS.txt, docs/dev/policies.txt, docs/dev/website.txt, docs/dev/release.txt from all but the "To Do" list itself in docs/dev/todo.txt.
    • Moved "Future Plans" from HISTORY.txt to new "Priorities" section of docs/dev/todo.txt.
    • Added THANKS.txt from "Acknowledgements" in HISTORY.txt.
    • Added "How To Report Bugs" to BUGS.txt.
    • Went through all the sources and docs (including under web/) and updated links. Mostly done by Felix Wiemann; thanks Felix! (Still need to update links in the sandboxes.)

Specific:

  • BUGS.txt: Added to project.
  • THANKS.txt: Added to project.
  • docutils/__init__.py:
    • 0.3.4: Post-release.
  • docutils/core.py:
    • Added special error handling & advice for UnicodeEncodeError.
    • Refactored Publisher.publish (simplified exception handling & extracted debug dumps).
    • Renamed "enable_exit" parameter of convenience functions to "enable_exit_status".
    • Enabled traceback (exception propagation) by default in programmatic convenience functions.
    • Now publish_file and publish_cmdline convenience functions return the encoded string results in addition to their regular I/O.
    • Extracted common code from publish_file, publish_string, and publish_parts, into new publish_programmatically. Extracted settings code to Publisher.process_programmatic_settings.
    • In Publisher.publish, disabled settings_overrides when settings is supplied; redundant.
  • docutils/frontend.py:
    • Added help text for "--output-encoding-error-handler" and "--error-encoding-error-handler".
    • Renamed "--exit" to "--exit-status".
    • Simplified default-setting code.
  • docutils/parsers/rst/__init__.py:
    • Added "--pep-base-url" and "--rfc-base-url" options.
  • docutils/parsers/rst/directives/__init__.py:
    • Added several directive option conversion functions.
  • docutils/parsers/rst/directives/body.py:
    • Moved "table" directive to tables.py.
  • docutils/parsers/rst/directives/tables.py: Table-related directives, added to project.
  • docutils/writers/latex2e.py:
    • Added "--table-style=(standard|booktabs|nolines)"
    • figures get "here" option (LaTeX per default puts them at bottom), and figure content is centered.
    • Rowspan support for tables.
    • Fix: admonition titles before first section.
    • Replace -- in literal by -{}- because fontencoding T1 has endash.
    • Replave _ in literal by an underlined blank, because it has the correct width.
    • Fix: encode pdfbookmark titles, # broke pdflatex.
    • A few unicode replacements, if output_encoding != utf
  • docs/index.txt: "Docutils Project Documentation Overview", added to project.
  • docs/api/cmdline-tool.txt: "Inside A Docutils Command-Line Front-End Tool", added to project.
  • docs/api/publisher.txt: "The Docutils Publisher", added to project.
  • docs/api/runtime-settings.txt: "Docutils Runtime Settings", added to project.
  • docs/dev/policies.txt: Added to project (extracted from docs/dev/todo.txt, formerly spec/notes.txt).
  • docs/dev/release.txt: Added to project (extracted from docs/dev/todo.txt, formerly spec/notes.txt).
  • docs/dev/testing.txt: Added to project.
  • docs/dev/website.txt: Added to project (extracted from docs/dev/todo.txt, formerly spec/notes.txt).
  • docs/ref/rst/directives.txt:
    • Added directives: "table", "csv-table".
  • docs/user/rst/cheatsheet.txt: "The reStructuredText Cheat Sheet" added to project. 1 page for syntax, and a 1 page reference for directives and roles. Source text to be used as-is; not meant to be converted to HTML.
  • test/functional/, contents, and test/test_functional.py: Added to project.

Release 0.3.3 (2004-05-09)

  • docutils/__init__.py:
    • 0.3.1: Reorganized config file format (multiple sections); see docs/config.txt.
    • Added unknown_reference_resolvers attribute to TransformSpec.
    • 0.3.2: Interpreted text reorganization.
    • 0.3.3: Released.
  • docutils/core.py:
    • Catch system messages to stop tracebacks from parsing errors.
    • Catch exceptions during processing report & exit without tracebacks, except when "--traceback" used.
    • Reordered components for OptionParser; application comes last.
    • Added "config_section" parameter to several methods and functions, allowing front ends to easily specify their config file sections.
    • Added publish_parts convenience function to allow access to individual parts of a document.
  • docutils/examples.py: Added to project; practical examples of Docutils client code, to be used as-is or as models for variations.
  • docutils/frontend.py:
    • Added "--traceback" & "--no-traceback" options ("traceback" setting).
    • Implemented support for config file reorganization: standard_config_files moved from ConfigParser to OptionParser; added OptionParser.get_config_file_settings() and .get_standard_config_settings(); support for old "[options]" section (with deprecation warning) and mapping from old to new settings.
    • Reimplemented setting validation.
    • Enabled flexible boolean values: yes/no, true/false, on/off.
    • Added Values, a subclass of optparse.Values, with support for list setting attributes.
    • Added support for new DOCUTILSCONFIG environment variable; thanks to Beni Cherniavsky.
    • Added "--no-section-numbering" option.
  • docutils/io.py:
    • Catch IOErrors when opening source & destination files, report & exit without tracebacks. Added handle_io_errors parameter to FileInput & FileOutput to enable caller error handling.
  • docutils/nodes.py:
    • Changed SparseNodeVisitor and GenericNodeVisitor dynamic method definitions (via exec) to dynamic assignments (via setattr); thanks to Roman Suzi.
    • Encapsulated visitor dynamic assignments in a function; thanks to Ian Bicking.
    • Added indirect_reference_name attribute to the Targetable class. This attribute holds the whitespace_normalized_name (contains mixed case) of a target.
  • docutils/statemachine.py:
    • Renamed StringList.strip_indent to .trim_left.
    • Added StringList.get_2D_block.
  • docutils/utils.py:
    • Added "level" attribute to SystemMessage exceptions.
  • docutils/languages/af.py: Added to project; Afrikaans mappings by Jannie Hofmeyr.
  • docutils/languages/cs.py: Added to project; Czech mappings by Marek Blaha.
  • docutils/languages/eo.py: Added to project; Esperanto mappings by Marcelo Huerta San Martin.
  • docutils/languages/pt_br.py: Added to project; Brazilian Portuguese mappings by Lalo Martins.
  • docutils/languages/ru.py: Added to project; Russian mappings by Roman Suzi.
  • docutils/parsers/rst/roles.py: Added to project. Contains interpreted text role functions, a registry for interpreted text roles, and an API for adding to and retrieving from the registry. Contributed by Edward Loper.
  • docutils/parsers/rst/states.py:
    • Updated RSTState.nested_parse for "include" in table cells.
    • Allowed true em-dash character and "---" as block quote attribution marker.
    • Added support for <angle-bracketed> complex option arguments (option lists).
    • Fixed handling of backslashes in substitution definitions.
    • Fixed off-by-1 error with extra whitespace after substitution definition directive.
    • Added inline markup parsing to field lists' field names.
    • Added support for quoted (and unindented) literal blocks. Driven in part by a bribe from Frank Siebenlist (thanks!).
    • Parser now handles escapes in URIs correctly.
    • Made embedded-URIs' reference text omittable. Idea from Beni Cherniavsky.
    • Refactored explicit target processing code.
    • Added name attribute to references containing the reference name only through whitespace_normalize_name (no case changes).
    • parse_target no longer returns the refname after going through normalize_name. This is now handled in make_target.
    • Fixed bug relating to role-less interpreted text in non-English contexts.
    • Reorganized interpreted text processing; moved code into the new roles.py module. Contributed by Edward Loper.
    • Refactored Body.parse_directive into run_directive and parse_directive_block.
  • docutils/parsers/rst/tableparser.py:
    • Reworked for StringList, to support "include" directives in table cells.
  • docutils/parsers/rst/directives/__init__.py:
    • Renamed unchanged() directive option conversion function to unchanged_required, and added a new unchanged.
    • Catch unicode value too high error; fixes bug 781766.
    • Beefed up directive error reporting.
  • docutils/parsers/rst/directives/body.py:
    • Added basic "table" directive.
  • docutils/parsers/rst/directives/images.py:
    • Added "target" option to "image" directive.
    • Added name attribute to references containing the reference name only through whitespace_normalize_name (no case changes).
  • docutils/parsers/rst/directives/misc.py:
    • Isolated the import of the urllib2 module; was causing problems on SourceForge (libssl.so.2 unavailable?).
    • Added the "role" directive for declaring custom interpreted text roles.
  • docutils/parsers/rst/directives/parts.py:
    • The "contents" directive does more work up-front, creating the "topic" and "title", and leaving the "pending" node for the transform. Allows earlier reference resolution; fixes subtle bug.
  • docutils/parsers/rst/languages/af.py: Added to project; Afrikaans mappings by Jannie Hofmeyr.
  • docutils/parsers/rst/languages/cs.py: Added to project; Czech mappings by Marek Blaha.
  • docutils/parsers/rst/languages/eo.py: Added to project; Esperanto mappings by Marcelo Huerta San Martin.
  • docutils/parsers/rst/languages/pt_br.py: Added to project; Brazilian Portuguese mappings by Lalo Martins.
  • docutils/parsers/rst/languages/ru.py: Added to project; Russian mappings by Roman Suzi.
  • docutils/transforms/parts.py:
    • The "contents" directive does more work up-front, creating the "topic" and "title", and leaving the "pending" node for the transform. Allows earlier reference resolution; fixes subtle bug.
    • Added support for disabling of section numbering.
  • docutils/transforms/references.py:
    • Verifying that external targets are truly targets and not indirect references. This is because we are now adding a "name" attribute to references in addition to targets. Note sure if this is correct!
    • Added code to hook into the unknown_reference_resolvers list for a transformer in resolve_indirect_target. This allows the unknown_reference_resolvers to keep around indirect targets which docutils doesn't know about.
    • Added specific error message for duplicate targets.
  • docutils/transforms/universal.py:
    • Added FilterMessages transform (removes system messages below the verbosity threshold).
    • Added hook (via docutils.TransformSpec.unknown_reference_resolvers) to FinalCheckVisitor for application-specific handling of unresolvable references.
    • Added specific error message for duplicate targets.
  • docutils/writers/__init__.py:
    • Added assemble_parts method to the Writer class to allow for access to a documents individual parts.
    • Documented & set default for Writer.output attribute.
  • docutils/writers/html4css1.py:
    • Fixed unicode handling of attribute values (bug 760673).
    • Prevent duplication of "class" attribute values (bug report from Kirill Lapshin).
    • Improved table grid/border handling (prompted by report from Bob Marshall).
    • Added support for table titles.
    • Added "<title />" for untitled docs, for XHTML conformance; thanks to Darek Suchojad.
    • Added functionality to keep track of individual parts of a document and store them in a dictionary as the "parts" attribute of the writer. Contributed by Reggie Dugard at the Docutils sprint at PyCon DC 2004.
    • Added proper support for the "scale" attribute of the "image" element. Contributed by Brent Cook.
    • Added --initial-header-level option.
    • Fixed bug: the body_pre_docinfo segment depended on there being a docinfo; if no docinfo, the document title was incorporated into the body segment. Adversely affected the publish_parts interface.
  • docutils/writers/latex2e.py:
    • Changed default stylesheet to "no stylesheet" to avoid latex complaining about a missing file.
    • Added options and support: --compound-enumerators, --section-prefix-for-enumerators, and --section-enumerator-separator. By John F Meinel Jr (SF patch 934322).
    • Added option --use-verbatim-when-possible, to avoid problematic characters (eg, '~' in italian) in literal blocks.
    • It's now possible to use four section levels in the book and report LaTeX classes. The default article class still has three levels limit.
  • docs/config.txt: "Docutils Configuration Files", added to project. Moved config file entry descriptions from tools.txt.
  • docs/tools.txt:
    • Moved config file entry descriptions to config.txt.
  • spec/notes.txt: Continual updates. Added "Setting Up For Docutils Development".
  • spec/howto/rst-roles.txt: "Creating reStructuredText Interpreted Text Roles", added to project.
  • spec/rst/reStructuredText.txt:
    • Added description of support for <angle-bracketed> complex option arguments to option lists.
    • Added subsections for indented and quoted literal blocks.
  • test: Continually adding & updating tests.
    • Added test/test_settings.py & test/data/config_*.txt support files.
    • Added test/test_writers/test_htmlfragment.py.
  • test/DocutilsTestSupport.py:
    • Refactored LaTeX publisher test suite/case class names to make testing other writers easier.
    • Added HtmlWriterPublishTestCase and HtmlFragmentTestSuite classes to test the processing of HTML fragments which use the new publish_parts convenience function.
  • tools/buildhtml.py:
    • Added support for the "--prune" option.
    • Removed dependency on pep2html.py; plaintext PEPs no longer supported.
  • tools/docutils.conf:
    • Updated for configuration file reorganization.
  • tools/rst2html.py:
    • copied from tools/html.py
  • setup.py:
    • added a 'scripts' section to configuration
    • added 'tools/rst2html.py' to the scripts section

Release 0.3 (2003-06-24)

General:

  • Renamed "attribute" to "option" for directives/extensions.
  • Renamed transform method "transform" to "apply".
  • Renamed "options" to "settings" for runtime settings (as set by command-line options). Sometimes "option" (singular) became "settings" (plural). Some variations below:
    • document.options -> document.settings (stored in other objects as well)
    • option_spec -> settings_spec (not directives though)
    • OptionSpec -> SettingsSpec
    • cmdline_options -> settings_spec
    • relative_path_options -> relative_path_settings
    • option_default_overrides -> settings_default_overrides
    • Publisher.set_options -> Publisher.get_settings

Specific:

  • COPYING.txt: Added "Public Domain Dedication".
  • FAQ.txt: Frequently asked questions, added to project.
  • setup.py:
    • Updated with PyPI Trove classifiers.
    • Conditional installation of third-party modules.
  • docutils/__init__.py:
    • Bumped version to 0.2.1 to reflect changes to I/O classes.
    • Bumped version to 0.2.2 to reflect changes to stylesheet options.
    • Factored SettingsSpec out of Component; separately useful.
    • Bumped version to 0.2.3 because of the new "--embed-stylesheet" option and its effect on the PEP template & writer.
    • Bumped version to 0.2.4 due to changes to the PEP template & stylesheet.
    • Bumped version to 0.2.5 to reflect changes to Reporter output.
    • Added TransformSpec class for new transform system.
    • Bumped version to 0.2.6 for API changes (renaming).
    • Bumped version to 0.2.7 for new docutils.core.publish_* convenience functions.
    • Added Component.component_type attribute.
    • Bumped version to 0.2.8 because of the internal parser switch from plain lists to the docutils.statemachine.StringList objects.
    • Bumped version to 0.2.9 because of the frontend.py API changes.
    • Bumped version to 0.2.10 due to changes to the project layout (third-party modules removed from the "docutils" package), and signature changes in io.Input/io.Output.
    • Changed version to 0.3.0 for release.
  • docutils/core.py:
    • Made publish() a bit more convenient.
    • Generalized Publisher.set_io.
    • Renamed publish() to publish_cmdline(); rearranged its parameters; improved its docstring.
    • Added publish_file() and publish_string().
    • Factored Publisher.set_source() and .set_destination() out of .set_io.
    • Added support for "--dump-pseudo-xml", "--dump-settings", and "--dump-transforms" hidden options.
    • Added Publisher.apply_transforms() method.
    • Added Publisher.set_components() method; support for publish_*() conveninece functions.
    • Moved config file processing to docutils/frontend.py.
    • Added support for exit status ("exit_level" setting & enable_exit parameter for Publisher.publish() and convenience functions).
  • docutils/frontend.py:
    • Check for & exit on identical source & destination paths.
    • Fixed bug with absolute paths & "--config".
    • Set non-command-line defaults in OptionParser.__init__(): _source & _destination.
    • Distributed relative_path_settings to components; updated OptionParser.populate_from_components() to combine it all.
    • Require list of keys in make_paths_absolute (was implicit in global relative_path_settings).
    • Added "--expose-internal-attribute", "--dump-pseudo-xml", "--dump-settings", and "--dump-transforms" hidden options.
    • Removed nasty internals-fiddling ConfigParser.get_section code, replaced with correct code.
    • Added validation functionality for config files.
    • Added "--error-encoding" option/setting, "_disable_config" internal setting.
    • Added encoding validation; updated "--input-encoding" and "--output-encoding"; added "--error-encoding-error-handler" and "--output-encoding-error-handler".
    • Moved config file processing from docutils/core.py.
    • Updated OptionParser.populate_from_components to handle new SettingsSpec.settings_defaults dict.
    • Added support for "-" => stdin/stdout.
    • Added "exit_level" setting ("--exit" option).
  • docutils/io.py:
    • Split IO classes into subclasses of Input and Output.
    • Added automatic closing to FileInput and FileOutput.
    • Delayed opening of FileOutput file until write() called.
    • FileOutput.write() now returns the encoded output string.
    • Try to get path/stream name automatically in FileInput & FileOutput.
    • Added defaults for source & destination paths.
    • Allow for Unicode I/O with an explicit "unicode" encoding.
    • Added Output.encode().
    • Removed dependency on runtime settings; pass encoding directly.
    • Recognize Unicode strings in Input.decode().
    • Added support for output encoding error handlers.
  • docutils/nodes.py:
    • Added "Invisible" element category class.
    • Changed Node.walk() & .walkabout() to permit more tree modification during a traversal.
    • Added element classes: line_block, generated, address, sidebar, rubric, attribution, admonition, superscript, subscript, inline
    • Added support for lists of nodes to Element.insert().
    • Fixed parent linking in Element.replace().
    • Added new abstract superclass FixedTextElement; adds "xml:space" attribute.
    • Added support for "line" attribute of system_message nodes.
    • Added support for the observer pattern from utils.Reporter. Added parse_messages and transform_messages attributes to document, removed messages. Added note_parse_message and note_transform_message methods.
    • Added support for improved diagnostics:
      • Added "document", "source", and "line" internal attributes to Node, set by Node.setup_child().
      • Converted variations on node.parent = self to self.setup_child(node).
      • Added document.current_source & .current_line attributes, and .note_source observer method.
      • Changed "system_message" output to GNU-Tools format.
    • Added a "rawsource" attribute to the Text class, for text before backslash-escape resolution.
    • Support for new transform system.
    • Reworked pending element.
    • Fixed XML DOM bug (SF #660611).
    • Removed the interpeted element class and added title_reference, abbreviation, acronym.
    • Made substitutions case-sensitive-but-forgiving; moved some code from the parser.
    • Fixed Unicode bug on element attributes (report: William Dode).
  • docutils/optik.py: Removed from project; replaced with extras/optparse.py and extras/textwrap.py. These will be installed only if they're not already present in the Python installation.
  • docutils/roman.py: Moved to extras/roman.py; this will be installed only if it's not already present in the Python installation.
  • docutils/statemachine.py:
    • Factored out State.add_initial_transitions() so it can be extended.
    • Converted whitespace-specific "blank" and "indent" transitions from special-case code to ordinary transitions: removed StateMachineWS.check_line() & .check_whitespace(), added StateWS.add_initial_transitions() method, ws_patterns & ws_initial_transitions attributes.
    • Removed State.match_transition() after merging it into .check_line().
    • Added StateCorrection exception.
    • Added support for StateCorrection in StateMachine.run() (moved TransitionCorrection support there too.)
    • Changed StateMachine.next_line() and .goto_line() to raise EOFError instead of IndexError.
    • Added State.no_match method.
    • Added support for the Observer pattern, triggered by input line changes.
    • Added strip_top parameter to StateMachineWS.get_first_known_indented.
    • Made context a parameter to StateMachine.run().
    • Added ViewList & StringList classes; extract_indented() becomes StringList.get_indented().
    • Added StateMachine.insert_input().
    • Fixed ViewList slice handling for Python 2.3. Patch from (and thanks to) Fred Drake.
  • docutils/utils.py:
    • Added a source attribute to Reporter instances and system_message elements.
    • Added an observer pattern to utils.Reporter to keep track of system messages.
    • Fixed bugs in relative_path().
    • Added support for improved diagnostics.
    • Moved normalize_name() to nodes.py (fully_normalize_name).
    • Added support for encoding Reporter stderr output, and encoding error handlers.
    • Reporter keeps track of the highest level system message yet generated.
  • docutils/languages: Fixed bibliographic field language lookups.
  • docutils/languages/es.py: Added to project; Spanish mappings by Marcelo Huerta San Martin.
  • docutils/languages/fr.py: Added to project; French mappings by Stefane Fermigier.
  • docutils/languages/it.py: Added to project; Italian mappings by Nicola Larosa.
  • docutils/languages/sk.py: Added to project; Slovak mappings by Miroslav Vasko.
  • docutils/parser/__init__.py:
    • Added Parser.finish_parse() method.
  • docutils/parser/rst/__init__.py:
    • Added options: "--pep-references", "--rfc-references", "--tab-width", "--trim-footnote-reference-space".
  • docutils/parsers/rst/states.py:
    • Changed "title under/overline too short" system messages from INFO to WARNING, and fixed its insertion location.
    • Fixed enumerated list item parsing to allow paragraphs & section titles to begin with enumerators.
    • Converted system messages to use the new "line" attribute.
    • Fixed a substitution reference edge case.
    • Added support for "--pep-references" and "--rfc-references" options; reworked Inliner code to make customization easier.
    • Removed field argument parsing.
    • Added support for short section title over/underlines.
    • Fixed "simple reference name" regexp to ignore text like "object.__method__"; not an anonymous reference.
    • Added support for improved diagnostics.
    • Reworked directive API, based on Dethe Elza's contribution. Added Body.parse_directive(), .parse_directive_options(), .parse_directive_arguments() methods.
    • Added ExtensionOptions class, to parse directive options without parsing field bodies. Factored Body.parse_field_body() out of Body.field(), overridden in ExtensionOptions.
    • Improved definition list term/classifier parsing.
    • Added warnings for unknown directives.
    • Renamed Stuff to Struct.
    • Now flagged as errors: transitions at the beginning or end of sections, empty sections (except title), and empty documents.
    • Updated for statemachine.StringList.
    • Enabled recognition of schemeless email addresses in targets.
    • Added support for embedded URIs in hyperlink references.
    • Added backslash-escapes to inline markup end-string suffix.
    • Added support for correct interpreted text processing.
    • Fixed nested title parsing (topic, sidebar directives).
    • Added special processing of backslash-escaped whitespace (idea from David Abrahams).
    • Made substitutions case-sensitive-but-forgiving; moved some code to docutils.nodes.
    • Added support for block quote attributions.
    • Added a kludge to work-around a conflict between the bubble-up parser strategy and short titles (<= 3 char-long over- & underlines). Fixes SF bug #738803 "infinite loop with multiple titles" submitted by Jason Diamond.
    • Added explicit interpreted text roles for standard inline markup: "emphasis", "strong", "literal".
    • Implemented "superscript" and "subscript" interpreted text roles.
    • Added initial support for "abbreviation" and "acronym" roles; incomplete.
    • Added support for "--trim-footnote-reference-space" option.
    • Optional space before colons in directives & hyperlink targets.
  • docutils/parsers/rst/tableparser.py:
    • Fixed a bug that was producing unwanted empty rows in "simple" tables.
    • Detect bad column spans in "simple" tables.
  • docutils/parsers/rst/directives: Updated all directive functions to new API.
  • docutils/parsers/rst/directives/__init__.py:
    • Added flag(), unchanged(), path(), nonnegative_int(), choice(), and class_option() directive option helper functions.
    • Added warnings for unknown directives.
    • Return None for missing directives.
    • Added register_directive(), thanks to William Dode and Paul Moore.
  • docutils/parsers/rst/directives/admonitions.py:
    • Added "admonition" directive.
  • docutils/parsers/rst/directives/body.py: Added to project. Contains the "topic", "sidebar" (from Patrick O'Brien), "line-block", "parsed-literal", "rubric", "epigraph", "highlights" and "pull-quote" directives.
  • docutils/parsers/rst/directives/images.py:
    • Added an "align" attribute to the "image" & "figure" directives (by Adam Chodorowski).
    • Added "class" option to "image", and "figclass" to "figure".
  • docutils/parsers/rst/directives/misc.py:
    • Added "include", "raw", and "replace" directives, courtesy of Dethe Elza.
    • Added "unicode" and "class" directives.
  • docutils/parsers/rst/directives/parts.py:
    • Added the "sectnum" directive; by Dmitry Jemerov.
    • Added "class" option to "contents" directive.
  • docutils/parsers/rst/directives/references.py: Added to project. Contains the "target-notes" directive.
  • docutils/parsers/rst/languages/__init__.py:
    • Return None from get_language() for missing language modules.
  • docutils/parsers/rst/languages/de.py: Added to project; German mappings by Engelbert Gruber.
  • docutils/parsers/rst/languages/en.py:
    • Added interpreted text roles mapping.
  • docutils/parsers/rst/languages/es.py: Added to project; Spanish mappings by Marcelo Huerta San Martin.
  • docutils/parsers/rst/languages/fr.py: Added to project; French mappings by William Dode.
  • docutils/parsers/rst/languages/it.py: Added to project; Italian mappings by Nicola Larosa.
  • docutils/parsers/rst/languages/sk.py: Added to project; Slovak mappings by Miroslav Vasko.
  • docutils/readers/__init__.py:
    • Added support for the observer pattern from utils.Reporter, in Reader.parse and Reader.transform.
    • Removed Reader.transform() method.
    • Added default parameter values to Reader.__init__() to make instantiation easier.
    • Removed bogus aliases: "restructuredtext" is not a Reader.
  • docutils/readers/pep.py:
    • Added the peps.TargetNotes transform to the Reader.
    • Removed PEP & RFC reference detection code; moved to parsers/rst/states.py as options (enabled here by default).
    • Added support for pre-acceptance PEPs (no PEP number yet).
    • Moved Inliner & made it a class attribute of Reader for easy subclassing.
  • docutils/readers/python: Python Source Reader subpackage added to project, including preliminary versions of:
    • __init__.py
    • moduleparser.py: Parser for Python modules.
  • docutils/transforms/__init__.py:
    • Added Transformer class and completed transform reform.
    • Added unknown_reference_resolvers list for each transformer. This list holds the list of functions provided by each component of the transformer that help resolve references.
  • docutils/transforms/frontmatter.py:
    • Improved support for generic fields.
    • Fixed bibliographic field language lookups.
  • docutils/transforms/misc.py: Added to project. Miscellaneous transforms.
  • docutils/transforms/parts.py:
    • Moved the "id" attribute from TOC list items to the references (Contents.build_contents()).
    • Added the SectNum transform; by Dmitry Jemerov.
    • Added "class" attribute support to Contents.
  • docutils/transforms/peps.py:
    • Added mask_email() function, updating to pep2html.py's functionality.
    • Linked "Content-Type: text/x-rst" to PEP 12.
    • Added the TargetNotes PEP-specific transform.
    • Added TargetNotes.cleanup_callback.
    • Added title check to Headers.
  • docutils/transforms/references.py:
    • Added the TargetNotes generic transform.
    • Split Hyperlinks into multiple transforms.
    • Fixed bug with multiply-indirect references (report: Bruce Smith).
    • Added check for circular indirect references.
    • Made substitutions case-sensitive-but-forgiving.
  • docutils/transforms/universal.py:
    • Added support for the "--expose-internal-attributes" option.
    • Removed Pending transform classes & data.
  • docutils/writers/__init__.py:
    • Removed Writer.transform() method.
  • docutils/writers/docutils-xml.py:
    • Added XML and doctype declarations.
    • Added "--no-doctype" and "--no-xml-declaration" options.
  • docutils/writers/html4css1.py:
    • "name" attributes only on these tags: a, applet, form, frame, iframe, img, map.
    • Added "name" attribute to <a> in section titles for Netscape 4 support (bug report: Pearu Peterson).
    • Fixed targets (names) on footnote, citation, topic title, problematic, and system_message nodes (for Netscape 4).
    • Changed field names from "<td>" to "<th>".
    • Added "@" to "&#64;" encoding to thwart address harvesters.
    • Improved the vertical whitespace optimization; ignore "invisible" nodes (targets, comments, etc.).
    • Improved inline literals with <span class="pre"> around chunks of text and &nbsp; for runs of spaces.
    • Improved modularity of output; added self.body_pre_docinfo and self.docinfo segments.
    • Added support for "line_block", "address" elements.
    • Improved backlinks (footnotes & system_messages).
    • Improved system_message output.
    • Redefined "--stylesheet" as containing an invariant URL, used verbatim. Added "--stylesheet-path", interpreted w.r.t. the working directory.
    • Added "--footnote-references" option (superscript or brackets).
    • Added "--compact-lists" and "--no-compact-lists" options.
    • Added "--embed-stylesheet" and "--link-stylesheet" options; factored out HTMLTranslator.get_stylesheet_reference().
    • Improved field list rendering.
    • Added Docutils version to "generator" meta tag.
    • Fixed a bug with images; they must be inline, so wrapped in <p>.
    • Improved layout of <pre> HTML source.
    • Fixed attribute typo on <colspec>.
    • Refined XML prologue.
    • Support for no stylesheet.
    • Removed "interpreted" element support.
    • Added support for "title_reference", "sidebar", "attribution", "rubric", and generic "admonition" elements.
    • Added "--attribution" option.
    • Added support for "inline", "subscript", "superscript" elements.
    • Added initial support for "abbreviation" and "acronym"; incomplete.
  • docutils/writers/latex2e.py: LaTeX Writer, added by Engelbert Gruber (from the sandbox).
    • Added french.
    • Double quotes in literal blocks (special treatment for de/ngerman).
    • Added '--hyperlink-color' option ('0' turns off coloring of links).
    • Added "--attribution" option.
    • Right align attributions.
  • docutils/writers/pep_html.py:
    • Parameterized output encoding in PEP template.
    • Reworked substitutions from locals() into subs dict.
    • Redefined "--pep-stylesheet" as containing an invariant URL, used verbatim. Added "--pep-stylesheet-path", interpreted w.r.t. the working directory.
    • Added an override on the "--footnote-references" option.
    • Factored out HTMLTranslator.get_stylesheet_reference().
    • Added Docutils version to "generator" meta tag.
    • Added a "DO NOT EDIT THIS FILE" comment to generated HTML.
  • docs/tools.txt:
    • Added a "silent" setting for buildhtml.py.
    • Added a "Getting Help" section.
    • Rearranged the structure.
    • Kept up to date, with new settings, command-line options etc.
    • Added section for rst2latex.py (Engelbert Gruber).
    • Converted settings table into a definition list.
  • docs/rst/quickstart.txt:
    • Added a table of contents.
    • Added feedback information.
    • Added mention of minimum section title underline lengths.
    • Removed the 4-character minimum for section title underlines.
  • docs/rst/quickref.html:
    • Added a "Getting Help" section.
    • Added a style to make section title backlinks more subtle.
    • Added mention of minimum section title underline lengths.
    • Removed the 4-character minimum for section title underlines.
  • extras: Directory added to project; contains third-party modules that Docutils depends on (optparse, textwrap, roman). These are only installed if they're not already present.
  • licenses: Directory added to project; contains copies of license files for non-public-domain files.
  • spec/doctree.txt:
    • Changed the focus. It's about DTD elements: structural relationships, semantics, and external (public) attributes. Not about the element class library.
    • Moved some implementation-specific stuff into docutils.nodes docstrings.
    • Wrote descriptions of all common attributes and parameter entities. Filled in introductory material.
    • Working through the element descriptions: 55 down, 37 to go.
    • Removed "Representation of Horizontal Rules" to spec/rst/alternatives.txt.
  • spec/docutils.dtd:
    • Added "generated" inline element.
    • Added "line_block" body element.
    • Added "auto" attribute to "title".
    • Changed content models of "literal_block" and "doctest_block" to %text.model.
    • Added %number; attribute type parameter entity.
    • Changed %structural.elements; to %section.elements.
    • Updated attribute types; made more specific.
    • Added "address" bibliographic element.
    • Added "line" attribute to system_message element.
    • Removed "field_argument" element; "field_name" may contain multiple words and whitespace.
    • Changed public identifier to docutils.sf.net.
    • Removed "interpreted" element; added "title_reference", "abbreviation", "acronym".
    • Removed "refuri" attribute from "footnote_reference" and "citation_reference".
    • Added "sidebar", "rubric", "attribution", "admonition", "superscript", "subscript", and "inline" elements.
  • spec/pep-0256.txt: Converted to reStructuredText & updated.
  • spec/pep-0257.txt: Converted to reStructuredText & updated.
  • spec/pep-0258.txt: Converted to reStructuredText & updated.
  • spec/semantics.txt: Updated with text from a Doc-SIG response to Dallas Mahrt.
  • spec/transforms.txt: Added to project.
  • spec/howto: Added subdirectory, for developer how-to docs.
  • spec/howto/rst-directives.txt: Added to project. Original by Dethe Elza, edited & extended by David Goodger.
  • spec/howto/i18n.txt: Docutils Internationalization. Added to project.
  • spec/rst/alternatives.txt:
    • Added "Doctree Representation of Transitions" from spec/doctree.txt.
    • Updated "Inline External Targets" & closed the debate.
    • Added ideas for interpreted text syntax extensions.
    • Added "Nested Inline Markup" section.
  • spec/rst/directives.txt:
    • Added directives: "topic", "sectnum", "target-notes", "line-block", "parsed-literal", "include", "replace", "sidebar", "admonition", "rubric", "epigraph", "highlights", "unicode" and "class".
    • Formalized descriptions of directive details.
    • Added an "align" attribute to the "image" & "figure" directives (by Adam Chodorowski).
    • Added "class" options to "topic", "sidebar", "line-block", "parsed-literal", "contents", and "image"; and "figclass" to "figure".
  • spec/rst/interpreted.txt: Added to project. Descriptions of interpreted text roles.
  • spec/rst/introduction.txt:
    • Added pointers to material for new users.
  • spec/rst/reStructuredText.txt:
    • Disambiguated comments (just add a newline after the "::").
    • Updated enumerated list description; added a discussion of the second-line validity checking.
    • Updated directive description.
    • Added a note redirecting newbies to the user docs.
    • Expanded description of inline markup start-strings in non-markup contexts.
    • Removed field arguments and made field lists a generic construct.
    • Removed the 4-character minimum for section title underlines.
    • Clarified term/classifier delimiter & inline markup ambiguity (definition lists).
    • Added "Embedded URIs".
    • Updated "Interpreted Text" section.
    • Added "Character-Level Inline Markup" section.
  • test: Continually adding & updating tests.
    • Moved test/test_rst/ to test/test_parsers/test_rst/.
    • Moved test/test_pep/ to test/test_readers/test_pep/.
    • Added test/test_readers/test_python/.
    • Added test/test_writers/ (Engelbert Gruber).
  • tools:
    • Made the locale.setlocale() calls in front ends fault-tolerant.
  • tools/buildhtml.py:
    • Added "--silent" option.
    • Fixed bug with absolute paths & "--config".
    • Updated for new I/O classes.
    • Added some exception handling.
    • Separated publishers' setting defaults; prevents interference.
    • Updated for new publish_file() convenience function.
  • tools/pep-html-template:
    • Allow for "--embed-stylesheet".
    • Added Docutils version to "generator" meta tag.
    • Added a "DO NOT EDIT THIS FILE" comment to generated HTML.
    • Conform to XHTML spec.
  • tools/pep2html.py:
    • Made argv a parameter to main().
    • Added support for "Content-Type:" header & arbitrary PEP formats.
    • Linked "Content-Type: text/plain" to PEP 9.
    • Files skipped (due to an error) are not pushed onto the server.
    • Updated for new I/O classes.
    • Added check_requirements() & pep_type_error().
    • Added some exception handling.
    • Updated for new publish_string() convenience function.
    • Added a "DO NOT EDIT THIS FILE" comment to generated HTML.
  • tools/quicktest.py:
    • Added "-V"/"--version" option.
  • tools/rst2latex.py: LaTeX front end, added by Engelbert Gruber.
  • tools/unicode2rstsubs.py: Added to project. Produces character entity files (reSructuredText substitutions) from the MathML master unicode.xml file.
  • tools/editors: Support code for editors, added to project. Contains emacs/restructuredtext.el.
  • tools/stylesheets/default.css: Moved into the stylesheets directory.
    • Added style for chunks of inline literals.
    • Removed margin for first child of table cells.
    • Right-aligned field list names.
    • Support for auto-numbered section titles in TOCs.
    • Increased the size of inline literals (<tt>) in titles.
    • Restored the light gray background for inline literals.
    • Added support for "line_block" elements.
    • Added style for "address" elements.
    • Removed "a.footnote-reference" style; doing it with <sup> now.
    • Improved field list rendering.
    • Vertical whitespace improvements.
    • Removed "a.target" style.
  • tools/stylesheets/pep.css:
    • Fixed nested section margins.
    • Other changes parallel those of ../default.css.

Release 0.2 (2002-07-31)

General:

  • The word "component" was being used ambiguously. From now on, "component" will be used to mean "Docutils component", as in Reader, Writer, Parser, or Transform. Portions of documents (Table of Contents, sections, etc.) will be called "document parts".
  • Did a grand renaming: a lot of verylongnames became very_long_names.
  • Cleaned up imports: no more relative package imports or comma-separated lists of top-level modules.
  • Added support for an option values object which carries default settings and overrides (from command-line options and library use).
  • Added internal Unicode support, and support for both input and output encodings.
  • Added support for the docutils.io.IO class & subclasses.

Specific:

  • docutils/__init__.py:
    • Added ApplicationError and DataError, for use throughout the package.
    • Added Component base class for Docutils components; implements the supports method.
    • Added __version__ (thus, docutils.__version__).
  • docutils/core.py:
    • Removed many keyword parameters to Publisher.__init__() and publish(); bundled into an option values object. Added "argv", "usage", "description", and "option_spec" parameters for command-line support.
    • Added Publisher.process_command_line() and .set_options() methods.
    • Reworked I/O model for docutils.io wrappers.
    • Updated Publisher.set_options(); now returns option values object.
    • Added support for configuration files (/etc/docutils.conf, ./docutils.conf, ~/.docutils).
    • Added Publisher.setup_option_parser().
    • Added default usage message and description.
  • docutils/frontend.py: Added to project; support for front-end (command-line) scripts. Option specifications may be augmented by components. Requires Optik (http://optik.sf.net/) for option processing (installed locally as docutils/optik.py).
  • docutils/io.py: Added to project; uniform API for a variety of input output mechanisms.
  • docutils/nodes.py:
    • Added TreeCopyVisitor class.
    • Added a copy method to Node and subclasses.
    • Added a SkipDeparture exception for visitors.
    • Renamed TreePruningException from VisitorException.
    • Added docstrings to TreePruningException, subclasses, and Nodes.walk().
    • Improved docstrings.
    • Added SparseNodeVisitor, refined NodeVisitor.
    • Moved utils.id() to nodes.make_id() to avoid circular imports.
    • Added decoration, header, and footer node classes, and PreDecorative mixin.
    • Reworked the name/id bookkeeping; to document, removed explicit_targets and implicit_targets attributes, added nametypes attribute and set_name_id_map method.
    • Added NodeFound exception, for use with NodeVisitor traversals.
    • Added document.has_name() method.
    • Fixed DOM generation for list-attributes.
    • Added category class Labeled (used by footnotes & citations).
    • Added Element.set_class() method (sets "class" attribute).
  • docutils/optik.py: Added to project. Combined from the Optik package, with added option groups and other modifications. The use of this module is probably only temporary.
  • docutils/statemachine.py:
    • Added runtime_init method to StateMachine and State.
    • Added underscores to improve many awkward names.
    • In string2lines(), changed whitespace normalizing translation table to regexp; restores Python 2.0 compatibility with Unicode.
  • docutils/urischemes.py:
    • Filled in some descriptions.
    • Added "shttp" scheme.
  • docutils/utils.py:
    • Added clean_rcs_keywords function (moved from docutils/transforms/frontmatter.py DocInfo.filter_rcs_keywords).
    • Added underscores to improve many awkward names.
    • Changed names of Reporter's thresholds: warning_level -> report_level; error_level -> halt_level.
    • Moved utils.id() to nodes.make_id().
    • Added relative_path(source, target).
  • docutils/languages/de.py: German mappings; added to project. Thanks to Gunnar Schwant for the translations.
  • docutils/languages/en.py: Added "Dedication" bibliographic field mappings.
  • docutils/languages/sv.py: Swedish mappings; added to project by Adam Chodorowski.
  • docutils/parsers/rst/states.py:
    • Added underscores to improve many awkward names.
    • Added RFC-2822 header support.
    • Extracted the inline parsing code from RSTState to a separate class, Inliner, which will allow easy subclassing.
    • Made local bindings for memo container & often-used contents (reduces code complexity a lot). See RSTState.runtime_init().
    • RSTState.parent replaces RSTState.statemachine.node.
    • Added MarkupMismatch exception; for late corrections.
    • Added -/: characters to inline markup's start string prefix, / to end string suffix.
    • Fixed a footnote bug.
    • Fixed a bug with literal blocks.
    • Applied patch from Simon Budig: simplified regexps with symbolic names, removed Inliner.groups and Body.explicit.groups.
    • Converted regexps from '%s' % var to '%(var)s' % locals().
    • Fixed a bug in Inliner.interpreted_or_phrase_ref().
    • Allowed non-ASCII in "simple names" (directive names, field names, references, etc.).
    • Converted Inliner.patterns.initial to be dynamically built from parts with build_regexp() function.
    • Changed Inliner.inline_target to .inline_internal_target.
    • Updated docstrings.
    • Changed "table" to "grid_table"; added "simple_table" support.
  • docutils/parsers/rst/tableparser.py:
    • Changed TableParser to GridTableParser.
    • Added SimpleTableParser.
    • Refactored naming.
  • docutils/parsers/rst/directives/__init__.py: Added "en" (English) as a fallback language for directive names.
  • docutils/parsers/rst/directives/html.py: Changed the meta directive to use a pending element, used only by HTML writers.
  • docutils/parsers/rst/directives/parts.py: Renamed from components.py.
    • Added "backlinks" attribute to "contents" directive.
  • docutils/parsers/rst/languages/sv.py: Swedish mappings; added to project by Adam Chodorowski.
  • docutils/readers/__init__.py: Gave Readers more control over choosing and instantiating Parsers.
  • docutils/readers/pep.py: Added to project; for PEP processing.
  • docutils/transforms/__init__.py: Transform.__init__() now requires a component parameter.
  • docutils/transforms/components.py: Added to project; transforms related to Docutils components.
  • docutils/transforms/frontmatter.py:
    • In DocInfo.extract_authors, check for a single "author" in an "authors" group, and convert it to a single "author" element.
    • Added support for "Dedication" and generic bibliographic fields.
  • docutils/transforms/peps.py: Added to project; PEP-specific.
  • docutils/transforms/parts.py: Renamed from old components.py.
    • Added filter for Contents, to use alt-text for inline images, and to remove inline markup that doesn't make sense in the ToC.
    • Added "name" attribute to TOC topic depending on its title.
    • Added support for optional TOC backlinks.
  • docutils/transforms/references.py: Fixed indirect target resolution in Hyperlinks transform.
  • docutils/transforms/universal.py:
    • Changed Messages transform to properly filter out system messages below the warning threshold.
    • Added Decorations transform (support for --generator, --date, --time, --source-link options).
  • docutils/writers/__init__.py: Added "pdf" alias in anticipation of Engelbert Gruber's PDF writer.
  • docutils/writers/html4css1.py:
    • Made XHTML-compatible (switched to lowercase element & attribute names; empty tag format).
    • Escape double-dashes in comment text.
    • Improved boilerplate & modularity of output.
    • Exposed modular output in Writer class.
    • Added a "generator" meta tag to <head>.
    • Added support for the --stylesheet option.
    • Added support for decoration, header, and footer elements.
    • In HTMLTranslator.attval(), changed whitespace normalizing translation table to regexp; restores Python 2.0 compatibility with Unicode.
    • Added the translator class as instance variable to the Writer, to make it easily subclassable.
    • Improved option list spacing (thanks to Richard Jones).
    • Modified field list output.
    • Added backlinks to footnotes & citations.
    • Added percentage widths to "<col>" tags (from colspec).
    • Option lists: "<code>" changed to "<kbd>", option_argument "<span>" changed to "<var>".
    • Inline literals: "<code>" changed to "<tt>".
    • Many changes to optimize vertical space: compact simple lists etc.
    • Add a command-line options & directive attributes to control TOC and footnote/citation backlinks.
    • Added support for optional footnote/citation backlinks.
    • Added support for generic bibliographic fields.
    • Identify backrefs.
    • Relative URLs for stylesheet links.
  • docutils/writers/pep_html.py: Added to project; HTML Writer for PEPs (subclass of html4css1.Writer).
  • docutils/writers/pseudoxml.py: Renamed from pprint.py.
  • docutils/writers/docutils_xml.py: Added to project; trivial writer of the Docutils internal doctree in XML.
  • docs/tools.txt: "Docutils Front-End Tools", added to project.
  • spec/doctree.txt:
    • Changed the title to "The Docutils Document Tree".
    • Added "Hyperlink Bookkeeping" section.
  • spec/docutils.dtd:
    • Added decoration, header, and footer elements.
    • Brought interpreted element in line with the parser: changed attribute "type" to "role", added "position".
    • Added support for generic bibliographic fields.
  • spec/notes.txt: Continual updates. Added "Project Policies".
  • spec/pep-0256.txt: Updated. Added "Roadmap to the Doctring PEPs" section.
  • spec/pep-0257.txt: Clarified prohibition of signature repetition.
  • spec/pep-0258.txt: Updated. Added text from pysource.txt and mailing list discussions.
  • spec/pep-0287.txt:
    • Renamed to "reStructuredText Docstring Format".
    • Minor edits.
    • Reworked Q&A as an enumerated list.
    • Converted to reStructuredText format.
  • spec/pysource.dtd:
    • Reworked structural elements, incorporating ideas from Tony Ibbs.
  • spec/pysource.txt: Removed from project. Moved much of its contents to pep-0258.txt.
  • spec/rst/alternatives.txt:
    • Expanded auto-enumerated list idea; thanks to Fred Bremmer.
    • Added "Inline External Targets" section.
  • spec/rst/directives.txt:
    • Added "backlinks" attribute to "contents" directive.
  • spec/rst/problems.txt:
    • Updated the Enumerated List Markup discussion.
    • Added new alternative table markup syntaxes.
  • spec/rst/reStructuredText.txt:
    • Clarified field list usage.
    • Updated enumerated list description.
    • Clarified purpose of directives.
    • Added -/: characters to inline markup's start string prefix, / to end string suffix.
    • Updated "Authors" bibliographic field behavior.
    • Changed "inline hyperlink targets" to "inline internal targets".
    • Added "simple table" syntax to supplement the existing but newly-renamed "grid tables".
    • Added cautions for anonymous hyperlink use.
    • Added "Dedication" and generic bibliographic fields.
  • test: Made test modules standalone (subdirectories became packages).
  • test/DocutilsTestSupport.py:
    • Added support for PEP extensions to reStructuredText.
    • Added support for simple tables.
    • Refactored naming.
  • test/package_unittest.py: Renamed from UnitTestFolder.py.
    • Now supports true packages containing test modules (__init__.py files required); fixes duplicate module name bug.
  • test/test_pep/: Subpackage added to project; PEP testing.
  • test/test_rst/test_SimpleTableParser.py: Added to project.
  • tools:
    • Updated html.py and publish.py front-end tools to use the new command-line processing facilities of docutils.frontend (exposed in docutils.core.Publisher), reducing each to just a few lines of code.
    • Added locale.setlocale() calls to front-end tools.
  • tools/buildhtml.py: Added to project; batch-generates .html from all the .txt files in directories and subdirectories.
  • tools/default.css:
    • Added support for header and footer elements.
    • Added styles for "Dedication" topics (biblio fields).
  • tools/docutils.conf: A configuration file; added to project.
  • tools/docutils-xml.py: Added to project.
  • tools/pep.py: Added to project; PEP to HTML front-end tool.
  • tools/pep-html-template: Added to project.
  • tools/pep2html.py: Added to project from Python (nondist/peps). Added support for Docutils (reStructuredText PEPs).
  • tools/quicktest.py:
    • Added the --attributes option, hacked a bit.
    • Added a second command-line argument (output file); cleaned up.
  • tools/stylesheets/: Subdirectory added to project.
  • tools/stylesheets/pep.css: Added to project; stylesheet for PEPs.

Release 0.1 (2002-04-20)

This is the first release of Docutils, merged from the now inactive reStructuredText and Docstring Processing System projects. For the pre-Docutils history, see the reStructuredText HISTORY and the DPS HISTORY files.

General changes: renamed 'dps' package to 'docutils'; renamed 'restructuredtext' subpackage to 'rst'; merged the codebases; merged the test suites (reStructuredText's test/test_states renamed to test/test_rst); and all modifications required to make it all work.

  • docutils/parsers/rst/states.py:
    • Improved diagnostic system messages for missing blank lines.
    • Fixed substitution_reference bug.
View document source. Generated on: 2004-07-01 22:43 UTC. Generated by Docutils from reStructuredText source.


README: Docutils

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Date: | 2004-06-10
Web site: | http://docutils.sourceforge.net/
Copyright: | This document has been placed in the public domain.

Contents

  • Quick-Start
  • Purpose
  • Releases & Snapshots
  • Requirements
  • Project Files & Directories
  • Installation
    • GNU/Linux, BSDs, Unix, Mac OS X, etc.
    • Windows
    • Mac OS 8/9
  • Usage
  • Running the Test Suite
  • Getting Help

Thank you for downloading the Python Docutils project archive. As this is a work in progress, please check the project website for updated working files (snapshots). This project should be considered highly experimental; APIs are subject to change at any time.

Quick-Start

This is for those who want to get up & running quickly. Read on for complete details.

  1. Get and install the latest release of Python, available from

    http://www.python.org/

    Python 2.2 or later 1 is required; Python 2.2.2 or later is recommended.

  2. Use the latest Docutils code. Get the code from CVS or from the snapshot:

    http://docutils.sf.net/docutils-snapshot.tgz

    See Releases & Snapshots below for details.

  3. Unpack the tarball in a temporary directory (not directly in Python's site-packages) and install with the standard

    python setup.py install

    See Installation below for details.

  4. Use a front-end tool from the "tools" subdirectory of the same directory as in step 3. For example:

    cd tools
./rst2html.py test.txt test.html

    See Usage below for details.

Purpose

The purpose of the Docutils project is to create a set of tools for processing plaintext documentation into useful formats, such as HTML, XML, and TeX. Support for the following sources has been implemented:

  • Standalone files.
  • PEPs (Python Enhancement Proposals).

Support for the following sources is planned:

  • Inline documentation from Python modules and packages, extracted with namespace context. This is the focus of the current development effort.
  • Email (RFC-822 headers, quoted excerpts, signatures, MIME parts).
  • Wikis, with global reference lookups of "wiki links".
  • Compound documents, such as multiple chapter files merged into a book.
  • And others as discovered.

Releases & Snapshots

Putting together an official "Release" of Docutils is a significant effort, so it isn't done that often. In the meantime, the CVS snapshots always contain the latest code and documentation, usually updated within an hour of changes being committed to the repository, and usually bug-free:

  • Snapshot of Docutils code, documentation, front-end tools, and tests: http://docutils.sf.net/docutils-snapshot.tgz
  • Snapshot of the Sandbox (experimental, contributed code): http://docutils.sf.net/docutils-sandbox-snapshot.tgz
  • Snapshot of web files (the files that generate the web site): http://docutils.sf.net/docutils-web-snapshot.tgz

To keep up to date on the latest developments, download fresh copies of the snapshots regularly. New functionality is being added weekly, sometimes daily. (There's also the CVS repository, and a mailing list for CVS messages. See the web site [address above] or docs/dev/policies.txt for details.)

Requirements

To run the code, Python 2.2 or later 1 must already be installed. The latest release is recommended. Python is available from http://www.python.org/.

The Python Imaging Library, or PIL, is used for some image manipulation operations if it is installed.

Docutils uses Greg Ward's Optik/optparse option processing package. It is included in the Docutils distribution. Python 2.3 and later come with optparse in the standard library; in this case, the Docutils copy is not installed.

[1] (1, 2) Python 2.1 may be used providing the compiler package is installed. The compiler package can be found in the Tools/ directory of Python 2.1's source distribution.

Project Files & Directories

  • README.txt: You're reading it.
  • COPYING.txt: Public Domain Dedication and copyright details for non-public-domain files (most are PD).
  • FAQ.txt: Docutils Frequently Asked Questions.
  • HISTORY.txt: Release notes for the current and previous project releases.
  • setup.py: Installation script. See "Installation" below.
  • install.py: Quick & dirty installation script. Just run it.
  • docutils: The project source directory, installed as a Python package.
  • extras: Directory for third-party modules that Docutils depends on. These are only installed if they're not already present.
  • docs: The project documentation directory. Read docs/index.txt for an overview, which is especially interesting for developers.
  • docs/user: The project user documentation directory. Contains the following documents, among others:
    • docs/user/tools.txt: Docutils Front-End Tools
    • docs/user/latex.txt: Docutils LaTeX Writer
    • docs/user/rst/quickstart.txt: A ReStructuredText Primer
    • docs/user/rst/quickref.html: Quick reStructuredText (HTML only)
  • docs/ref: The project reference directory. docs/ref/rst/restructuredtext.txt is the reStructuredText reference.
  • licenses: Directory containing copies of license files for non-public-domain files.
  • tools: Directory for Docutils front-end tools. See docs/user/tools.txt for documentation.
  • test: Unit tests. Not required to use the software, but very useful if you're planning to modify it. See Running the Test Suite below.

Installation

The first step is to expand the .tar.gz or .tgz archive in a temporary directory (not directly in Python's site-packages). It contains a distutils setup file "setup.py". OS-specific installation instructions follow.

GNU/Linux, BSDs, Unix, Mac OS X, etc.

  1. Open a shell.

  2. Go to the directory created by expanding the archive:

    cd <archive_directory_path>

  3. Install the package:

    python setup.py install

    If the python executable isn't on your path, you'll have to specify the complete path, such as /usr/local/bin/python. You may need root permissions to complete this step.

You can also just run install.py; it does the same thing.

Windows

  1. Open a DOS box (Command Shell, MSDOS Prompt, or whatever they're calling it these days).

  2. Go to the directory created by expanding the archive:

    cd <archive_directory_path>

  3. Install the package:

    <path_to_python.exe>\python setup.py install

If your system is set up to run Python when you double-click on .py files, you can run install.py to do the same as the above.

Mac OS 8/9

  1. Open the folder containing the expanded archive.

  2. Double-click on the file "setup.py", which should be a "Python module" file.

    If the file isn't a "Python module", the line endings are probably also wrong, and you will need to set up your system to recognize ".py" file extensions as Python files. See http://gotools.sourceforge.net/mac/python.html for detailed instructions. Once set up, it's easiest to start over by expanding the archive again.

  3. The distutils options window will appear. From the "Command" popup list choose "install", click "Add", then click "OK".

If install.py is a "Python module" (see step 2 above if it isn't), you can run it (double-click) instead of the above. The distutils options window will not appear.

Usage

After unpacking and installing the Docutils package, the following shell commands will generate HTML for all included documentation:

cd <archive_directory_path>/tools
./buildhtml.py ../

The final directory name of the <archive_directory_path> is "docutils" for snapshots. For official releases, the directory may be called "docutils-X.Y", where "X.Y" is the release version. Alternatively:

cd <archive_directory_path>
tools/buildhtml.py --config=tools/docutils.conf

Some files may generate system messages (warnings and errors). The tools/test.txt file (under the archive directory) contains 5 intentional errors. (They test the error reporting mechanism!)

There are many front-end tools in the unpacked "tools" subdirectory. You may want to begin with the "rst2html.py" front-end tool. Most tools take up to two arguments, the source path and destination path, with STDIN and STDOUT being the defaults. Use the "--help" option to the front-end tools for details on options and arguments. See Docutils Front-End Tools (docs/user/tools.txt) for full documentation.

The package modules are continually growing and evolving. The docutils.statemachine module is usable independently. It contains extensive inline documentation (in reStructuredText format of course).

Contributions are welcome!

Running the Test Suite

To run the entire test suite, after installation open a shell and use the following commands:

cd <archive_directory_path>/test
./alltests.py

You should see a long line of periods, one for each test, and then a summary like this:

Ran 518 tests in 24.653s

OK
Elapsed time: 26.189 seconds

The number of tests will grow over time, and the times reported will depend on the computer running the tests. The difference between the two times represents the time required to set up the tests (import modules, create data structures, etc.).

If any of the tests fail, please open a bug report or send email 2. Please include all relevant output, information about your operating system, Python version, and Docutils version. To see the Docutils version, use these commands:

cd ../tools
./quicktest.py --version

Getting Help

If you have questions or need assistance with Docutils or reStructuredText, please post a message to the Docutils-Users mailing list 2.

[2] (1, 2) Due to overwhelming amounts of spam, the docutils-users@lists.sourceforge.net mailing list has been set up for subscriber posting only. Non-subscribers who post to docutils-users will receive a message with "Subject: Your message to Docutils-users awaits moderator approval". Legitimate messages are accepted and posted as soon as possible (a list administrator must verify the message manually). If you'd like to subscribe to docutils-users, please visit <http://lists.sourceforge.net/lists/listinfo/docutils-users>.
View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Acknowledgements

Author: | David Goodger
Contact: | goodger@python.org
Date: | 2004-06-10
Revision: | 1.2
Copyright: | This document has been placed in the public domain.

I would like to acknowledge the people who have made a direct impact on the Docutils project, knowingly or not, in terms of encouragement, suggestions, criticism, bug reports, code contributions, cash donations, tasty treats, and related projects:

  • Aahz
  • David Abrahams
  • David Ascher
  • Heiko Baumann
  • Eric Bellot
  • Ian Bicking
  • Marek Blaha
  • Martin Blais
  • Stephen Boulet
  • Fred Bremmer
  • Simon Budig
  • Bill Bumgarner
  • Brett Cannon
  • Greg Chapman
  • Nicolas Chauveau
  • Beni Cherniavsky
  • Adam Chodorowski
  • Brent Cook
  • Laura Creighton
  • Artur de Sousa Rocha
  • Stephan Deibel & Wing IDE
  • Jason Diamond
  • William Dode
  • Fred Drake
  • Reggie Dugard
  • Dethe Elza
  • Marcus Ertl
  • Benja Fallenstein
  • fantasai
  • Stefane Fermigier
  • Jim Fulton
  • Peter Funk
  • Lele Gaifax
  • Dinu C. Gherman
  • Matt Gilbert
  • Jorge Gonzalez
  • Engelbert Gruber
  • Jacob Hallen
  • Simon Hefti
  • Doug Hellmann
  • Marc Herbert
  • Juergen Hermann
  • Jannie Hofmeyr
  • Steve Holden
  • Michael Hudson
  • Marcelo Huerta San Martin
  • Ludger Humbert
  • Jeremy Hylton
  • Tony Ibbs
  • Alan Jaffray
  • Dmitry Jemerov
  • Richard Jones
  • Andreas Jung
  • Garth Kidd
  • Axel Kollmorgen
  • Jeff Kowalczyk
  • Dave Kuhlman
  • Lloyd Kvam
  • Kirill Lapshin
  • Nicola Larosa
  • Daniel Larsson
  • Marc-Andre Lemburg
  • Julien Letessier
  • Wolfgang Lipp
  • Edward Loper
  • Dallas Mahrt
  • Ken Manheimer
  • Bob Marshall
  • Mark McEahern
  • Vincent McIntyre
  • John F Meinel Jr
  • Vasko Miroslav
  • Skip Montanaro
  • Paul Moore
  • Nigel W. Moriarty
  • Mark Nodine
  • Patrick K. O'Brien
  • Michel Pelletier
  • Sam Penrose
  • Tim Peters
  • Pearu Peterson
  • Mark Pilgrim
  • Brett g Porter
  • David Priest
  • Jens Quade
  • Andy Robinson
  • Tavis Rudd
  • Tracy Ruggles
  • Oliver Rutherfurd
  • Kenichi Sato
  • Ueli Schlaepfer
  • Gunnar Schwant
  • Bill Sconce
  • Frank Siebenlist
  • Bruce Smith
  • Darek Suchojad
  • Roman Suzi
  • tav
  • Kent Tenney
  • Bob Tolbert
  • Paul Tremblay
  • Laurence Tratt
  • Adrian van den Dries
  • Guido van Rossum
  • Paul Viren
  • Martin von Loewis
  • Greg Ward
  • Barry Warsaw
  • Edward Welbourne
  • Felix Wiemann
  • Ka-Ping Yee
  • Moshe Zadka

Thank you!

Special thanks to SourceForge and the Python Software Foundation.

Hopefully I haven't forgotten anyone or misspelled any names; apologies (and please let me know!) if I have.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils Configuration Files

Author: | David Goodger
Contact: | goodger@python.org
Revision: | 1.19
Date: | 2004-06-16
Copyright: | This document has been placed in the public domain.

Contents

  • Configuration File Syntax
  • Configuration File Sections & Entries
    • [general]
      • [parsers]
        • [restructuredtext parser]
      • [readers]
        • [standalone reader]
        • [pep reader]
        • [python reader]
      • [writers]
        • [docutils_xml writer]
        • [html4css1 writer]
          • [pep_html writer]
        • [latex2e writer]
        • [pseudoxml writer]
      • [applications]
        • [buildhtml application]
        • [docfactory application]
    • Other Settings
  • Old-Format Configuration Files

Configuration files are used for persistent customization; they can be set once and take effect every time you use a front-end tool. Configuration file settings override the built-in defaults, and command-line options override all.

By default, Docutils checks the following places for configuration files, in the following order:

  1. /etc/docutils.conf: This is a system-wide configuration file, applicable to all Docutils processing on the system.
  2. ./docutils.conf: This is a project-specific configuration file, located in the current directory. The Docutils front end has to be executed from the directory containing this configuration file for it to take effect (note that this may have nothing to do with the location of the source files). Settings in the project-specific configuration file will override corresponding settings in the system-wide file.
  3. ~/.docutils: This is a user-specific configuration file, located in the user's home directory. Settings in this file will override corresponding settings in both the system-wide and project-specific configuration files.

If more than one configuration file is found, all will be read but later entries will override earlier ones. For example, a "stylesheet" entry in a user-specific configuration file will override a "stylesheet" entry in the system-wide file.

The default implicit config file paths can be overridden by the DOCUTILSCONFIG environment variable. DOCUTILSCONFIG should contain a colon-separated (semicolon-separated on Windows) sequence of config file paths to search for; leave it empty to disable implicit config files altogether. Tilde-expansion is performed on paths. Paths are interpreted relative to the current working directory. Empty path items are ignored.

In addition, a configuration file may be explicitly specified with the "--config" command-line option. This configuration file is read after the three implicit ones listed above (or the ones defined by the DOCUTILSCONFIG environment variable), and its entries will have priority.

Configuration File Syntax

Configuration files use the standard ConfigParser.py Python module. From its documentation:

The configuration file consists of sections, lead by a "[section]" header and followed by "name: value" entries, with continuations in the style of RFC 822; "name=value" is also accepted. Note that leading whitespace is removed from values. ... Lines beginning with "#" or ";" are ignored and may be used to provide comments.

Note

No format string interpolation is done.

Configuration file entry names correspond to internal runtime settings. Underscores ("_") and hyphens ("-") can be used interchangably in entry names; hyphens are automatically converted to underscores.

For on/off switch settings (booleans), the following values are recognized:

  • On: "true", "yes", "on", "1"
  • Off: "false", "no", "off", "0", "" (no value)

Configuration File Sections & Entries

Below are the Docutils runtime settings, listed by config file section. Any setting may be specified in any section, but only settings from active sections will be used. Sections correspond to Docutils components (module name or alias; section names are always in lowercase letters). Each Docutils application uses a specific set of components; corresponding configuration file sections are applied when the application is used. Configuration sections are applied in general-to-specific order, as follows:

  1. [general]

  2. [parsers], parser dependencies, and the section specific to the Parser used ("[... parser]"). Currently, only [restructuredtext parser] is applicable.

  3. [readers], reader dependencies, and the section specific to the Reader used ("[... reader]"). For example, [pep reader] depends on [standalone reader].

  4. [writers], writer dependencies, and the section specific to the Writer used ("[... writer]"). For example, [pep_html writer] depends on [html4css1 writer].

  5. [applications], application dependencies, and the section

    specific to the Application (front-end tool) in use ("[... application]").

Since any setting may be specified in any section, this ordering allows component- or application-specific overrides of earlier settings. For example, there may be Reader-specific overrides of general settings; Writer-specific overrides of Parser settings; Application-specific overrides of Writer settings; and so on.

If multiple configuration files are applicable, the process is completed (all sections are applied in the order given) for each one before going on to the next. For example, a "[pep_html writer] stylesheet" setting in an earlier configuration file would be overridden by an "[html4css1 writer] stylesheet" setting in a later file.

Some knowledge of Python is assumed for some attributes.

[general]

Settings in the "[general]" section are always applied.

datestamp

Include a time/datestamp in the document footer. Contains a format string for Python's time.strftime. See the time module documentation.

Default: None. Options: --date, -d, --time, -t, --no-datestamp.

Configuration file entry examples:

# Equivalent to --date command-line option, results in
# ISO 8601 extended format datestamp, e.g. "2001-12-21":
datestamp: %Y-%m-%d

# Equivalent to --time command-line option, results in
# date/timestamp like "2001-12-21 18:43 UTC":
datestamp: %Y-%m-%d %H:%M UTC

# Disables datestamp; equivalent to --no-datestamp:
datestamp:
debug

Report debug-level system messages.

Default: don't (None). Options: --debug, --no-debug.

dump_internals

At the end of processing, write all internal attributes of the document (document.__dict__) to stderr.

Default: don't (None). Options: --dump-internals (hidden, for development use only).

dump_pseudo_xml

At the end of processing, write the pseudo-XML representation of the document to stderr.

Default: don't (None). Options: --dump-pseudo-xml (hidden, for development use only).

dump_settings

At the end of processing, write all Docutils settings to stderr.

Default: don't (None). Options: --dump-settings (hidden, for development use only).

dump_transforms

At the end of processing, write a list of all transforms applied to the document to stderr.

Default: don't (None). Options: --dump-transforms (hidden, for development use only).

error_encoding

The text encoding for error output.

Default: "ascii". Options: --error-encoding, -e.

error_encoding_error_handler

The error handler for unencodable characters in error output. See output_encoding_error_handler for acceptable values.

Default: "backslashreplace" for Python 2.3 and later; "replace" otherwise. Options: --error-encoding-error-handler, --error-encoding, -e.

exit_status_level

A system message level threshold; non-halting system messages at or above this level will produce a non-zero exit status at normal exit. Exit status is the maximum system message level plus 10 (11 for INFO, etc.).

Default: disabled (5). Options: --exit-status.

expose_internals

List of internal attribues to expose as external attributes (with "internal:" namespace prefix). To specify multiple attributes in configuration files, use colons to separate names; on the command line, the option may be used more than once.

Default: don't (None). Options: --expose-internal-attribute (hidden, for development use only).

footnote_backlinks

Enable or disable backlinks from footnotes and citations to their references.

Default: enabled (1). Options: --footnote-backlinks, --no-footnote-backlinks.

generator

Include a "Generated by Docutils" credit and link in the document footer.

Default: off (None). Options: --generator, -g, --no-generator.

halt_level

The threshold at or above which system messages are converted to exceptions, halting execution immediately. If traceback is set, the exception will propagate; otherwise, Docutils will exit.

Default: severe (4). Options: --halt, --strict.

input_encoding

The text encoding for input.

Default: auto-detect (None). Options: --input-encoding, -i.

language_code

ISO 639 2-letter language code (3-letter codes used only if no 2-letter code exists).

Default: English ("en"). Options: --language, -l.

output_encoding

The text encoding for output.

Default: "UTF-8". Options: --output-encoding, -o.

output_encoding_error_handler

The error handler for unencodable characters in the output. Acceptable values include:

strict
Raise an exception in case of an encoding error.
replace
Replace malformed data with a suitable replacement marker, such as "?".
ignore
Ignore malformed data and continue without further notice.
xmlcharrefreplace
(Python 2.3+) Replace with the appropriate XML character reference, such as "&#8224;".
backslashreplace
(Python 2.3+) Replace with backslashed escape sequences, such as "\u2020".

Acceptable values are the same as for the "error" parameter of Python's encode string method; other values may be defined in applications or in future versions of Python.

Default: "strict". Options: --output-encoding-error-handler, --output-encoding, -o.

report_level

Verbosity threshold at or above which system messages are reported.

Default: warning (2). Options: --report, -r, --verbose, -v, --quiet, -q.

sectnum_xform

Enable or disable the section numbering transform (docutils.transforms.parts.SectNum).

Default: enabled (1). Options: --no-section-numbering.

source_link

Include a "View document source" link in the document footer. URL will be relative to the destination.

Default: don't (None). Options: --source-link, -s, --no-source-link.

source_url

An explicit URL for a "View document source" link, used verbatim.

Default: compute if source_link (None). Options: --source-url, --no-source-link.

toc_backlinks

Enable backlinks from section titles to table of contents entries ("entry"), to the top of the TOC ("top"), or disable ("none").

Default: "entry". Options: --toc-entry-backlinks, --toc-top-backlinks, --no-toc-backlinks.

traceback

Enable Python tracebacks when halt-level system messages and other exceptions occur. Useful for debugging, and essential for issue reports. Exceptions are allowed to propagate, instead of being caught and reported (in a user-friendly way) by Docutils.

Default: disabled (None). Options: --traceback, --no-traceback.

warning_stream

Path to a file for the output of system messages (warnings) 1.

Default: stderr (None). Options: --warnings.

[parsers]

Docutils currently supports only one parser, for reStructuredText.

[restructuredtext parser]

pep_references

Recognize and link to standalone PEP references (like "PEP 258").

Default: disabled (None); enabled (1) in PEP Reader. Options: --pep-references.

pep_base_url

Base URL for PEP references.

Default: "http://www.python.org/peps/". Option: --pep-base-url.

rfc_references

Recognize and link to standalone RFC references (like "RFC 822").

Default: disabled (None); enabled (1) in PEP Reader. Options: --rfc-references.

rfc_base_url

Base URL for RFC references.

Default: "http://www.faqs.org/rfcs/". Option: --rfc-base-url.

tab_width

Number of spaces for hard tab expansion.

Default: 8. Options: --tab-width.

trim_footnote_reference_space

Remove spaces before footnote references.

Default: don't (None). Options: --trim-footnote-reference-space.

[readers]

[standalone reader]

docinfo_xform

Enable or disable the bibliographic field list transform (docutils.transforms.frontmatter.DocInfo).

Default: enabled (1). Options: --no-doc-info.

doctitle_xform

Enable or disable the promotion of a lone top-level section title to document title (and subsequent section title to document subtitle promotion; docutils.transforms.frontmatter.DocTitle).

Default: enabled (1). Options: --no-doc-title.

[pep reader]

The pep_references and rfc_references options ([restructuredtext parser]) are set on by default.

[python reader]

Under construction.

[writers]

[docutils_xml writer]

doctype_declaration

Generate XML with a DOCTYPE declaration.

Default: do (1). Options: --no-doctype.

indents

Generate XML with indents and newlines.

Default: don't (None). Options: --indents.

newlines

Generate XML with newlines before and after tags.

Default: don't (None). Options: --newlines.

xml_declaration

Generate XML with an XML declaration. Also defined for the HTML Writer.

Caution!

The XML declaration carries text encoding information, without which standard tools may be unable to read the generated XML.

Default: do (1). Options: --no-xml-declaration.

[html4css1 writer]

attribution

Format for block quote attributions: one of "dash" (em-dash prefix), "parentheses"/"parens", or "none". Also defined for the LaTeX Writer.

Default: "dash". Options: --attribution.

compact_lists

Remove extra vertical whitespace between items of bullet lists and enumerated lists, when list items are "simple" (i.e., all items each contain one paragraph and/or one "simple" sublist only).

Default: enabled (1). Options: --compact-lists, --no-compact-lists.

embed_stylesheet

Embed the stylesheet in the output HTML file. The stylesheet file must be accessible during processing. The stylesheet is embedded inside a comment, so it must not contain the text "--" (two hyphens).

Default: link, don't embed (None). Options: --embed-stylesheet, --link-stylesheet.

footnote_references

Format for footnote references, one of "superscript" or "brackets". Also defined for the LaTeX Writer.

Default: "superscript"; "brackets" in PEP/HTML Writer. Options: --footnote-references.

initial_header_level

The initial level for header elements. This does not affect the document title & subtitle; see doctitle_xform.

Default: 1 (for "<h1>"). Option: --initial-header-level.

stylesheet

CSS stylesheet URL, used verbatim. Overridden by "stylesheet_path" URL option (--stylesheet-path). Also defined for the LaTeX Writer.

Default: "default.css". Options: --stylesheet.

stylesheet_path

Path to CSS stylesheet 1. Overrides "stylesheet" URL option (--stylesheet). Path is adjusted relative to the output HTML file. Also defined for the LaTeX Writer.

Default: None. Options: --stylesheet-path.

xml_declaration

Generate XML with an XML declaration. Also defined for the Docutils XML Writer.

Caution!

The XML declaration carries text encoding information, without which standard tools may be unable to read the generated XML.

Default: do (1). Options: --no-xml-declaration.

[pep_html writer]

The PEP/HTML Writer derives from the standard HTML Writer, and shares all settings defined in the [html4css1 writer] section. The "[html4css1 writer]" section is processed before "[pep_html writer]".

no_random

Workaround for platforms which core-dump on "import random".

Default: random enabled (None). Options: --no-random (hidden).

pep_home

Home URL prefix for PEPs.

Default: current directory ("."). Options: --pep-home.

template

Path to PEP template file 1.

Default: "pep-html-template" (in current directory). Options: --template.

python_home

Python's home URL.

Default: parent directory (".."). Options: --python-home.

[latex2e writer]

use_latex_toc

To get pagenumbers in the table of contents the table of contents must be generated by latex. Usually latex must be run twice to get numbers correct.

Note: LaTeX will number the sections, which might be a bug in this case.

Default: off. Option: --use-latex-toc.

use_latex_footnotes

Use LaTeX-footnotes not a figure simulation. This might give no Hyperrefs on /to footnotes, but should be able to handle an unlimited number of footnotes.

Default: off. Option: --use-latex-footnotes.

hyperlink_color

Color of any hyperlinks embedded in text. Use "0" to disable coloring of links.

Default: "blue", use "0" to disable. Option: --hyperlink-color.

documentclass

Specify latex documentclass, but beaware that books have chapters articles not.

Default: "article". Option: --documentclass.

documentoptions

Specify document options. Multiple options can be given, separated by commas.

Default is "10pt". Option: --documentoptions.

stylesheet

Specify a stylesheet file. The file will be input by latex in the document header. If this is set to "" disables generation of input latex command. Also defined for the HTML Writer.

Default: no stylesheet (""). Option: --stylesheet.

stylesheet_path

Path to stylesheet 1. Overrides "stylesheet" setting (--stylesheet). XXX LaTeX semantics? Also defined for the HTML Writer.

Default: None. Option: --stylesheet-path.

footnote_references

Format for footnote references: one of "superscript" or "brackets". Also defined for the HTML Writer.

Default is "brackets". Option: --footnote-references.

attribution

Format for block quote attributions, the same as for the html-writer: one of "dash" (em-dash prefix), "parentheses"/"parens" or "none". Also defined for the HTML Writer.

Default: "dash". Option: --attribution.

compound_enumerators

Enable or disable compound enumerators for nested enumerated lists (e.g. "1.2.a.ii").

Default: disabled (None). Options: --compound-enumerators, --no-compound-enumerators.

section_prefix_for_enumerators

Enable or disable section ("." subsection ...) prefixes for compound enumerators. This has no effect unless compound_enumerators are enabled.

Default: disabled (None). Options: --section-prefix-for-enumerators, --no-section-prefix-for-enumerators.

section_enumerator_separator

The separator between section number prefix and enumerator for compound enumerated lists (see compound_enumerators).

Generally it isn't recommended to use both sub-sections and nested enumerated lists with compound enumerators. This setting avoids ambiguity in the situation where a section "1" has a list item enumerated "1.1", and subsection "1.1" has list item "1". With a separator of ".", these both would translate into a final compound enumerator of "1.1.1". With a separator of "-", we get the unambiguous "1-1.1" and "1.1-1".

Default: "-". Option: --section-enumerator-separator.

table_style

Specify the drawing of separation lines.

  • "standard" lines around and between cells.
  • "booktabs" a line above and below the table and one after the head.
  • "nolines".

[pseudoxml writer]

No settings are defined for this Writer.

[applications]

[buildhtml application]

prune

List of directories not to process. To specify multiple directories in configuration files, use colon-separated paths; on the command line, the option may be used more than once.

Default: none ([]). Options: --prune.

recurse

Recursively scan subdirectories, or ignore subdirectories.

Default: recurse (1). Options: --recurse, --local.

silent

Work silently (no progress messages). Independent of "report_level".

Default: show progress (None). Options: --silent.

[docfactory application]

(To be completed.)

Other Settings

These settings are only effective as command-line options, positional arguments, or for internal use; setting them in configuration files has no effect.

config

Path to a configuration file to read (if it exists) 1. Settings may override defaults and earlier settings. The config file is processed immediately. Multiple --config options may be specified; each will be processed in turn.

Filesystem path settings contained within the config file will be interpreted relative to the config file's location (not relative to the current working directory).

Default: None. Options: --config.

_directories

(buildhtml.py front end.) List of paths to source directories, set from positional arguments.

Default: current working directory (None). No command-line options.

_disable_config

Prevent standard configuration files from being read. For internal use only.

Default: config files enabled (None). No command-line options.

_destination

Path to output destination, set from positional arguments.

Default: stdout (None). No command-line options.

_source

Path to input source, set from positional arguments.

Default: stdin (None). No command-line options.

[1] (1, 2, 3, 4, 5) Path relative to the working directory of the process at launch.

Old-Format Configuration Files

Formerly, Docutils configuration files contained a single "[options]" section only. This was found to be inflexible, and in August 2003 Docutils adopted the current component-based configuration file sections as described above. Docutils will still recognize the old "[options]" section, but complains with a deprecation warning.

To convert existing config files, the easiest way is to change the section title: change "[options]" to "[general]". Most settings haven't changed. The only ones to watch out for are these:

Old-Format Setting | New Section & Setting
pep_stylesheet | [pep_html writer] stylesheet
pep_stylesheet_path | [pep_html writer] stylesheet_path
pep_template | [pep_html writer] template
View document source. Generated on: 2004-07-01 22:50 UTC. Generated by Docutils from reStructuredText source.


Generating LaTeX with Docutils

Author: | Engelbert Gruber
Contact: | grubert@users.sourceforge.net
Revision: | 1.49
Date: | 2004-06-25
Copyright: | This document has been placed in the public domain.

Contents

  • Introduction
  • Options
    • Font selection
    • Unicode
    • Table of figures
    • Section numbering
    • images
  • Commands directly to LaTeX
    • Custom title page
  • Problems
    • footnotes and citations
    • Tilde in italian
    • Tables
      • Notes
    • Miscellaneous

Introduction

Producing LaTeX code from reST input could be done in at least two ways:

  1. Transform the internal markup into corresponding LaTeX markup e.g. a section title would be written as `\section{this section ...}.
  2. Using LaTeX as a typesetting system to produce desired paperwork without caring about loosing document structure information.

The former might be preferable, but limits to LaTeXs capabilities, so in reality it is a mix. The reality is that LaTeX has a titlepage with title, author and date, by default only title is used. Author and date are shown in the docutils docinfo table and set to blank for LaTeX. To get author and date set by LaTeX specify option "use-LaTeX-docinfo".

Options

Configuration can be done in two ways (again):

  1. Options to the docutils tool: e.g. language selection.
  2. Options to LaTeX via the stylesheet file.

The generated LaTeX documents should be kept processable by a standard LaTeX installation (if such a thing exists), therefore the document contains default settings. To allow overwriting defaults the stylesheet is included at last.

Run rst2latex.py --help to see the command-line options, or have look in config documentytion.

Configuration Issue | Description
papersize

Default: a4paper. Paper geometry can be changed using \geometry{xxx} entries.

Some possibilities:

  • a4paper, b3paper, letterpaper, executivepaper, legalpaper
  • landscape, portrait, twoside.

and a ton of other option setting margins.

An example:

\geometry{a5paper,landscape}
paragraph indent

By default LaTeX indents the forst line in a paragraph. The following lines set indentation to zero but add a vertical space between paragraphs.:

\setlength{\parindent}{0pt}
\setlength{\parskip}{6pt plus 2pt minus 1pt}
admonitionwidth

The width for admonitions. Default: 0.9*textwidth, this can be changed e.g.:

\setlength{\admonitionwidth}{0.7\textwidth}
docinfowidth

The width for the docinfo table. Default: 0.9*textwidth, changed to e.g.:

\setlength{\docinfowidth}{0.7\textwidth}
rubric style

The header contains the definition of a new LaTeX command rubric. Inserting:

\renewcommand{\rubric}[1]{\subsection*{
  ~\hfill {\color{red} #1} \hfill ~}}

sets rubric to subsection style in red.

Default: subsection style italic.
line spacing

Is done with package setspace, which gives singlespace, onehalfspace and doublespace commands. To get documentwide double spacing, add this to your stylesheet

\usepackage{setspace}
\doublespacing

Another way

\linespread{1.55}

And yet another, add doublesp to the documentoptions and e.g.

\setstretch{1.7}

for bigger linespacing.
font selection | see below

Font selection

When generating pdf-files from LaTeX, use the pdflatex command, the files are a lot smaller if postscript fonts are used. This was fixed by putting \usepackage{times} into the stylesheet.

It is said that the typewriter font in computer modern font, the default LaTeX font package, is too heavy compared to the others. There is a package or some commands too fix this, which i currently cannot find.

Some people diagnose a similar unbalance for the postscript fonts, the package to fix this is \usepackage{pslatex}. pslatex in contrast to the standard LaTeX fonts has a bold typewriter font.

As times does not use the appropriate mathematical fonts and pslatex does not work with T1 encodings one should use:

\usepackage{mathptmx}
\usepackage[scaled=.90]{helvet}
\usepackage{courier}

Unicode

The generated LaTeX documents are in latin1 encoding per default, if unicode characters are required one must set --output-encoding=utf-8 install LaTeX unicode support and add:

\usepackage{ucs}
\usepackage[utf8]{inputenc}

to the stylesheet.

Table of figures

A table of figures can be generated by a command directly to LaTeX:

.. raw:: LaTeX

   \listoffigures

LaTeX also has a command \listoftables.

Section numbering

If section numbering and LaTeX table of contents is used LaTeX and docutils will number sections. To switch off displaying of LaTeX's numbers one has to add following lines to the stylesheet

% no section number display
\makeatletter
\def\@seccntformat#1{}
\makeatother
% no numbers in toc
\renewcommand{\numberline}[1]{}

images

Images are included in LaTeX by the graphicx package. The supported image formats depend on the used driver (dvi, dvips, pdftex, ...).

pdf-image inclusion in pdf files fails, specify --graphicx-option=pdftex or --graphicx-option=auto.

Commands directly to LaTeX

By means of the reST-raw directive one can give commands directly to LaTeX, e.g. forcing a page break:

.. raw:: LaTeX

   \newpage

Or setting formulas in LaTeX:

.. raw:: LaTeX

   $$x^3 + 3x^2a + 3xa^2 + a^3,$$

Or making a colorbox: If someone wants to get a red background for a textblock, she/he can put definecolor{bg}{rgb}{.9,0,0} into style.tex and in reStructuredText do something like this:

|begincolorbox|
Nobody expects the spanish inquisition.
|endcolorbox|

.. |begincolorbox| raw:: LaTeX

   \\begin{center}
   \\colorbox{bg}{
   \\parbox{0.985\\linewidth}{

.. |endcolorbox| raw:: LaTeX

   }}
   \\end{center}

Custom title page

Currently maketitle only shows the title and subtitle, date and author are shown in the docinfo table.

To change the titlepage layout, e.g. see fancyhdr, one must redefine the maketitle command in the stylesheet:

\renewcommand{\maketitle}{
  \begin{titlepage}
    \begin{center}
    \textsf{TITLE \@title} \\
    Date: \today
    \end{center}
  \end{titlepage}
}

\@title contains the title.

Problems

Open to be fixed or open to discussion.

footnotes and citations

Initially both were implemented using figures, because hyperlinking back and forth seemed to be impossible. Later images were put into figures.

This results in footnotes images and figures possibly being mixed at page foot.

  • Use LaTeX footnotes and citations for printing or more complex layout.
  • Footnotes and citations done with figures might excell in hyperlink support.

If use-latex-citations is used a bibliography is inserted right at the end of the document. This should be customizable.

Tilde in italian

Does not work, or only in verbatim or verb. Could be fixed by using T1, but the hyphen in literal environments is different:

  • Inline or in verbatim it is lower than it should. If package courier is used acroread 5 puts it too high, xpdf shows it at the same height as the horizontal line of +.
  • In a block with markup two hyphens become one (endashed ?). and <<, >> become separate characters too.

Tables

Tablewidth:

reST-documents line length is assumed to be 80 characters. The tablewidth is set relative to this value. If someone produces documents with line length of 132 this will fail.

Table width is tried to fit in page even if it is wider than the assumed linewidth, still assumed linewidth is a hook.

  • In tools.txt the option tables right column, there should be some more spacing between the description and the next paragraph "Default:".

    Paragraph separation in tables is hairy. see http://www.tex.ac.uk/cgi-bin/texfaq2html?label=struttab

    • The strut solution did not work.
    • setting extrarowheight added ad top of row not between paragraphs in a cell. ALTHOUGH i set it to 2pt because, text is too close to the topline.
    • baselineskip/stretch does not help.

  • Should there be two hlines after table head and on table end ?

  • Table: multicol cells are always {l}.

  • The contents of a rowspan cell do not influence table height. (Maybe if we put a tabular inside ?)

  • Table heads and footer for longtable (firstpage lastpage ..).

  • Table cells with multirow and multicolumn

  • literal-blocks in table cells:

    • If verbatim or flushleft is used one gets vertical space above and below.
    • This is bad for the topmost paragraph in a cell, therefore the writer uses raggedright.
    • Ragged right fails on followup paragraphs as the vertical space would be missing.

Notes

  • table-style booktabs: booktabs.sty 1.00 does not work with longtable.

Miscellaneous

  • Selection of LaTeX fontsize configurable.

  • Assumed reST linelength for table width setting configurable.

  • literal-block indentation configurable.

  • The underscore _ does not work in literal-blocks, has different width. I tried

    • Underline a blank \underline{ } to low, and does not fit the font. But on many systems underscore is an underlined blank.
    • Escape it {\_} to narrow
    • Subscript a hyphen $_{\tt-}$ too wide and thin
    • $_{-}$ the same.

  • recongize LaTeX and replace by \LaTeX.

  • Support embed-stylesheet.

  • the ^-sign is problematic: using mathmode wedge is usually the wrong font.

  • Sidebar handling.

  • Maybe add end of line after term in definition list. see

    http://roundup.sf.net/doc-0.5/features.pdf

  • Pdfbookmark level 4 (and greater) does not work (might be settable but OTOH).

  • center subsection{Abstract} gives a LaTeX error here. ! LaTeX Error: Something's wrong--perhaps a missing \item. Committed a HACK: centering by hfill.

  • Document errors are also too silent.

  • Use optionlist for docinfo, the table does only work for single page.

  • Consider peter funk's hooks for TeXpert:

    • Define his own document preamble (including the choice to choose his own documentclass. That would make the --documentclass option superfluous). I suggest to call this option --preamble
    • Use two additional hooks to put additional stuff just behind the \begin{document} and just before the \end{document} macros. Typical uses would be \tableofcontents, \listoffigures and \appendix, \makeindex, \makeglossary and some such for larger documents.

  • Hyphens: co-developers should be co--developers ?

  • The indentional problematic error in test.txt is not referring anywhere.

  • Footnotes are not all on the same page (as in tools/test.txt) and do not link back and forth.

  • No link to system errors.

  • Hyperlinks are not hyphenated this leads to bad spacing. see tools/test.txt 2.14 directives directives

  • Meta keywords into pdf ?

  • Pagestyle headings does not work, when sections are starred.

  • For additional docinfo items: the field_body is inserted as text, i.e. no markup is done.

  • Multiple author entries in docinfo (same thing as in html).

  • pslatex has not textbackslash in bold ?

  • make use (but test against pslatex)

    \textbackslash      \backslash \
\textbar            \mid       |
\textless           <          <
\textgreater        >          >
\textasciicircum    \hat       ^
\textasciitilde     \tilde     ~
\textbullet         \bullet    gefüllter Kreis auf halber Höhe
\textperiodcentered \cdot      ·
\textvisiblespace   (n.v.)     sichtbares Leerzeichen

  • keep literal-blocks together on a page, avoid pagebreaks.

    failed experiments up to now: samepage, minipage, pagebreak 1 to 4 before the block.

View document source. Generated on: 2004-07-01 22:50 UTC. Generated by Docutils from reStructuredText source.


Docutils Front-End Tools

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.45
Date: | 2004-06-07
Copyright: | This document has been placed in the public domain.

Contents

  • Introduction
    • Getting Help
  • The Tools
    • buildhtml.py
    • rst2html.py
      • Stylesheets
    • pep.py
    • pep2html.py
    • rst2latex.py
    • docutils-xml.py
    • publish.py
    • quicktest.py
  • Customization
    • Command-Line Options
    • Configuration Files

Introduction

Once the Docutils package is unpacked, you will discover a "tools" directory containing several front ends for common Docutils processing. Rather than a single all-purpose program, Docutils has many small front ends, each specialized for a specific "Reader" (which knows how to interpret a file in context), a "Parser" (which understands the syntax of the text), and a "Writer" (which knows how to generate a specific data format).

Most front ends have common options and the same command-line usage pattern:

toolname [options] [<source> [<destination]]

(The exceptions are buildhtml.py and pep2html.py.) See rst2html.py for concrete examples. Each tool has a "--help" option which lists the command-line options and arguments it supports. Processing can also be customized with configuration files.

The two arguments, "source" and "destination", are optional. If only one argument (source) is specified, the standard output (stdout) is used for the destination. If no arguments are specified, the standard input (stdin) is used for the source as well.

Getting Help

First, try the "--help" option each front-end tool has.

Users who have questions or need assistance with Docutils or reStructuredText should post a message to the Docutils-Users mailing list. The Docutils project web site has more information.

The Tools

buildhtml.py

Readers: | Standalone, PEP
Parser: | reStructuredText
Writers: | HTML, PEP/HTML

Use buildhtml.py to generate .html from all the .txt files (including PEPs) in each <directory> given, and their subdirectories too. (Use the --local option to skip subdirectories.)

Usage:

buildhtml.py [options] [<directory> ...]

After unpacking the Docutils package, the following shell commands will generate HTML for all included documentation:

cd docutils/tools
buildhtml.py ..

For official releases, the directory may be called "docutils-X.Y", where "X.Y" is the release version. Alternatively:

cd docutils
tools/buildhtml.py --config=tools/docutils.conf

The current directory (and all subdirectories) is chosen by default if no directory is named. Some files may generate system messages (tools/test.txt contains intentional errors); use the --quiet option to suppress all warnings. The --config option ensures that the correct stylesheets, templates, and settings are in place (a docutils.conf configuration file in the current directory is picked up automatically). Command-line options may be used to override config file settings or replace them altogether.

rst2html.py

Reader: | Standalone
Parser: | reStructuredText
Writer: | HTML

The rst2html.py front end reads standalone reStructuredText source files and produces HTML 4 (XHTML 1) output compatible with modern browsers. For example, to process a reStructuredText file "test.txt" into HTML:

rst2html.py test.txt test.html

In fact, there is a "test.txt" file in the "tools" directory. It contains "at least one example of each reStructuredText construct", including several intentional errors (to test the error reporting system). Use it to put the system through its paces and compare input to output.

Now open the "test.html" file in your favorite browser to see the results. To get a footer with a link to the source file, date & time of processing, and links to the Docutils projects, add some options:

rst2html.py -stg test.txt test.html

Stylesheets

rst2html.py inserts into the generated HTML a link to a cascading stylesheet, defaulting to "default.css" (override with a "--stylesheet" or "--stylesheet-path" command-line option or with configuration file settings). The "tools/stylesheets/default.css" stylesheet is provided for basic use. To experiment with styles, rather than editing the default stylesheet (which will be updated as the project evolves), it is recommended to use an "@import" statement to create a "wrapper" stylesheet. For example, a "my.css" stylesheet could contain the following:

@import url(default.css);

h1, h2, h3, h4, h5, h6, p.topic-title {
  font-family: sans-serif }

Generate HTML with the following command:

rst2html.py -stg --stylesheet my.css test.txt test.html

When viewed in a browser, the new "wrapper" stylesheet will change the typeface family of titles to "sans serif", typically Helvetica or Arial. Other styles will not be affected. Styles in wrapper stylesheets override styles in imported stylesheets, enabling incremental experimentation.

pep.py

Reader: | PEP
Parser: | reStructuredText
Writer: | PEP/HTML

pep.py reads a new-style PEP (marked up with reStructuredText) and produces HTML. It requires a template file and a stylesheet. By default, it makes use of a "pep-html-template" file and a "default.css" stylesheet in the current directory, but these can be overridden by command-line options or configuration files. The "tools/stylesheets/pep.css" stylesheet is intended specifically for PEP use.

The "docutils.conf" configuration file in the "tools" directory of Docutils contains a default setup for use in processing the PEP files (docs/peps/pep-*.txt) into HTML. It specifies a default template (tools/pep-html-template) and a default stylesheet (tools/stylesheets/pep.css). See Stylesheets above for more information.

pep.py can be run from the tools directory or from the docs/peps/ directory, by adjusting the settings. These two sets of commands are equivalent:

cd <path-to-docutils>/tools
# This will pick up the "docutils.conf" file automatically:
pep.py ../docs/peps/pep-0287.txt ../docs/peps/pep-0287.html

cd <path-to-docutils>/docs/peps
# Must tell the tool where to find the config file:
../../tools/pep.py --config ../../tools/docutils.conf \
    pep-0287.txt pep-0287.html

pep2html.py

Reader: | PEP
Parser: | reStructuredText
Writer: | PEP/HTML

pep2html.py is a modified version of the original script by Fredrik Lundh, with support for Docutils added. It reads the beginning of a PEP text file to determine the format (old-style indented or new-style reStructuredText) and processes accordingly. Since it does not use the Docutils front end mechanism (the common command-line options are not supported), it can only be configured using configuration files. The template and stylesheet requirements of pep2html.py are the same as those of pep.py above.

Arguments to pep2html.py may be a list of PEP numbers or .txt files. If no arguments are given, all files of the form "pep-*.txt" are processed.

rst2latex.py

Reader: | Standalone
Parser: | reStructuredText
Writer: | LaTeX2e

The rst2latex.py front end reads standalone reStructuredText source files and produces LaTeX2e output. For example, to process a reStructuredText file "test.txt" into LaTeX:

rst2latex.py test.txt test.tex

The output file "test.tex" should then be processed with latex or pdflatex to get a typeset document.

Some limitations and difference apply:

  • GIF, JPG and PNG images are not handled, when processed with latex; use pdflatex instead.
  • Only the Latin-1 output encoding has been tested up to now (Latin-1 has been made the default output encoding for LaTeX).
  • The optional stylesheet file allows the inclusion of special packages or overwriting default settings for LaTeX.
  • Not all constructs are possible, see Generating LaTeX with Docutils.

docutils-xml.py

Reader: | Standalone
Parser: | reStructuredText
Writer: | XML (Docutils native)

The docutils-xml.py front end produces Docutils-native XML output. This can be transformed with standard XML tools such as XSLT processors into arbitrary final forms.

publish.py

Reader: | Standalone
Parser: | reStructuredText
Writer: | Pseudo-XML

publish.py is used for debugging the Docutils "Reader to Transform to Writer" pipeline. It produces a compact pretty-printed "pseudo-XML", where nesting is indicated by indentation (no end-tags). External attributes for all elements are output, and internal attributes for any leftover "pending" elements are also given.

quicktest.py

Reader: | N/A
Parser: | reStructuredText
Writer: | N/A

The quicktest.py tool is used for testing the reStructuredText parser. It does not use a Docutils Reader or Writer or the standard Docutils command-line options. Rather, it does its own I/O and calls the parser directly. No transforms are applied to the parsed document. Various forms output are possible:

  • Pretty-printed pseudo-XML (default)
  • Test data (Python list of input and pseudo-XML output strings; useful for creating new test cases)
  • Pretty-printed native XML
  • Raw native XML (with or without a stylesheet reference)

Customization

Command-Line Options

Each front-end tool supports command-line options for one-off customization. For persistent customization, use configuration files. Command-line options take priority over configuration file settings.

Use the "--help" option on each of the front ends to list the command-line options it supports. Command-line options and their corresponding configuration file entry names are listed in the Docutils Configuration Files document.

Configuration Files

Configuration files are used for persistent customization; they can be set once and take effect every time you use a front-end tool.

For details, see Docutils Configuration Files.

View document source. Generated on: 2004-07-01 22:50 UTC. Generated by Docutils from reStructuredText source.


=====================================================
 The reStructuredText_ Cheat Sheet: Syntax Reminders
=====================================================
:Info: See <http://docutils.sf.net/rst.html> for introductory docs.
:Author: David Goodger <goodger@python.org>
:Date: $Date: 2004/06/25 21:44:27 $
:Revision: $Revision: 1.7 $
:Description: This is a "docinfo block", or bibliographic field list

Section Structure
=================
Section titles are underlined or overlined & underlined.

Body Elements
=============
Grid table:

+--------------------------------+-----------------------------------+
| Paragraphs are flush-left,     | Literal block, preceded by "::":: |
| separated by blank lines.      |                                   |
|                                |     Indented                      |
|     Block quotes are indented. |                                   |
+--------------------------------+ or::                              |
| >>> print 'Doctest block'      |                                   |
| Doctest block                  | > Quoted                          |
+--------------------------------+-----------------------------------+

Simple tables:

================  ====================================================
List Type         Examples
================  ====================================================
Bullet list       * items begin with "-", "+", or "*"
Enumerated list   1. items use any variation of "1.", "A)", and "(i)"
Definition list   Term is flush-left : optional classifier
                      Definition is indented, no blank line between
Field list        :field name: field body
Option list       -o  at least 2 spaces between option & description
================  ====================================================

================  ====================================================
Explicit Markup   Examples (only visible in the `text source <cheatsheet.txt>`_)
================  ====================================================
Footnote          .. [1] Manually numbered or [#] auto-numbered
                     (even [#labelled]) or [*] auto-symbol
Citation          .. [CIT2002] A citation.
Hyperlink Target  .. _reStructuredText: http://docutils.sf.net/rst.html
                  .. _indirect target: reStructuredText_
                  .. _internal target:
Anonymous Target  __ http://docutils.sf.net/docs/ref/rst/restructuredtext.html
Directive ("::")  .. image:: images/biohazard.png
Substitution Def  .. |substitution| replace:: like an inline directive
Comment           .. is anything else
================  ====================================================

Inline Markup
=============
*emphasis*; **strong emphasis**; `interpreted text`; `interpreted text
with role`:emphasis:; ``inline literal text``; standalone hyperlink,
http://docutils.sourceforge.net; named reference, reStructuredText_;
`anonymous reference`__; footnote reference, [1]_; citation reference,
[CIT2002]_; |substitution|; _`inline internal target`.

Directive Quick Reference
=========================
See <http://docutils.sf.net/docs/ref/rst/directives.html> for full info.

================  ====================================================
Directive Name    Description
================  ====================================================
attention         Specific admonition; also "caution", "danger",
                  "error", "hint", "important", "note", "tip", "warning"
admonition        Generic titled admonition: ``.. admonition:: By The Way``
image             ``.. image:: picture.png``; many options possible
figure            Like "image", but with optional caption and legend
topic             ``.. topic:: Title``; like a mini section
sidebar           ``.. sidebar:: Title``; like a mini parallel document
line-block        Line breaks and leading whitespace is significant;
                  useful for addresses & verse
parsed-literal    A literal block with parsed inline markup
rubric            ``.. rubric:: Informal Heading``
epigraph          Block quote with class="epigraph"
highlights        Block quote with class="highlights"
pull-quote        Block quote with class="pull-quote"
table             Create a titled table (new in Docutils 0.3.1)
csv-table         Create a titled table from CSV data (requires Python 2.3+;
                  new in Docutils 0.3.4)
contents          Generate a table of contents
sectnum           Automatically number sections, subsections, etc.
target-notes      Create an explicit footnote for each external target
meta              HTML-specific metadata
include           Read an external reST file as if it were inline
raw               Non-reST data passed untouched to the Writer
replace           Replacement text for substitution definitions
unicode           Unicode character code conversion for substitution defs
class             Set a "class" attribute on the next element
role              Create a custom interpreted text role (new in Docutils 0.3.2)
================  ====================================================

Interpreted Text Role Quick Reference
=====================================
See <http://docutils.sf.net/docs/ref/rst/roles.html> for full info.

================  ====================================================
Role Name         Description
================  ====================================================
emphasis          Equivalent to *emphasis*
literal           Equivalent to ``literal``, except not as literal
PEP               Reference to a numbered Python Enhancement Proposal
RFC               Reference to a numbered Internet Request For Comments
strong            Equivalent to **strong**
sub               Subscript
sup               Superscript
title             Title reference (book, etc.); standard default role
================  ====================================================


A ReStructuredText Primer

Author: | Richard Jones
Version: | 1.17
Copyright: | This document has been placed in the public domain.

Contents

  • Structure
  • Text styles
  • Lists
  • Preformatting (code samples)
  • Sections
    • Document Title / Subtitle
  • Images
  • What Next?

The text below contains links that look like "(quickref)". These are relative links that point to the Quick reStructuredText user reference. If these links don't work, please refer to the master quick reference document.

Structure

From the outset, let me say that "Structured Text" is probably a bit of a misnomer. It's more like "Relaxed Text" that uses certain consistent patterns. These patterns are interpreted by a HTML converter to produce "Very Structured Text" that can be used by a web browser.

The most basic pattern recognised is a paragraph (quickref). That's a chunk of text that is separated by blank lines (one is enough). Paragraphs must have the same indentation -- that is, line up at their left edge. Paragraphs that start indented will result in indented quote paragraphs. For example:

This is a paragraph.  It's quite
short.

   This paragraph will result in an indented block of
   text, typically used for quoting other text.

This is another one.

Results in:

This is a paragraph. It's quite short.

This paragraph will result in an indented block of text, typically used for quoting other text.

This is another one.

Text styles

(quickref)

Inside paragraphs and other bodies of text, you may additionally mark text for italics with "*italics*" or bold with "**bold**".

If you want something to appear as a fixed-space literal, use "``double back-quotes``". Note that no further fiddling is done inside the double back-quotes -- so asterisks "*" etc. are left alone.

If you find that you want to use one of the "special" characters in text, it will generally be OK -- reStructuredText is pretty smart. For example, this * asterisk is handled just fine. If you actually want text *surrounded by asterisks* to not be italicised, then you need to indicate that the asterisk is not special. You do this by placing a backslash just before it, like so "\*" (quickref), or by enclosing it in double back-quotes (inline literals), like this:

``\*``

Lists

Lists of items come in three main flavours: enumerated, bulleted and definitions. In all list cases, you may have as many paragraphs, sublists, etc. as you want, as long as the left-hand side of the paragraph or whatever aligns with the first line of text in the list item.

Lists must always start a new paragraph -- that is, they must appear after a blank line.

enumerated lists (numbers, letters or roman numerals; quickref)

Start a line off with a number or letter followed by a period ".", right bracket ")" or surrounded by brackets "( )" -- whatever you're comfortable with. All of the following forms are recognised:

1. numbers

A. upper-case letters
   and it goes over many lines

   with two paragraphs and all!

a. lower-case letters

   3. with a sub-list starting at a different number
   4. make sure the numbers are in the correct sequence though!

I. upper-case roman numerals

i. lower-case roman numerals

(1) numbers again

1) and again

Results in (note: the different enumerated list styles are not always supported by every web browser, so you may not get the full effect here):

  1. numbers

  1. upper-case letters and it goes over many lines

    with two paragraphs and all!

  1. lower-case letters
    3. with a sub-list starting at a different number
    4. make sure the numbers are in the correct sequence though!
  1. upper-case roman numerals
  1. lower-case roman numerals
  1. numbers again
  1. and again
bulleted lists (quickref)

Just like enumerated lists, start the line off with a bullet point character - either "-", "+" or "*":

* a bullet point using "*"

  - a sub-list using "-"

    + yet another sub-list

  - another item

Results in:

  • a bullet point using "*"
    • a sub-list using "-"
      • yet another sub-list
    • another item
definition lists (quickref)

Unlike the other two, the definition lists consist of a term, and the definition of that term. The format of a definition list is:

what
  Definition lists associate a term with a definition.

*how*
  The term is a one-line phrase, and the definition is one or more
  paragraphs or body elements, indented relative to the term.
  Blank lines are not allowed between term and definition.

Results in:

what
Definition lists associate a term with a definition.
how
The term is a one-line phrase, and the definition is one or more paragraphs or body elements, indented relative to the term. Blank lines are not allowed between term and definition.

Preformatting (code samples)

(quickref)

To just include a chunk of preformatted, never-to-be-fiddled-with text, finish the prior paragraph with "::". The preformatted block is finished when the text falls back to the same indentation level as a paragraph prior to the preformatted block. For example:

An example::

    Whitespace, newlines, blank lines, and all kinds of markup
      (like *this* or \this) is preserved by literal blocks.
  Lookie here, I've dropped an indentation level
  (but not far enough)

no more example

Results in:

An example:

  Whitespace, newlines, blank lines, and all kinds of markup
    (like *this* or \this) is preserved by literal blocks.
Lookie here, I've dropped an indentation level
(but not far enough)

no more example

Note that if a paragraph consists only of "::", then it's removed from the output:

::

    This is preformatted text, and the
    last "::" paragraph is removed

Results in:

This is preformatted text, and the
last "::" paragraph is removed

Sections

(quickref)

To break longer text up into sections, you use section headers. These are a single line of text (one or more words) with adornment: an underline alone, or an underline and an overline together, in dashes "-----", equals "======", tildes "~~~~~~" or any of the non-alphanumeric characters = - ` : ' " ~ ^ _ * + # < > that you feel comfortable with. An underline-only adornment is distinct from an overline-and-underline adornment using the same character. The underline/overline must be at least as long as the title text. Be consistent, since all sections marked with the same adornment style are deemed to be at the same level:

Chapter 1 Title
===============

Section 1.1 Title
-----------------

Subsection 1.1.1 Title
~~~~~~~~~~~~~~~~~~~~~~

Section 1.2 Title
-----------------

Chapter 2 Title
===============

This results in the following structure, illustrated by simplified pseudo-XML:

<section>
    <title>
        Chapter 1 Title
    <section>
        <title>
            Section 1.1 Title
        <section>
            <title>
                Subsection 1.1.1 Title
    <section>
        <title>
            Section 1.2 Title
<section>
    <title>
        Chapter 2 Title

(Pseudo-XML uses indentation for nesting and has no end-tags. It's not possible to show actual processed output, as in the other examples, because sections cannot exist inside block quotes. For a concrete example, compare the section structure of this document's source text and processed output.)

Note that section headers are available as link targets, just using their name. To link to the Lists heading, I write "Lists_". If the heading has a space in it like text styles, we need to quote the heading "`text styles`_".

Document Title / Subtitle

The title of the whole document is distinct from section titles and may be formatted somewhat differently (e.g. the HTML writer by default shows it as a centered heading).

To indicate the document title in reStructuredText, use a unique adornment style at the beginning of the document. To indicate the document subtitle, use another unique adornment style immediately after the document title. For example:

================
 Document Title
================
----------
 Subtitle
----------

Section Title
=============

...

Note that "Document Title" and "Section Title" above both use equals signs, but are distict and unrelated styles. The text of overline-and-underlined titles (but not underlined-only) may be inset for aesthetics.

Images

(quickref)

To include an image in your document, you use the the image directive. For example:

.. image:: images/biohazard.png

results in:

The images/biohazard.png part indicates the filename of the image you wish to appear in the document. There's no restriction placed on the image (format, size etc). If the image is to appear in HTML and you wish to supply additional information, you may:

.. image:: images/biohazard.png
   :height: 100
   :width: 200
   :scale: 50
   :alt: alternate text

See the full image directive documentation for more info.

What Next?

This primer introduces the most common features of reStructuredText, but there are a lot more to explore. The Quick reStructuredText user reference is a good place to go next. For complete details, the reStructuredText Markup Specification is the place to go 1.

Users who have questions or need assistance with Docutils or reStructuredText should post a message to the Docutils-Users mailing list. The Docutils project web site has more information.

[1] If that relative link doesn't work, try the master document: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html.
View document source. Generated on: 2004-07-01 22:51 UTC. Generated by Docutils from reStructuredText source.


Quick re Structured Text

http://docutils.sourceforge.net/docs/user/rst/quickref.html
Being a cheat-sheet for reStructuredText
Updated $Date: 2004/06/11 19:51:17 $

Copyright: This document has been placed in the public domain.

The full details of the markup may be found on the reStructuredText page. This document is just intended as a reminder.

Links that look like "(details)" point into the HTML version of the full reStructuredText specification document. These are relative links; if they don't work, please use the master "Quick reStructuredText" document.

Contents

  • Inline Markup
  • Escaping with Backslashes
  • Section Structure
  • Paragraphs
  • Bullet Lists
  • Enumerated Lists
  • Definition Lists
  • Field Lists
  • Option Lists
  • Literal Blocks
  • Block Quotes
  • Doctest Blocks
  • Tables
  • Transitions
  • Explicit Markup
    • Footnotes
    • Citations
    • Hyperlink Targets
      • External Hyperlink Targets
      • Internal Hyperlink Targets
      • Indirect Hyperlink Targets
      • Implicit Hyperlink Targets
    • Directives
    • Substitution References and Definitions
    • Comments
  • Getting Help

Inline Markup

(details)

Inline markup allows words and phrases within text to have character styles (like italics and boldface) and functionality (like hyperlinks).

Plain text | Typical result | Notes
*emphasis* | emphasis | Normally rendered as italics.
**strong emphasis** | strong emphasis | Normally rendered as boldface.
`interpreted text` | (see note at right) | The rendering and meaning of interpreted text is domain- or application-dependent. It can be used for things like index entries or explicit descriptive markup (like program identifiers).
``inline literal`` | inline literal | Normally rendered as monospaced text. Spaces should be preserved, but line breaks will not be.
reference_ | reference | A simple, one-word hyperlink reference. See Hyperlinks.
`phrase reference`_ | phrase reference | A hyperlink reference with spaces or punctuation needs to be quoted with backquotes. See Hyperlinks.
anonymous__ | anonymous | With two underscores instead of one, both simple and phrase references may be anonymous (the reference text is not repeated at the target). See Hyperlinks.
_`inline internal target` | inline internal target | A crossreference target within text. See Hyperlinks.
|substitution reference| | (see note at right) | The result is substituted in from the substitution definition. It could be text, an image, a hyperlink, or a combination of these and others.
footnote reference [1]_ | footnote reference 1 | See Footnotes.
citation reference [CIT2002]_ | citation reference [CIT2002] | See Citations.
http://docutils.sf.net/ | http://docutils.sf.net/ | A standalone hyperlink.

Asterisk, backquote, vertical bar, and underscore are inline delimiter characters. Asterisk, backquote, and vertical bar act like quote marks; matching characters surround the marked-up word or phrase, whitespace or other quoting is required outside them, and there can't be whitespace just inside them. If you want to use inline delimiter characters literally, escape (with backslash) or quote them (with double backquotes; i.e. use inline literals).

In detail, the reStructuredText specification says that in inline markup, the following rules apply to start-strings and end-strings (inline markup delimiters):

  1. The start-string must start a text block or be immediately preceded by whitespace or any of ' " ( [ { or <.
  2. The start-string must be immediately followed by non-whitespace.
  3. The end-string must be immediately preceded by non-whitespace.
  4. The end-string must end a text block (end of document or followed by a blank line) or be immediately followed by whitespace or any of ' " . , : ; ! ? - ) ] } / \ or >.
  5. If a start-string is immediately preceded by one of ' " ( [ { or <, it must not be immediately followed by the corresponding character from ' " ) ] } or >.
  6. An end-string must be separated by at least one character from the start-string.
  7. An unescaped backslash preceding a start-string or end-string will disable markup recognition, except for the end-string of inline literals.

Also remember that inline markup may not be nested (well, except that inline literals can contain any of the other inline markup delimiter characters, but that doesn't count because nothing is processed).

Escaping with Backslashes

(details)

reStructuredText uses backslashes ("\") to override the special meaning given to markup characters and get the literal characters themselves. To get a literal backslash, use an escaped backslash ("\\"). For example:

Raw reStructuredText | Typical result
*escape* ``with`` "\" | escape with ""
\*escape* \``with`` "\\" | *escape* ``with`` "\"

In Python strings it will, of course, be necessary to escape any backslash characters so that they actually reach reStructuredText. The simplest way to do this is to use raw strings:

Python string | Typical result
r"""\*escape* \`with` "\\"""" | *escape* `with` "\"
"""\\*escape* \\`with` "\\\\"""" | *escape* `with` "\"
"""\*escape* \`with` "\\"""" | escape with ""

Section Structure

(details)

Plain text | Typical result
===== Title ===== Subtitle -------- Titles are underlined (or over- and underlined) with a printing nonalphanumeric 7-bit ASCII character. Recommended choices are "``= - ` : ' " ~ ^ _ * + # < >``". The underline/overline must be at least as long as the title text. A lone top-level (sub)section is lifted up to be the document's (sub)title. | Title Subtitle Titles are underlined (or over- and underlined) with a printing nonalphanumeric 7-bit ASCII character. Recommended choices are "= - ` : ' " ~ ^ _ * + # < >". The underline/overline must be at least as long as the title text. A lone top-level (sub)section is lifted up to be the document's (sub)title.

Paragraphs

(details)

Plain text | Typical result
This is a paragraph. Paragraphs line up at their left edges, and are normally separated by blank lines. | This is a paragraph. Paragraphs line up at their left edges, and are normally separated by blank lines.

Bullet Lists

(details)

Plain text Typical result
Bullet lists:

- This is item 1
- This is item 2

- Bullets are "-", "*" or "+".
  Continuing text must be aligned
  after the bullet and whitespace.

Note that a blank line is required
before the first item and after the
last, but is optional between items.

Bullet lists:
  • This is item 1
  • This is item 2
  • Bullets are "-", "*" or "+". Continuing text must be aligned after the bullet and whitespace.

Note that a blank line is required before the first item and after the last, but is optional between items.

Enumerated Lists

(details)

Plain text Typical result
Enumerated lists:

3. This is the first item
4. This is the second item
5. Enumerators are arabic numbers,
   single letters, or roman numerals
6. List items should be sequentially
   numbered, but need not start at 1
   (although not all formatters will
   honour the first index).

Enumerated lists:
  3. This is the first item
  4. This is the second item
  5. Enumerators are arabic numbers, single letters, or roman numerals
  6. List items should be sequentially numbered, but need not start at 1 (although not all formatters will honour the first index).

Definition Lists

(details)

Plain text Typical result
Definition lists:

what
  Definition lists associate a term with
  a definition.

how
  The term is a one-line phrase, and the
  definition is one or more paragraphs or
  body elements, indented relative to the
  term. Blank lines are not allowed
  between term and definition. 		Definition lists:
what
Definition lists associate a term with a definition.
how
The term is a one-line phrase, and the definition is one or more paragraphs or body elements, indented relative to the term. Blank lines are not allowed between term and definition.

Field Lists

(details)

Plain text Typical result
:Authors:
    Tony J. (Tibs) Ibbs,
    David Goodger

    (and sundry other good-natured folks)

:Version: 1.0 of 2001/08/08
:Dedication: To my father.

Authors: | Tony J. (Tibs) Ibbs, David Goodger
 | (and sundry other good-natured folks)
Version: | 1.0 of 2001/08/08
Dedication: | To my father.

Field lists are used as part of an extension syntax, such as options for directives, or database-like records meant for further processing. Field lists may also be used as generic two-column table constructs in documents.

Option Lists

(details)

Plain text Typical result

-a            command-line option "a"
-b file       options can have arguments
              and long descriptions
--long        options can be long also
--input=file  long options can also have
              arguments
/V            DOS/VMS-style options too

-a | command-line option "a"
-b file | options can have arguments and long descriptions
--long | options can be long also
--input=file | long options can also have arguments
/V | DOS/VMS-style options too

There must be at least two spaces between the option and the description.

Literal Blocks

(details)

Plain text Typical result
A paragraph containing only two colons
indicates that the following indented
or quoted text is a literal block.

::

  Whitespace, newlines, blank lines, and
  all kinds of markup (like *this* or
  \this) is preserved by literal blocks.

  The paragraph containing only '::'
  will be omitted from the result.

The ``::`` may be tacked onto the very
end of any paragraph. The ``::`` will be
omitted if it is preceded by whitespace.
The ``::`` will be converted to a single
colon if preceded by text, like this::

  It's very convenient to use this form.

Literal blocks end when text returns to
the preceding paragraph's indentation.
This means that something like this
is possible::

      We start here
    and continue here
  and end here.

Per-line quoting can also be used on
unindented literal blocks:

> Useful for quotes from email and
> for Haskell literate programming.

A paragraph containing only two colons indicates that the following indented or quoted text is a literal block.

  Whitespace, newlines, blank lines, and
  all kinds of markup (like *this* or
  \this) is preserved by literal blocks.

  The paragraph containing only '::'
  will be omitted from the result.

The :: may be tacked onto the very end of any paragraph. The :: will be omitted if it is preceded by whitespace. The :: will be converted to a single colon if preceded by text, like this:

  It's very convenient to use this form.

Literal blocks end when text returns to the preceding paragraph's indentation. This means that something like this is possible:

      We start here
    and continue here
  and end here.

Per-line quoting can also be used on unindented literal blocks:

  > Useful for quotes from email and
  > for Haskell literate programming.

Block Quotes

(details)

Plain text Typical result
Block quotes are just:

    Indented paragraphs,

        and they may nest.

Block quotes are just:

Indented paragraphs,

and they may nest.

Doctest Blocks

(details)

Plain text Typical result

Doctest blocks are interactive
Python sessions. They begin with
"``>>>``" and end with a blank line.

>>> print "This is a doctest block."
This is a doctest block.

Doctest blocks are interactive Python sessions. They begin with ">>>" and end with a blank line.

>>> print "This is a doctest block."
This is a doctest block.

"The doctest module searches a module's docstrings for text that looks like an interactive Python session, then executes all such sessions to verify they still work exactly as shown." (From the doctest docs.)

Tables

(details)

There are two syntaxes for tables in reStructuredText. Grid tables are complete but cumbersome to create. Simple tables are easy to create but limited (no row spans, etc.).

Plain text Typical result

Grid table:

+------------+------------+-----------+
| Header 1   | Header 2   | Header 3  |
+============+============+===========+
| body row 1 | column 2   | column 3  |
+------------+------------+-----------+
| body row 2 | Cells may span columns.|
+------------+------------+-----------+
| body row 3 | Cells may  | - Cells   |
+------------+ span rows. | - contain |
| body row 4 |            | - blocks. |
+------------+------------+-----------+

Grid table:

Header 1 | Header 2 | Header 3
body row 1 | column 2 | column 3
body row 2 | Cells may span columns.
body row 3 Cells may
span rows.
  • Cells
  • contain
  • blocks.
body row 4

Simple table:

=====  =====  ======
   Inputs     Output
------------  ------
  A      B    A or B
=====  =====  ======
False  False  False
True   False  True
False  True   True
True   True   True
=====  =====  ======

Simple table:

Inputs |  | Output
A | B | A or B
False | False | False
True | False | True
False | True | True
True | True | True

Transitions

(details)

Plain text Typical result

A transition marker is a horizontal line
of 4 or more repeated punctuation
characters.

------------

A transition should not begin or end a
section or document, nor should two
transitions be immediately adjacent.

A transition marker is a horizontal line of 4 or more repeated punctuation characters.

A transition should not begin or end a section or document, nor should two transitions be immediately adjacent.

Transitions are commonly seen in novels and short fiction, as a gap spanning one or more lines, marking text divisions or signaling changes in subject, time, point of view, or emphasis.

Explicit Markup

Explicit markup blocks are used for constructs which float (footnotes), have no direct paper-document representation (hyperlink targets, comments), or require specialized processing (directives). They all begin with two periods and whitespace, the "explicit markup start".

Footnotes

(details)

Plain text Typical result
Footnote references, like [5]_.
Note that footnotes may get
rearranged, e.g., to the bottom of
the "page".

.. [5] A numerical footnote. Note
   there's no colon after the ``]``.

Footnote references, like 5. Note that footnotes may get rearranged, e.g., to the bottom of the "page".

[5] A numerical footnote. Note there's no colon after the ].
Autonumbered footnotes are
possible, like using [#]_ and [#]_.

.. [#] This is the first one.
.. [#] This is the second one.

They may be assigned 'autonumber
labels' - for instance,
[#fourth]_ and [#third]_.

.. [#third] a.k.a. third_

.. [#fourth] a.k.a. fourth_

Autonumbered footnotes are possible, like using 1 and 2.

They may be assigned 'autonumber labels' - for instance, 4 and 3.

[1] | This is the first one.
[2] | This is the second one.
[3] | a.k.a. third
[4] | a.k.a. fourth
Auto-symbol footnotes are also
possible, like this: [*]_ and [*]_.

.. [*] This is the first one.
.. [*] This is the second one.

Auto-symbol footnotes are also possible, like this: * and †.

[*] | This is the first symbol footnote
[†] | This is the second one.

The numbering of auto-numbered footnotes is determined by the order of the footnotes, not of the references. For auto-numbered footnote references without autonumber labels ("[#]_"), the references and footnotes must be in the same relative order. Similarly for auto-symbol footnotes ("[*]_").

Citations

(details)

Plain text Typical result
Citation references, like [CIT2002]_.
Note that citations may get
rearranged, e.g., to the bottom of
the "page".

.. [CIT2002] A citation
   (as often used in journals).

Citation labels contain alphanumerics,
underlines, hyphens and fullstops.
Case is not significant.

Given a citation like [this]_, one
can also refer to it like this_.

.. [this] here.

Citation references, like [CIT2002]. Note that citations may get rearranged, e.g., to the bottom of the "page".

Citation labels contain alphanumerics, underlines, hyphens and fullstops. Case is not significant.

Given a citation like [this], one can also refer to it like this.

[CIT2002] | A citation (as often used in journals).
[this] | here.

Hyperlink Targets

(details)

External Hyperlink Targets

Plain text Typical result
External hyperlinks, like Python_.

.. _Python: http://www.python.org/

Fold-in form
Indirect hyperlinks, like Python.
Call-out form
External hyperlinks, like Python.

Python: http://www.python.org/

"Fold-in" is the representation typically used in HTML documents (think of the indirect hyperlink being "folded in" like ingredients into a cake), and "call-out" is more suitable for printed documents, where the link needs to be presented explicitly, for example as a footnote.

Internal Hyperlink Targets

Plain text Typical result
Internal crossreferences, like example_.

.. _example:

This is an example crossreference target.

Fold-in form
Internal crossreferences, like example

This is an example crossreference target.

Call-out form
Internal crossreferences, like example

example:
This is an example crossreference target.

Indirect Hyperlink Targets

(details)

Plain text Typical result
Python_ is `my favourite
programming language`__.

.. _Python: http://www.python.org/

__ Python_

Python is my favourite programming language.

The second hyperlink target (the line beginning with "__") is both an indirect hyperlink target (indirectly pointing at the Python website via the "Python_" reference) and an anonymous hyperlink target. In the text, a double-underscore suffix is used to indicate an anonymous hyperlink reference. In an anonymous hyperlink target, the reference text is not repeated. This is useful for references with long text or throw-away references, but the target should be kept close to the reference to prevent them going out of sync.

Implicit Hyperlink Targets

(details)

Section titles, footnotes, and citations automatically generate hyperlink targets (the title text or footnote/citation label is used as the hyperlink name).

Plain text | Typical result
Titles are targets, too ======================= Implict references, like `Titles are targets, too`_. | Titles are targets, too Implict references, like Titles are targets, too.

Directives

(details)

Directives are a general-purpose extension mechanism, a way of adding support for new constructs without adding new syntax. For a description of all standard directives, see reStructuredText Directives.

Plain text | Typical result
For instance: .. image:: images/ball1.gif | For instance:

Substitution References and Definitions

(details)

Substitutions are like inline directives, allowing graphics and arbitrary constructs within text.

Plain text | Typical result
The |biohazard| symbol must be used on containers used to dispose of medical waste. .. |biohazard| image:: biohazard.png | The biohazard symbol must be used on containers used to dispose of medical waste.

Comments

(details)

Any text which begins with an explicit markup start but doesn't use the syntax of any of the constructs above, is a comment.

Plain text Typical result
.. This text will not be shown
   (but, for instance, in HTML might be
   rendered as an HTML comment)
An empty "comment" does not
"consume" following blocks.

..

        So this block is not "lost",
        despite its indentation.

An empty "comment" does not "consume" following blocks.
So this block is not "lost", despite its indentation.

Getting Help

Users who have questions or need assistance with Docutils or reStructuredText should post a message to the Docutils-Users mailing list. The Docutils project web site has more information.

Authors: Tibs (tibs@tibsnjoan.co.uk) and David Goodger (goodger@python.org)


The Docutils Document Tree

A Guide to the Docutils DTD

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.41
Date: | 2004-06-03
Copyright: | This document has been placed in the public domain.

Contents

  • Element Hierarchy
  • Element Reference
  • Attribute Reference
  • Parameter Entity Reference

This document describes the XML data structure of Docutils documents: the relationships and semantics of elements and attributes. The Docutils document structure is formally defined by the Docutils Generic DTD XML document type definition, docutils.dtd, which is the definitive source for details of element structural relationships.

This document does not discuss implementation details. Those can be found in internal documentation (docstrings) for the docutils.nodes module, where the document tree data structure is implemented in a class library.

The reader is assumed to have some familiarity with XML or SGML, and an understanding of the data structure meaning of "tree". For a list of introductory articles, see Introducing the Extensible Markup Language (XML).

The reStructuredText markup is used for illustrative examples throughout this document. For a gentle introduction, see A ReStructuredText Primer. For complete technical details, see the reStructuredText Markup Specification.

Element Hierarchy

  • Structural Elements
    • Structural Subelements
      • Bibliographic Elements
      • Decorative Elements
  • Body Elements
    • Simple Body Elements
    • Compound Body Elements
      • Body Subelements
  • Inline Elements

Below is a simplified diagram of the hierarchy of elements in the Docutils document tree structure. An element may contain any other elements immediately below it in the diagram. Notes are written in square brackets. Element types in parentheses indicate recursive or one-to-many relationships; sections may contain (sub)sections, tables contain further body elements, etc.

+--------------------------------------------------------------------+
| document  [may begin with a title, subtitle, docinfo, decoration]  |
|                             +--------------------------------------+
|                             | sections  [each begins with a title] |
+-----------------------------+-------------------------+------------+
| [body elements:]                                      | (sections) |
|         | - literal | - lists  |       | - hyperlink  +------------+
|         |   blocks  | - tables |       |   targets    |
| para-   | - doctest | - block  | foot- | - sub. defs  |
| graphs  |   blocks  |   quotes | notes | - comments   |
+---------+-----------+----------+-------+--------------+
| [text]+ | [text]    | (body elements)  | [text]       |
| (inline +-----------+------------------+--------------+
| markup) |
+---------+

The Docutils document model uses a simple, recursive model for section structure. A document node may contain body elements and section elements. Sections in turn may contain body elements and sections. The level (depth) of a section element is determined from its physical nesting level; unlike other document models (<h1> in HTML, <sect1> in DocBook, <div1> in XMLSpec) the level is not incorporated into the element name.

The Docutils document model uses strict element content models. Every element has a unique structure and semantics, but elements may be classified into general categories (below). Only elements which are meant to directly contain text data have a mixed content model, where text data and inline elements may be intermixed. This is unlike the much looser HTML document model, where paragraphs and text data may occur at the same level.

Structural Elements

Structural elements may only contain child elements; they do not directly contain text data. Structural elements may contain body elements or further structural elements. Structural elements can only be child elements of other structural elements.

Category members: document, section, topic, sidebar

Structural Subelements

Structural subelements are child elements of structural elements. Simple structuctural subelements (title, subtitle) contain text data; the others are compound and do not directly contain text data.

Category members: title, subtitle, docinfo, decoration, transition

Bibliographic Elements

The docinfo element is an optional child of document. It groups bibliographic elements together. All bibliographic elements except authors and field contain text data. authors contains further bibliographic elements (most notably author). field contains field_name and field_body body subelements.

Category members: address, author, authors, contact, copyright, date, field, organization, revision, status, version

Decorative Elements

The decoration element is also an optional child of document. It groups together elements used to generate page headers and footers.

Category members: footer, header

Body Elements

Body elements are contained within structural elements and compound body elements. There are two subcategories of body elements: simple and compound.

Category members: admonition, attention, block_quote, bullet_list, caution, citation, comment, danger, definition_list, doctest_block, enumerated_list, error, field_list, figure, footnote, hint, image, important, line_block, literal_block, note, option_list, paragraph, pending, raw, rubric, substitution_definition, system_message, table, target, tip, warning

Simple Body Elements

Simple body elements directly are empty or contain text data. Those that contain text data may also contain inline elements. Such elements therefore have a "mixed content model".

Category members: comment, doctest_block, image, line_block, literal_block, paragraph, pending, raw, rubric, substitution_definition, target

Compound Body Elements

Compound body elements contain local substructure (body subelements) and further body elements. They do not directly contain text data.

Category members: admonition, attention, block_quote, bullet_list, caution, citation, danger, definition_list, enumerated_list, error, field_list, figure, footnote, hint, important, note, option_list, system_message, table, tip, warning

Body Subelements

Compound body elements contain specific subelements (e.g. bullet_list contains list_item). Subelements may themselves be compound elements (containing further child elements, like field) or simple data elements (containing text data, like field_name). These subelements always occur within specific parent elements, never at the body element level (beside paragraphs, etc.).

Category members (simple): attribution, caption, classifier, colspec, field_name, label, option_argument, option_string, term

Category members (compound): definition, definition_list_item, description, entry, field, field_body, legend, list_item, option, option_group, option_list_item, row, tbody, tgroup, thead

Inline Elements

Inline elements directly contain text data, and may also contain further inline elements. Inline elements are contained within simple body elements. Most inline elements have a "mixed content model".

Category members: abbreviation, acronym, citation_reference, emphasis, footnote_reference, generated, image, inline, literal, problematic, reference, strong, subscript, substitution_reference, superscript, target, title_reference, raw

Element Reference

  • abbreviation
  • acronym
  • address
  • admonition
  • attention
  • attribution
  • author
  • authors
  • block_quote
  • bullet_list
  • caption
  • caution
  • citation
  • citation_reference
  • classifier
  • colspec
  • comment
  • contact
  • copyright
  • danger
  • date
  • decoration
  • definition
  • definition_list
  • definition_list_item
  • description
  • docinfo
  • doctest_block
  • document
  • emphasis
  • entry
  • enumerated_list
  • error
  • field
  • field_body
  • field_list
  • field_name
  • figure
  • footer
  • footnote
  • footnote_reference
  • generated
  • header
  • hint
  • image
  • important
  • inline
  • label
  • legend
  • line_block
  • list_item
  • literal
  • literal_block
  • note
  • option
  • option_argument
  • option_group
  • option_list
  • option_list_item
  • option_string
  • organization
  • paragraph
  • pending
  • problematic
  • raw
  • reference
  • revision
  • row
  • rubric
  • section
  • sidebar
  • status
  • strong
  • subscript
  • substitution_definition
  • substitution_reference
  • subtitle
  • superscript
  • system_message
  • table
  • target
  • tbody
  • term
  • tgroup
  • thead
  • tip
  • title
  • title_reference
  • topic
  • transition
  • version
  • warning

Each element in the DTD (document type definition) is described in its own section below. Each section contains an introduction plus the following subsections:

  • Details (of element relationships and semantics):

    • Category: One or more references to the element categories in Element Hierarchy above. Some elements belong to more than one category.
    • Parents: A list of elements which may contain the element.
    • Children: A list of elements which may occur within the element.
    • Analogues: Describes analogous elements in well-known document models such as HTML or DocBook. Lists similarities and differences.
    • Processing: Lists formatting or rendering recommendations for the element.

  • Content Model:

    The formal XML content model from the Docutils DTD, followed by:

    • Attributes: Describes (or refers to descriptions of) the possible values and semantics of each attribute.
    • Parameter Entities: Lists the parameter entities which directly or indirectly include the element.

  • Examples: reStructuredText examples are shown along with fragments of the document trees resulting from parsing. Pseudo-XML is used for the results of parsing and processing. Pseudo-XML is a representation of XML where nesting is indicated by indentation and end-tags are not shown. Some of the precision of real XML is given up in exchange for easier readability. For example, the following are equivalent:

    • Real XML:

      <document>
<section id="a-title" name="a title">
<title>A Title</title>
<paragraph>A paragraph.</paragraph>
</section>
</document>

    • Pseudo-XML:

      <document>
    <section id="a-title" name="a title">
        <title>
            A Title
        <paragraph>
            A paragraph.

Many of the element reference sections below are marked "to be completed". Please help complete this document by contributing to its writing.

abbreviation

To be completed.

acronym

To be completed.

address

The address element holds the surface mailing address information for the author (individual or group) of the document, or a third-party contact address. Its structure is identical to that of the line_block element: whitespace is significant, especially newlines.

Details

Category: | Bibliographic Elements
Parents: | The following elements may contain address: docinfo, authors
Children: | address elements contain text data plus inline elements.
Analogues: | address is analogous to the DocBook "address" element.
Processing: | As with the line_block element, newlines and other whitespace is significant and must be preserved. However, a monospaced typeface need not be used. See also docinfo.

Content Model

%text.model;

Attributes: | The address element contains the common attributes (id, name, dupname, source, and class), plus xml:space.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes address.

Examples

reStructuredText source:

Document Title
==============

:Address: 123 Example Ave.
          Example, EX

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <address>
            123 Example Ave.
            Example, EX

See docinfo for a more complete example, including processing context.

admonition

This element is a generic, titled admonition. Also see the specific admonition elements Docutils offers (in alphabetical order): caution, danger, error, hint, important, note, tip, warning.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain admonition.
Children: | admonition elements begin with a title and may contain one or more body elements.
Analogues: | admonition has no direct analogues in common DTDs. It can be emulated with primitives and type effects.
Processing: | Rendered distinctly (inset and/or in a box, etc.).

Content Model

(title, (%body.elements;)+)

Attributes: | The admonition element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes admonition. The %structure.model; parameter entity indirectly includes admonition.

Examples

reStructuredText source:

.. admonition:: And, by the way...

   You can make up your own admonition too.

Pseudo-XML fragment from simple parsing:

<admonition class="admonition-and-by-the-way">
    <title>
        And, by the way...
    <paragraph>
        You can make up your own admonition too.

attention

The attention element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): caution, danger, error, hint, important, note, tip, warning, and the generic admonition.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain attention.
Children: | attention elements contain one or more body elements.
Analogues: | attention has no direct analogues in common DTDs. It can be emulated with primitives and type effects.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "Attention!" (or similar).

Content Model

(%body.elements;)+

Attributes: | The attention element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes attention. The %structure.model; parameter entity indirectly includes attention.

Examples

reStructuredText source:

.. Attention:: All your base are belong to us.

Pseudo-XML fragment from simple parsing:

<attention>
    <paragraph>
        All your base are belong to us.

attribution

To be completed.

author

The author element holds the name of the author of the document.

Details

Category: | Bibliographic Elements
Parents: | The following elements may contain author: docinfo, authors
Children: | author elements may contain text data plus inline elements.
Analogues: | author is analogous to the DocBook "author" element.
Processing: | See docinfo.

Content Model

%text.model;

Attributes: | The author element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes author.

Examples

reStructuredText source:

Document Title
==============

:Author: J. Random Hacker

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <author>
            J. Random Hacker

See docinfo for a more complete example, including processing context.

authors

The authors element is a container for author information for documents with multiple authors.

Details

Category: | Bibliographic Elements
Parents: | Only the docinfo element contains authors.
Children: | authors elements may contain the following elements: author, organization, address, contact
Analogues: | authors is analogous to the DocBook "authors" element.
Processing: | See docinfo.

Content Model

((author, organization?, address?, contact?)+)

Attributes: | The authors element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes authors.

Examples

reStructuredText source:

Document Title
==============

:Authors: J. Random Hacker; Jane Doe

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <authors>
            <author>
                J. Random Hacker
            <author>
                Jane Doe

In reStructuredText, multiple author's names are separated with semicolons (";") or commas (","); semicolons take precedence. There is currently no way to represent the author's organization, address, or contact in a reStructuredText "Authors" field.

See docinfo for a more complete example, including processing context.

block_quote

The block_quote element is used for quotations set off from the main text (standalone).

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain block_quote.
Children: | block_quote elements contain body elements followed by an optional attribution element.
Analogues: | block_quote is analogous to the "blockquote" element in both HTML and DocBook.
Processing: | block_quote elements serve to set their contents off from the main text, typically with indentation and/or other decoration.

Content Model

((%body.elements;)+, attribution?)

Attributes: | The block_quote element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes block_quote. The %structure.model; parameter entity indirectly includes block_quote.

Examples

reStructuredText source:

As a great paleontologist once said,

    This theory, that is mine, is mine.

    -- Anne Elk (Miss)

Pseudo-XML fragment from simple parsing:

<paragraph>
    As a great paleontologist once said,
<block_quote>
    <paragraph>
        This theory, that is mine, is mine.
    <attribution>
        Anne Elk (Miss)

bullet_list

The bullet_list element contains list_item elements which are uniformly marked with bullets. Bullets are typically simple dingbats (symbols) such as circles and squares.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain bullet_list.
Children: | bullet_list elements contain one or more list_item elements.
Analogues: | bullet_list is analogous to the HTML "ul" element and to the DocBook "itemizedlist" element. HTML's "ul" is short for "unordered list", which we consider to be a misnomer. "Unordered" implies that the list items may be randomly rearranged without affecting the meaning of the list. Bullet lists are often ordered; the ordering is simply left implicit.
Processing: | Each list item should begin a new vertical block, prefaced by a bullet/dingbat.

Content Model

(list_item +)

Attributes: | The bullet_list element contains the common attributes (id, name, dupname, source, and class), plus bullet. bullet is used to record the style of bullet from the input data. In documents processed from reStructuredText, it contains one of "-", "+", or "*". It may be ignored in processing.
Parameter Entities:
 | The %body.elements; parameter entity directly includes bullet_list. The %structure.model; parameter entity indirectly includes bullet_list.

Examples

reStructuredText source:

- Item 1, paragraph 1.

  Item 1, paragraph 2.

- Item 2.

Pseudo-XML fragment from simple parsing:

<bullet_list bullet="-">
    <list_item>
        <paragraph>
            Item 1, paragraph 1.
        <paragraph>
            Item 1, paragraph 2.
    <list_item>
        <paragraph>
            Item 2.

See list_item for another example.

caption

To be completed.

caution

The caution element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention, danger, error, hint, important, note, tip, warning, and the generic admonition.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain caution.
Children: | caution elements contain one or more body elements.
Analogues: | caution is analogous to the DocBook "caution" element.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "Caution" (or similar).

Content Model

(%body.elements;)+

Attributes: | The caution element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes caution. The %structure.model; parameter entity indirectly includes caution.

Examples

reStructuredText source:

.. Caution:: Don't take any wooden nickels.

Pseudo-XML fragment from simple parsing:

<caution>
    <paragraph>
        Don't take any wooden nickels.

citation

To be completed.

citation_reference

To be completed.

classifier

The classifier element contains the classification or type of the term being defined in a definition_list. For example, it can be used to indicate the type of a variable.

Details

Category: | Body Subelements (simple)
Parents: | Only the definition_list_item element contains classifier.
Children: | classifier elements may contain text data plus inline elements.
Analogues: | classifier has no direct analogues in common DTDs. It can be emulated with primitives or type effects.
Processing: | See definition_list_item.

Content Model

%text.model;
Attributes: The classifier element contains only the common attributes: id, name, dupname, source, and class.

Examples

Here is a hypothetical data dictionary. reStructuredText source:

name : string
    Customer name.
i : int
    Temporary index variable.

Pseudo-XML fragment from simple parsing:

<definition_list>
    <definition_list_item>
        <term>
            name
        <classifier>
            string
        <definition>
            <paragraph>
                Customer name.
    <definition_list_item>
        <term>
            i
        <classifier>
            int
        <definition>
            <paragraph>
                Temporary index variable.

colspec

To be completed.

comment

To be completed.

contact

The contact element holds contact information for the author (individual or group) of the document, or a third-party contact. It is typically used for an email or web address.

Details

Category: | Bibliographic Elements
Parents: | The following elements may contain contact: docinfo, authors
Children: | contact elements may contain text data plus inline elements.
Analogues: | contact is analogous to the DocBook "email" element. The HTML "address" element serves a similar purpose.
Processing: | See docinfo.

Content Model

%text.model;

Attributes: | The contact element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes contact.

Examples

reStructuredText source:

Document Title
==============

:Contact: jrh@example.com

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <contact>
            <reference refuri="mailto:jrh@example.com">
                jrh@example.com

See docinfo for a more complete example, including processing context.

copyright

The copyright element contains the document's copyright statement.

Details

Category: | Bibliographic Elements
Parents: | Only the docinfo element contains copyright.
Children: | copyright elements may contain text data plus inline elements.
Analogues: | copyright is analogous to the DocBook "copyright" element.
Processing: | See docinfo.

Content Model

%text.model;

Attributes: | The copyright element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes copyright.

Examples

reStructuredText source:

Document Title
==============

:Copyright: This document has been placed in the public domain.

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <copyright>
            This document has been placed in the public domain.

See docinfo for a more complete example, including processing context.

danger

The danger element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention, caution, error, hint, important, note, tip, warning, and the generic admonition.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain danger.
Children: | danger elements contain one or more body elements.
Analogues: | danger has no direct analogues in common DTDs. It can be emulated with primitives and type effects.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "!DANGER!" (or similar).

Content Model

(%body.elements;)+

Attributes: | The danger element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes danger. The %structure.model; parameter entity indirectly includes danger.

Examples

reStructuredText source:

.. DANGER:: Mad scientist at work!

Pseudo-XML fragment from simple parsing:

<danger>
    <paragraph>
        Mad scientist at work!

date

The date element contains the date of publication, release, or last modification of the document.

Details

Category: | Bibliographic Elements
Parents: | Only the docinfo element contains date.
Children: | date elements may contain text data plus inline elements.
Analogues: | date is analogous to the DocBook "date" element.
Processing: | Often used with the RCS/CVS keyword "Date". See docinfo.

Content Model

%text.model;

Attributes: | The date element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes date.

Examples

reStructuredText source:

Document Title
==============

:Date: 2002-08-20

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <date>
            2002-08-20

See docinfo for a more complete example, including processing context.

decoration

The decoration element is a container for header and footer elements and potential future extensions. These elements are used for page navigation, notes, time/datestamp, etc. Currently only the footer element is implemented, populated with processing information (datestamp, a link to Docutils, etc.).

Details

Category: | Structural Subelements
Parents: | Only the document element contains decoration.
Children: | decoration elements may contain decorative elements.
Analogues: | There are no direct analogies to decoration in HTML or in DocBook. Equivalents are typically constructed from primitives and/or generated by the processing system.
Processing: | See the individual decorative elements.

Content Model

(header?, footer?)

Although the content model doesn't specifically require contents, no empty decoration elements are ever created.

Attributes: The decoration element contains only the common attributes: id, name, dupname, source, and class.

Examples

reStructuredText source:

A paragraph.

Complete pseudo-XML result after parsing and applying transforms, assuming that the datestamp command-line option or configuration setting has been supplied:

<document>
    <decoration>
        <footer>
            <paragraph>
                Generated on: 2002-08-20.
    <paragraph>
        A paragraph.

definition

The definition element is a container for the body elements used to define a term in a definition_list.

Details

Category: | Body Subelements (compound)
Parents: | Only definition_list_item elements contain definition.
Children: | definition elements may contain body elements.
Analogues: | definition is analogous to the HTML "dd" element and to the DocBook "listitem" element (inside a "variablelistentry" element).
Processing: | See definition_list_item.

Content Model

(%body.elements;)+
Attributes: The definition element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the definition_list, definition_list_item, and classifier elements.

definition_list

The definition_list element contains a list of terms and their definitions. It can be used for glossaries or dictionaries, to describe or classify things, for dialogues, or to itemize subtopics (such as in this reference).

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain definition_list.
Children: | definition_list elements contain one or more definition_list_item elements.
Analogues: | definition_list is analogous to the HTML "dl" element and to the DocBook "variablelist" element.
Processing: | See definition_list_item.

Content Model

(definition_list_item +)

Attributes: | The definition_list element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes definition_list. The %structure.model; parameter entity indirectly includes definition_list.

Examples

reStructuredText source:

Term
  Definition.

Term : classifier
    The ' : ' indicates a classifier in
    definition list item terms only.

Pseudo-XML fragment from simple parsing:

<definition_list>
    <definition_list_item>
        <term>
            Term
        <definition>
            <paragraph>
                Definition.
    <definition_list_item>
        <term>
            Term
        <classifier>
            classifier
        <definition>
            <paragraph>
                The ' : ' indicates a classifier in
                definition list item terms only.

See definition_list_item and classifier for further examples.

definition_list_item

The definition_list_item element contains a single term/definition pair (with optional classifier).

Details

Category: | Body Subelements (compound)
Parents: | Only the definition_list element contains definition_list_item.
Children: | definition_list_item elements each contain a single term, an optional classifier, and a definition.
Analogues: | definition_list_item is analogous to the DocBook "variablelistentry" element.
Processing: | The optional classifier can be rendered differently from the term. They should be separated visually, typically by spaces plus a colon or dash.

Content Model

(term, classifier?, definition)
Attributes: The definition_list_item element contains only the common attributes: id, name, dupname, source, and class.

Examples

reStructuredText source:

Tyrannosaurus Rex : carnivore
    Big and scary; the "Tyrant King".

Brontosaurus : herbivore
    All brontosauruses are thin at one end,
    much much thicker in the middle
    and then thin again at the far end.

    -- Anne Elk (Miss)

Pseudo-XML fragment from simple parsing:

<definition_list>
    <definition_list_item>
        <term>
            Tyrannosaurus Rex
        <classifier>
            carnivore
        <definition>
            <paragraph>
                Big and scary; the "Tyrant King".
    <definition_list_item>
        <term>
            Brontosaurus
        <classifier>
            herbivore
        <definition>
            <paragraph>
                All brontosauruses are thin at one end,
                much much thicker in the middle
                and then thin again at the far end.
            <paragraph>
                -- Anne Elk (Miss)

See definition_list and classifier for further examples.

description

The description element contains body elements, describing the purpose or effect of a command-line option or group of options.

Details

Category: | Body Subelements
Parents: | Only the option_list_item element contains description.
Children: | description elements may contain body elements.
Analogues: | description has no direct analogues in common DTDs.
Processing: | See option_list.

Content Model

(%body.elements;)+
Attributes: The description element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the option_list element.

docinfo

The docinfo element is a container for document bibliographic data, or meta-data (data about the document). It corresponds to the front matter of a book, such as the title page and copyright page.

Details

Category: Structural Subelements
Parents: Only the document element contains docinfo.
Children: docinfo elements contain bibliographic elements.
Analogues: docinfo is analogous to DocBook "info" elements ("bookinfo" etc.). There are no directly analogous HTML elements; the "meta" element carries some of the same information, albeit invisibly.
Processing:

The docinfo element may be rendered as a two-column table or in other styles. It may even be invisible or omitted from the processed output. Meta-data may be extracted from docinfo children; for example, HTML <meta> tags may be constructed.

When Docutils transforms a reStructuredText field_list into a docinfo element (see the examples below), RCS/CVS keywords are normally stripped from simple (one paragraph) field bodies. For complete details, please see RCS Keywords in the reStructuredText Markup Specification.

Content Model

(%bibliographic.elements;)+
Attributes: The docinfo element contains only the common attributes: id, name, dupname, source, and class.

Examples

Docinfo is represented in reStructuredText by a field_list in a bibliographic context: the first non-comment element of a document, after any document title/subtitle. The field list is transformed into a docinfo element and its children by a transform. Source:

Docinfo Example
===============

:Author: J. Random Hacker
:Contact: jrh@example.com
:Date: 2002-08-18
:Status: Work In Progress
:Version: 1
:Filename: $RCSfile: doctree.txt,v $
:Copyright: This document has been placed in the public domain.

Complete pseudo-XML result after parsing and applying transforms:

<document id="docinfo-example" name="docinfo example">
    <title>
        Docinfo Example
    <docinfo>
        <author>
            J. Random Hacker
        <contact>
            <reference refuri="mailto:jrh@example.com">
                jrh@example.com
        <date>
            2002-08-18
        <status>
            Work In Progress
        <version>
            1
        <field>
            <field_name>
                Filename
            <field_body>
                <paragraph>
                    doctree.txt
        <copyright>
            This document has been placed in the public domain.

Note that "Filename" is a non-standard docinfo field, so becomes a generic field element. Also note that the "RCSfile" keyword syntax has been stripped from the "Filename" data.

See field_list for an example in a non-bibliographic context. Also see the individual examples for the various bibliographic elements.

doctest_block

The doctest_block element is a Python-specific variant of literal_block. It is a block of text where line breaks and whitespace are significant and must be preserved. doctest_block elements are used for interactive Python interpreter sessions, which are distinguished by their input prompt: >>>. They are meant to illustrate usage by example, and provide an elegant and powerful testing environment via the doctest module in the Python standard library.

Details

Category: | Simple Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain doctest_block.
Children: | doctest_block elements may contain text data plus inline elements.
Analogues: | doctest_block is analogous to the HTML "pre" element and to the DocBook "programlisting" and "screen" elements.
Processing: | As with literal_block, doctest_block elements are typically rendered in a monospaced typeface. It is crucial that all whitespace and line breaks are preserved in the rendered form.

Content Model

%text.model;

Attributes: | The doctest_block element contains the common attributes (id, name, dupname, source, and class), plus xml:space.
Parameter Entities:
 | The %body.elements; parameter entity directly includes doctest_block. The %structure.model; parameter entity indirectly includes doctest_block.

Examples

reStructuredText source:

This is an ordinary paragraph.

>>> print 'this is a Doctest block'
this is a Doctest block

Pseudo-XML fragment from simple parsing:

<paragraph>
    This is an ordinary paragraph.
<doctest_block xml:space="preserve">
    >>> print 'this is a Doctest block'
    this is a Doctest block

document

The document element is the root (topmost) element of the Docutils document tree. document is the direct or indirect ancestor of every other element in the tree. It encloses the entire document tree. It is the starting point for a document.

Details

Category: | Structural Elements
Parents: | The document element has no parents.
Children: | document elements may contain structural subelements, structural elements, and body elements.
Analogues: | document is analogous to the HTML "html" element and to several DocBook elements such as "book".

Content Model

((title,
  subtitle?)?,
 docinfo?,
 decoration?,
 %structure.model;)

Depending on the source of the data and the stage of processing, the "document" may not initially contain a "title". A document title is not directly representable in reStructuredText. Instead, a lone top-level section may have its title promoted to become the document title, and similarly for a lone second-level (sub)section's title to become the document subtitle. The "docinfo" may be transformed from an initial field_list, and "decoration" is usually constructed programmatically.

See the %structure.model; parameter entity for details of the body of a document.

Attributes: The document element contains only the common attributes: id, name, dupname, source, and class.

Examples

reStructuredText source:

A Title
=======

A paragraph.

Complete pseudo-XML result from simple parsing:

<document>
    <section id="a-title" name="a title">
        <title>
            A Title
        <paragraph>
            A paragraph.

After applying transforms, the section title is promoted to become the document title:

<document id="a-title" name="a title">
    <title>
        A Title
    <paragraph>
        A paragraph.

emphasis

To be completed.

entry

To be completed.

enumerated_list

The enumerated_list element contains list_item elements which are uniformly marked with enumerator labels.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain enumerated_list.
Children: | enumerated_list elements contain one or more list_item elements.
Analogues: | enumerated_list is analogous to the HTML "ol" element and to the DocBook "orderedlist" element.
Processing: | Each list item should begin a new vertical block, prefaced by a enumeration marker (such as "1.").

Content Model

(list_item +)
Attributes:

The enumerated_list element contains the common attributes (id, name, dupname, source, and class), plus enumtype, prefix, suffix, and start.

enumtype is used to record the intended enumeration sequence, one of "arabic" (1, 2, 3, ...), "loweralpha" (a, b, c, ..., z), "upperalpha" (A, B, C, ..., Z), "lowerroman" (i, ii, iii, iv, ..., mmmmcmxcix [4999]), or "upperroman" (I, II, III, IV, ..., MMMMCMXCIX [4999]).

prefix stores the formatting characters used before the enumerator. In documents originating from reStructuredText data, it will contain either "" (empty string) or "(" (left parenthesis). It may or may not affect processing.

suffix stores the formatting characters used after the enumerator. In documents originating from reStructuredText data, it will contain either "." (period) or ")" (right parenthesis). Depending on the capabilities of the output format, this attribute may or may not affect processing.

start contains the ordinal value of the first item in the list, in decimal. For lists beginning at value 1 ("1", "a", "A", "i", or "I"), this attribute may be omitted.
Parameter Entities:
The %body.elements; parameter entity directly includes enumerated_list. The %structure.model; parameter entity indirectly includes enumerated_list.

Examples

reStructuredText source:

1. Item 1.

   (A) Item A.
   (B) Item B.
   (C) Item C.

2. Item 2.

Pseudo-XML fragment from simple parsing:

<enumerated_list enumtype="arabic" prefix="" suffix=".">
    <list_item>
        <paragraph>
            Item 1.
        <enumerated_list enumtype="upperalpha" prefix="(" suffix=")">
            <list_item>
                <paragraph>
                    Item A.
            <list_item>
                <paragraph>
                    Item B.
            <list_item>
                <paragraph>
                    Item C.
    <list_item>
        <paragraph>
            Item 2.

See list_item for another example.

error

The error element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention, caution, danger, hint, important, note, tip, warning, and the generic admonition.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain error.
Children: | error elements contain one or more body elements.
Analogues: | error has no direct analogues in common DTDs. It can be emulated with primitives and type effects.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "Error" (or similar).

Content Model

(%body.elements;)+

Attributes: | The error element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes error. The %structure.model; parameter entity indirectly includes error.

Examples

reStructuredText source:

.. Error:: Does not compute.

Pseudo-XML fragment from simple parsing:

<error>
    <paragraph>
        Does not compute.

field

The field element contains a pair of field_name and field_body elements.

Details

Category: | Body Subelements
Parents: | The following elements may contain field: docinfo, field_list
Children: | Each field element contains one field_name and one field_body element.
Analogues: | field has no direct analogues in common DTDs.
Processing: | See field_list.

Content Model

(field_name, field_body)

Attributes: | The field element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes field.

Examples

See the examples for the field_list and docinfo elements.

field_body

The field_body element contains body elements. It is analogous to a database field's data.

Details

Category: | Body Subelements
Parents: | Only the field element contains field_body.
Children: | field_body elements may contain body elements.
Analogues: | field_body has no direct analogues in common DTDs.
Processing: | See field_list.

Content Model

(%body.elements;)*
Attributes: The field_body element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the field_list and docinfo elements.

field_list

The field_list element contains two-column table-like structures resembling database records (label & data pairs). Field lists are often meant for further processing. In reStructuredText, field lists are used to represent bibliographic fields (contents of the docinfo element) and directive options.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain field_list.
Children: | field_list elements contain one or more field elements.
Analogues: | field_list has no direct analogues in common DTDs. It can be emulated with primitives such as tables.
Processing: | A field_list is typically rendered as a two-column list, where the first column contains "labels" (usually with a colon suffix). However, field lists are often used for extension syntax or special processing. Such structures do not survive as field lists to be rendered.

Content Model

(field +)

Attributes: | The field_list element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes field_list. The %structure.model; parameter entity indirectly includes field_list.

Examples

reStructuredText source:

:Author: Me
:Version: 1
:Date: 2001-08-11
:Parameter i: integer

Pseudo-XML fragment from simple parsing:

<field_list>
    <field>
        <field_name>
            Author
        <field_body>
            <paragraph>
                Me
    <field>
        <field_name>
            Version
        <field_body>
            <paragraph>
                1
    <field>
        <field_name>
            Date
        <field_body>
            <paragraph>
                2001-08-11
    <field>
        <field_name>
            Parameter i
        <field_body>
            <paragraph>
                integer

field_name

The field_name element contains text; it is analogous to a database field's name.

Details

Category: | Body Subelements (simple)
Parents: | Only the field element contains field_name.
Children: | field_name elements may contain text data plus inline elements.
Analogues: | field_name has no direct analogues in common DTDs.
Processing: | See field_list.

Content Model

%text.model;
Attributes: The field_name element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the field_list and docinfo elements.

figure

To be completed.

footer

The footer element is a container element whose contents are meant to appear at the bottom of a web page, or repeated at the bottom of every printed page. Currently the footer element may contain processing information (datestamp, a link to Docutils, etc.).

Details

Category: | Decorative Elements
Parents: | Only the decoration element contains footer.
Children: | footer elements may contain body elements.
Analogues: | There are no direct analogies to footer in HTML or DocBook. Equivalents are typically constructed from primitives and/or generated by the processing system.

Content Model

(%body.elements;)+
Attributes: The footer element contains only the common attributes: id, name, dupname, source, and class.

Examples

reStructuredText source:

A paragraph.

Complete pseudo-XML result after parsing and applying transforms, assuming that the datestamp command-line option or configuration setting has been supplied:

<document>
    <decoration>
        <footer>
            <paragraph>
                Generated on: 2002-08-20.
    <paragraph>
        A paragraph.

footnote

To be completed.

footnote_reference

To be completed.

generated

Docutils wraps generated elements around text that is inserted (generated) by Docutils; i.e., text that was not in the document, like section numbers inserted by the "sectnum" directive.

To be completed.

header

The header element is a container element whose contents are meant to appear at the top of a web page, or at the top of every printed page. Docutils does not yet make use of the header element.

Details

Category: | Decorative Elements
Parents: | Only the decoration element contains header.
Children: | header elements may contain body elements.
Analogues: | There are no direct analogies to header in HTML or DocBook. Equivalents are typically constructed from primitives and/or generated by the processing system.

Content Model

(%body.elements;)+
Attributes: The header element contains only the common attributes: id, name, dupname, source, and class.

Examples

None.

hint

The hint element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention, caution, danger, error, important, note, tip, warning, and the generic admonition.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain hint.
Children: | hint elements contain one or more body elements.
Analogues: | hint has no direct analogues in common DTDs. It can be emulated with primitives and type effects.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "Hint" (or similar).

Content Model

(%body.elements;)+

Attributes: | The hint element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes hint. The %structure.model; parameter entity indirectly includes hint.

Examples

reStructuredText source:

.. Hint:: It's bigger than a bread box.

Pseudo-XML fragment from simple parsing:

<hint>
    <paragraph>
        It's bigger than a bread box.

image

To be completed.

important

The important element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention, caution, danger, error, hint, note, tip, warning, and the generic admonition.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain important.
Children: | important elements contain one or more body elements.
Analogues: | important is analogous to the DocBook "important" element.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "Important" (or similar).

Content Model

(%body.elements;)+

Attributes: | The important element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes important. The %structure.model; parameter entity indirectly includes important.

Examples

reStructuredText source:

.. Important::

   * Wash behind your ears.
   * Clean up your room.
   * Back up your data.
   * Call your mother.

Pseudo-XML fragment from simple parsing:

<important>
    <bullet_list>
        <list_item>
            <paragraph>
                Wash behind your ears.
        <list_item>
            <paragraph>
                Clean up your room.
        <list_item>
            <paragraph>
                Back up your data.
        <list_item>
            <paragraph>
                Call your mother.

inline

To be completed.

label

To be completed.

legend

To be completed.

line_block

The line_block element contains a block of text where line breaks and whitespace are significant and must be preserved. line_block elements are commonly used for verse and addresses. See line_block for an alternative useful for program listings and interactive computer sessions.

Details

Category: | Simple Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain line_block.
Children: | line_block elements may contain text data plus inline elements.
Analogues: | line_block is analogous to the DocBook "literallayout" element and to the HTML "pre" element (with modifications to typeface styles).
Processing: | Unline literal_block, line_block elements are typically rendered in an ordinary text typeface. It is crucial that all whitespace and line breaks are preserved in the rendered form.

Content Model

%text.model;

Attributes: | The line_block element contains the common attributes (id, name, dupname, source, and class), plus xml:space.
Parameter Entities:
 | The %body.elements; parameter entity directly includes line_block. The %structure.model; parameter entity indirectly includes line_block.

Examples

reStructuredText uses a directive to indicate a line_block. Example source:

Take it away, Eric the Orchestra Leader!

.. line-block::

   A one, two, a one two three four

   Half a bee, philosophically,
       must, *ipso facto*, half not be.
   But half the bee has got to be,
       *vis a vis* its entity.  D'you see?

   But can a bee be said to be
       or not to be an entire bee,
           when half the bee is not a bee,
               due to some ancient injury?

   Singing...

Pseudo-XML fragment from simple parsing:

<paragraph>
    Take it away, Eric the Orchestra Leader!
<line_block xml:space="preserve">
    A one, two, a one two three four

    Half a bee, philosophically,
        must, <emphasis>ipso facto</emphasis>, half not be.
    But half the bee has got to be,
        <emphasis>vis a vis</emphasis>its entity.  D'you see?

    But can a bee be said to be
        or not to be an entire bee,
            when half the bee is not a bee,
                due to some ancient injury?

    Singing...

list_item

The list_item element is a container for the elements of a list item.

Details

Category: | Body Subelements (compound)
Parents: | The bullet_list and enumerated_list elements contain list_item.
Children: | list_item elements may contain body elements.
Analogues: | list_item is analogous to the HTML "li" element and to the DocBook "listitem" element.
Processing: | See bullet_list or enumerated_list.

Content Model

(%body.elements;)+
Attributes: The list_item element contains only the common attributes: id, name, dupname, source, and class.

Examples

reStructuredText source:

1. Outer list, item 1.

   * Inner list, item 1.
   * Inner list, item 2.

2. Outer list, item 2.

Pseudo-XML fragment from simple parsing:

<enumerated_list enumtype="arabic" prefix="" suffix=".">
    <list_item>
        <paragraph>
            Outer list, item 1.
        <bullet_list bullet="*">
            <list_item>
                <paragraph>
                    Inner list, item 1.
            <list_item>
                <paragraph>
                    Inner list, item 2.
    <list_item>
        <paragraph>
            Outer list, item 2.

See bullet_list or enumerated_list for further examples.

literal

To be completed.

literal_block

The literal_block element contains a block of text where line breaks and whitespace are significant and must be preserved. literal_block elements are commonly used for program listings and interactive computer sessions. See line_block for an alternative useful for verse and addresses.

Details

Category: | Simple Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain literal_block.
Children: | literal_block elements may contain text data plus inline elements.
Analogues: | literal_block is analogous to the HTML "pre" element and to the DocBook "programlisting" and "screen" elements.
Processing: | literal_block elements are typically rendered in a monospaced typeface. It is crucial that all whitespace and line breaks are preserved in the rendered form.

Content Model

%text.model;

Attributes: | The literal_block element contains the common attributes (id, name, dupname, source, and class), plus xml:space.
Parameter Entities:
 | The %body.elements; parameter entity directly includes literal_block. The %structure.model; parameter entity indirectly includes literal_block.

Examples

reStructuredText source:

Here is a literal block::

    if literal_block:
        text = 'is left as-is'
        spaces_and_linebreaks = 'are preserved'
        markup_processing = None

Pseudo-XML fragment from simple parsing:

<paragraph>
    Here is a literal block:
<literal_block xml:space="preserve">
    if literal_block:
        text = 'is left as-is'
        spaces_and_linebreaks = 'are preserved'
        markup_processing = None

note

The note element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention, caution, danger, error, hint, important, tip, warning, and the generic admonition.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain note.
Children: | note elements contain one or more body elements.
Analogues: | note is analogous to the DocBook "note" element.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "Note" (or similar).

Content Model

(%body.elements;)+

Attributes: | The note element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes note. The %structure.model; parameter entity indirectly includes note.

Examples

reStructuredText source:

.. Note:: Admonitions can be handy to break up a
   long boring technical document.

Pseudo-XML fragment from simple parsing:

<note>
    <paragraph>
        Admonitions can be handy to break up a
        long boring technical document.

option

The option element groups an option string together with zero or more option argument placeholders. Note that reStructuredText currently supports only one argument per option.

Details

Category: | Body Subelements
Parents: | Only the option_group element contains option.
Children: | Each option element contains one option_string and zero or more option_argument elements.
Analogues: | option has no direct analogues in common DTDs.
Processing: | See option_list.

Content Model

(option_string, option_argument *)
Attributes: The option element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the option_list element.

option_argument

The option_argument element contains placeholder text for option arguments.

Details

Category: | Body Subelements
Parents: | Only the option element contains option_argument.
Children: | option_argument elements contain text data only.
Analogues: | option_argument has no direct analogues in common DTDs.
Processing: | The value of the "delimiter" attribute is prefixed to the option_argument, separating it from its option_string or a preceding option_argument. The option_argument text is typically rendered in a monospaced typeface, possibly italicized or otherwise altered to indicate its placeholder nature.

Content Model

(#PCDATA)
Attributes:

The option_argument element contains the common attributes (id, name, dupname, source, and class), plus delimiter.

delimiter contains the text preceding the option_argument: either the text separating it from the option_string (typically either "=" or " ") or the text between option arguments (typically either "," or " ").

Examples

See the examples for the option_list element.

option_group

The option_group element groups together one or more option elements, all synonyms.

Details

Category: | Body Subelements
Parents: | Only the option_list_item element contains option_group.
Children: | option_group elements contain one or more option elements. option_group is an empty element and has no children. Each option_group element contains one _ and one _ element.
Analogues: | option_group has no direct analogues in common DTDs.
Processing: | Typically option elements within an option_group are joined together in a comma-separated list.

Content Model

(option_group, description)
Attributes: The option_group element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the option_list element.

option_list

Each option_list element contains a two-column list of command-line options and descriptions, documenting a program's options.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain option_list.
Children: | option_list elements contain one or more option_list_item elements.
Analogues: | option_list has no direct analogues in common DTDs. It can be emulated with primitives such as tables.
Processing: | An option_list is typically rendered as a two-column list, where the first column contains option strings and arguments, and the second column contains descriptions.

Content Model

(option_list_item +)

Attributes: | The option_list element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes option_list. The %structure.model; parameter entity indirectly includes option_list.

Examples

reStructuredText source:

-a            command-line option "a"
-1 file, --one=file, --two file
              Multiple options with arguments.

Pseudo-XML fragment from simple parsing:

<option_list>
    <option_list_item>
        <option_group>
            <option>
                <option_string>
                    -a
        <description>
            <paragraph>
                command-line option "a"
    <option_list_item>
        <option_group>
            <option>
                <option_string>
                    -1
                <option_argument delimiter=" ">
                    file
            <option>
                <option_string>
                    --one
                <option_argument delimiter="=">
                    file
            <option>
                <option_string>
                    --two
                <option_argument delimiter=" ">
                    file
        <description>
            <paragraph>
                Multiple options with arguments.

option_list_item

The option_list_item element is a container for a pair of option_group and description elements.

Details

Category: | Body Subelements
Parents: | Only the option_list element contains option_list_item.
Children: | Each option_list_item element contains one option_group and one description element.
Analogues: | option_list_item has no direct analogues in common DTDs.
Processing: | See option_list.

Content Model

(option_group, description)
Attributes: The option_list_item element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the option_list element.

option_string

The option_string element contains the text of a command-line option.

Details

Category: | Body Subelements
Parents: | Only the option element contains option_string.
Children: | option_string elements contain text data only.
Analogues: | option_string has no direct analogues in common DTDs.
Processing: | The option_string text is typically rendered in a monospaced typeface.

Content Model

(#PCDATA)
Attributes: The option_string element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the option_list element.

organization

The organization element contains the name of document author's organization, or the organization responsible for the document.

Details

Category: | Bibliographic Elements
Parents: | Only the docinfo element contains organization.
Children: | organization elements may contain text data plus inline elements.
Analogues: | organization is analogous to the DocBook "orgname", "corpname", or "publishername" elements.
Processing: | See docinfo.

Content Model

%text.model;

Attributes: | The organization element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes organization.

Examples

reStructuredText source:

Document Title
==============

:Organization: Humankind

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <organization>
            Humankind

See docinfo for a more complete example, including processing context.

paragraph

The paragraph element contains the text and inline elements of a single paragraph, a fundamental building block of documents.

Details

Category: | Simple Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain paragraph.
Children: | paragraph elements may contain text data plus inline elements.
Analogues: | paragraph is analogous to the HTML "p" element and to the DocBook "para" elements.

Content Model

%text.model;

Attributes: | The paragraph element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes paragraph. The %structure.model; parameter entity indirectly includes paragraph.

Examples

reStructuredText source:

A paragraph.

Pseudo-XML fragment from simple parsing:

<paragraph>
    A paragraph.

pending

To be completed.

problematic

To be completed.

raw

To be completed.

reference

To be completed.

revision

The revision element contains the revision number of the document. It can be used alone or in conjunction with version.

Details

Category: | Bibliographic Elements
Parents: | Only the docinfo element contains revision.
Children: | revision elements may contain text data plus inline elements.
Analogues: | revision is analogous to but simpler than the DocBook "revision" element. It closely matches the DocBook "revnumber" element, but in a simpler context.
Processing: | Often used with the RCS/CVS keyword "Revision". See docinfo.

Content Model

%text.model;

Attributes: | The revision element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes revision.

Examples

reStructuredText source:

Document Title
==============

:Version: 1
:Revision: b

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <version>
            1
        <revision>
            b

See docinfo for a more complete example, including processing context.

row

To be completed.

rubric

rubric n. 1. a title, heading, or the like, in a manuscript, book, statute, etc., written or printed in red or otherwise distinguished from the rest of the text. ...

—Random House Webster's College Dictionary, 1991

A rubric is like an informal heading that doesn't correspond to the document's structure.

To be completed.

section

The section element is the main unit of hierarchy for Docutils documents. Docutils section elements are a recursive structure; a section may contain other section elements, without limit. Paragraphs and other body elements may occur before a section, but not after it.

Details

Category: | Structural Elements
Parents: | The following elements may contain section: document, section
Children: | section elements begin with a title, and may contain body elements as well as transition, topic, and sidebar elements.
Analogues: | section is analogous to DocBook recursive "section" elements, and to HTML "div" elements combined with "h1" etc. title elements.

Content Model

(title,
 %structure.model;)

See the %structure.model; parameter entity for details of the body of a section.

Attributes: | The section element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %section.elements; parameter entity directly includes section. The %structure.model; parameter entity indirectly includes section.

Examples

reStructuredText source:

Title 1
=======
Paragraph 1.

Title 2
-------
Paragraph 2.

Title 3
=======
Paragraph 3.

Title 4
-------
Paragraph 4.

Complete pseudo-XML result after parsing:

<document>
    <section id="title-1" name="title 1">
        <title>
            Title 1
        <paragraph>
            Paragraph 1.
        <section id="title-2" name="title 2">
            <title>
                Title 2
            <paragraph>
                Paragraph 2.
    <section id="title-3" name="title 3">
        <title>
            Title 3
        <paragraph>
            Paragraph 3.
        <section id="title-4" name="title 4">
            <title>
                Title 4
            <paragraph>
                Paragraph 4.

sidebar

Sidebars are like miniature, parallel documents that occur inside other documents, providing related or reference material. A sidebar is typically offset by a border and "floats" to the side of the page; the document's main text may flow around it. Sidebars can also be likened to super-footnotes; their content is outside of the flow of the document's main text.

The sidebar element is a nonrecursive section-like construct which may occur at the top level of a section wherever a body element (list, table, etc.) is allowed. In other words, sidebar elements cannot nest inside body elements, so you can't have a sidebar inside a table or a list, or inside another sidebar (or topic).

Details

Category: | Structural Elements
Parents: | The following elements may contain sidebar: document, section
Children: | sidebar elements begin with a title and an optional subtitle and contain body elements.
Analogues: | sidebar is analogous to the DocBook "sidebar" element.
Processing: | A sidebar element should be set off from the rest of the document somehow, typically with a border. Sidebars typically "float" to the side of the page and the document's main text flows around them.

Content Model

(title, subtitle?,
 (%body.elements;)+)

Attributes: | The sidebar element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %structure.model; parameter entity directly includes sidebar.

Examples

The "sidebar" directive is used to create a sidebar element. reStructuredText source:

.. sidebar:: Title
   :subtitle: If Desired

   Body.

Pseudo-XML fragment from simple parsing:

<sidebar>
    <title>
        Title
    <subtitle>
        If Desired
    <paragraph>
        Body.

status

The status element contains a status statement for the document, such as "Draft", "Final", "Work In Progress", etc.

Details

Category: | Bibliographic Elements
Parents: | Only the docinfo element contains status.
Children: | status elements may contain text data plus inline elements.
Analogues: | status is analogous to the DocBook "status" element.
Processing: | See docinfo.

Content Model

%text.model;

Attributes: | The status element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes status.

Examples

reStructuredText source:

Document Title
==============

:Status: Work In Progress

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <status>
            Work In Progress

See docinfo for a more complete example, including processing context.

strong

To be completed.

subscript

To be completed.

substitution_definition

To be completed.

substitution_reference

To be completed.

subtitle

The subtitle element stores the subtitle of a document.

Details

Category: | Structural Subelements
Parents: | The document and sidebar elements may contain subtitle.
Children: | subtitle elements may contain text data plus inline elements.
Analogues: | subtitle is analogous to HTML header elements ("h2" etc.) and to the DocBook "subtitle" element.
Processing: | A document's subtitle is usually rendered smaller than its title.

Content Model

%text.model;
Attributes: The subtitle element contains only the common attributes: id, name, dupname, source, and class.

Examples

reStructuredText source:

=======
 Title
=======
----------
 Subtitle
----------

A paragraph.

Complete pseudo-XML result after parsing and applying transforms:

<document id="title" name="title">
    <title>
        Title
    <subtitle id="subtitle" name="subtitle">
        Subtitle
    <paragraph>
        A paragraph.

Note how two section levels have collapsed, promoting their titles to become the document's title and subtitle. Since there is only one structural element (document), the subsection's id and name attributes are stored in the subtitle element.

superscript

To be completed.

system_message

To be completed.

table

To be completed.

target

To be completed.

tbody

To be completed.

term

The term element contains a word or phrase being defined in a definition_list.

Details

Category: | Body Subelements (simple)
Parents: | Only the definition_list_item element contains term.
Children: | term elements may contain text data plus inline elements.
Analogues: | term is analogous to the HTML "dt" element and to the DocBook "term" element.
Processing: | See definition_list_item.

Content Model

%text.model;
Attributes: The term element contains only the common attributes: id, name, dupname, source, and class.

Examples

See the examples for the definition_list, definition_list_item, and classifier elements.

tgroup

To be completed.

thead

To be completed.

tip

The tip element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention, caution, danger, error, hint, important, note, warning, and the generic admonition.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain tip.
Children: | tip elements contain one or more body elements.
Analogues: | tip is analogous to the DocBook "tip" element.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "Tip" (or similar).

Content Model

(%body.elements;)+

Attributes: | The tip element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes tip. The %structure.model; parameter entity indirectly includes tip.

Examples

reStructuredText source:

.. Tip:: 15% if the service is good.

Pseudo-XML fragment from simple parsing:

<tip>
    <paragraph>
        15% if the service is good.

title

The title element stores the title of a document, section, topic, sidebar, or generic admonition.

Details

Category: | Structural Subelements
Parents: | The following elements may contain title: document, section, topic, sidebar, admonition
Children: | title elements may contain text data plus inline elements.
Analogues: | title is analogous to HTML "title" and header ("h1" etc.) elements, and to the DocBook "title" element.

Content Model

%text.model;
Attributes:

The title element contains the common attributes (id, name, dupname, source, and class), plus refid and auto.

refid is used as a backlink to a table of contents entry.

auto is used to indicate (with value "1") that the title has been numbered automatically.

Examples

reStructuredText source:

A Title
=======

A paragraph.

Pseudo-XML fragment from simple parsing:

<section id="a-title" name="a title">
    <title>
        A Title
    <paragraph>
        A paragraph.

title_reference

To be completed.

topic

The topic element is a nonrecursive section-like construct which may occur at the top level of a section wherever a body element (list, table, etc.) is allowed. In other words, topic elements cannot nest inside body elements, so you can't have a topic inside a table or a list, or inside another topic (or sidebar).

Details

Category: | Structural Elements
Parents: | The following elements may contain topic: document, section
Children: | topic elements begin with a title and may contain body elements.
Analogues: | topic is analogous to the DocBook "simplesect" element.
Processing: | A topic element should be set off from the rest of the document somehow, such as with indentation or a border.

Content Model

(title?,
 (%body.elements;)+)

Attributes: | The topic element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %structure.model; parameter entity directly includes topic.

Examples

The "topic" directive is used to create a topic element. reStructuredText source:

.. topic:: Title

   Body.

Pseudo-XML fragment from simple parsing:

<topic>
    <title>
        Title
    <paragraph>
        Body.

transition

The transition element is commonly seen in novels and short fiction, as a gap spanning one or more lines, with or without a type ornament such as a row of asterisks. Transitions separate other body elements, dividing a section into untitled divisions. A transition may not begin or end a section or document, nor may two transitions be immediately adjacent.

See Doctree Representation of Transitions in A Record of reStructuredText Syntax Alternatives.

Details

Category: | Structural Subelements
Parents: | The following elements may contain transition: document, section
Children: | transition is an empty element and has no children.
Analogues: | transition is analogous to the HTML "hr" element.
Processing: | The transition element is typically rendered as vertical whitespace (more than that separating paragraphs), with or without a horizontal line or row of asterisks. In novels, transitions are often represented as a row of three well-spaced asterisks with vertical space above and below.

Content Model

EMPTY

The transition element has no content; it is a "point element".

Attributes: | The transition element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %structure.model; parameter entity directly includes transition.

Examples

reStructuredText source:

Paragraph 1.

--------

Paragraph 2.

Complete pseudo-XML result after parsing:

<document>
    <paragraph>
        Paragraph 1.
    <transition>
    <paragraph>
        Paragraph 2.

version

The version element contains the version number of the document. It can be used alone or in conjunction with revision.

Details

Category: | Bibliographic Elements
Parents: | Only the docinfo element contains version.
Children: | version elements may contain text data plus inline elements.
Analogues: | version may be considered analogous to the DocBook "revision", "revnumber", or "biblioid" elements.
Processing: | Sometimes used with the RCS/CVS keyword "Revision". See docinfo and revision.

Content Model

%text.model;

Attributes: | The version element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %bibliographic.elements; parameter entity directly includes version.

Examples

reStructuredText source:

Document Title
==============

:Version: 1.1

Complete pseudo-XML result after parsing and applying transforms:

<document id="document-title" name="document title">
    <title>
        Document Title
    <docinfo>
        <version>
            1.1

See docinfo for a more complete example, including processing context.

warning

The warning element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention, caution, danger, error, hint, important, note, tip.

Details

Category: | Compound Body Elements
Parents: | All elements employing the %body.elements; or %structure.model; parameter entities in their content models may contain warning.
Children: | warning elements contain one or more body elements.
Analogues: | warning is analogous to the DocBook "warning" element.
Processing: | Rendered distinctly (inset and/or in a box, etc.), with the generated title "Warning" (or similar).

Content Model

(%body.elements;)+

Attributes: | The warning element contains only the common attributes: id, name, dupname, source, and class.
Parameter Entities:
 | The %body.elements; parameter entity directly includes warning. The %structure.model; parameter entity indirectly includes warning.

Examples

reStructuredText source:

.. WARNING:: Reader discretion is strongly advised.

Pseudo-XML fragment from simple parsing:

<warning>
    <paragraph>
        Reader discretion is strongly advised.

Attribute Reference

  • anonymous
  • auto
  • backrefs
  • bullet
  • class
  • delimiter
  • dupname
  • enumtype
  • id
  • name
  • prefix
  • refid
  • refname
  • refuri
  • source
  • start
  • suffix
  • xml:space

Common Attributes: Through the %basic.atts; parameter entity, all elements contain the following attributes: id, name, dupname, source, and class.

Attribute types:

CDATA
Character data. CDATA attributes may contain arbitrary text.
ID
Like a NMTOKEN, but it must begin with a letter (a "name production"). Identical ID values must not appear more than once in a document; i.e., ID values must uniquely identify their elements.
IDREF
A reference to an ID value (a name production) of another element.
IDREFS
One or more space-separated ID references (name productions).
NMTOKEN
A "name token". One or more of letters, digits, ".", "-", and "_".
NMTOKENS
One or more space-separated NMTOKEN names.
%yesorno;
No if zero ("0"), yes if any other value. This is a parameter entity which resolves to a NMTOKEN attribute type.
%number;
This emphasizes that the attribute value must be a number. This is a parameter entity which resolves to a NMTOKEN attribute type.
enumeration
The attribute value may be one of a specified list of values.

anonymous

Attribute type: %yesorno;. Default value: none (implies no).

The anonymous attribute is used for unnamed hyperlinks in the target and reference elements (via the %anonymous.att; parameter entity).

auto

Attribute type: CDATA. Default value: none.

The auto attribute is used to indicate automatically-numbered footnote, footnote_reference and title elements (via the %auto.att; parameter entity).

backrefs

Attribute type: IDREFS. Default value: none.

The backrefs attribute contains a space-separated list of id references, used for backlinks from footnote, citation, and system_message elements (via the %backrefs.att; parameter entity).

bullet

Attribute type: CDATA. Default value: none.

The bullet attribute is used in the bullet_list element.

class

Attribute type: NMTOKENS. Default value: none.

The class attribute contains one or more names used to classify an element. The purpose of the attribute is to indicate an "is-a" variant relationship, to allow an extensible way of defining sub-classes of existing elements. It can be used to carry context forward between a Docutils Reader and Writer, when a custom structure is reduced to a standardized document tree. One common use is in conjunction with stylesheets, to add selection criteria. It should not be used to carry formatting instructions or arbitrary content.

The class attribute's contents should be ignorable. Writers that are not familiar with the variant expressed should be able to ignore the attribute.

class is one of the common attributes, shared by all Docutils elements.

delimiter

Attribute type: CDATA. Default value: none.

The delimiter attribute is used in the option_argument element.

dupname

Attribute type: NMTOKENS. Default value: none.

The dupname attribute contains the name of an element when there has been a naming conflict. The contents of the dupname attribute would have been transferred from the name attribute. An element may have at most one of the name or dupname attributes, but not both. dupname is one of the common attributes, shared by all Docutils elements.

enumtype

Attribute type: enumeration, one of "arabic", "loweralpha", "upperalpha", "lowerroman", or "upperroman". Default value: none.

The enumtype attribute is used in the enumerated_list element.

id

Attribute type: ID. Default value: none.

The id attribute contains a unique identifier key. id is one of the common attributes, shared by all Docutils elements.

name

Attribute type: NMTOKENS. Default value: none.

The name attribute contains the name of an element, typically originating from the element's title or content. name must be unique; if there are name conflicts (two or more elements want to the same name), the contents will be transferred to the dupname attribute on the duplicate elements. An element may have at most one of the name or dupname attributes, but not both. name is one of the common attributes, shared by all Docutils elements.

prefix

Attribute type: CDATA. Default value: none.

The prefix attribute is used in the enumerated_list element.

refid

Attribute type: IDREF. Default value: none.

The refid attribute contains references to id attributes in other elements. It is used by the target, reference, footnote_reference, citation_reference, title and problematic elements (via the %refid.att; and %reference.atts; parameter entities).

refname

Attribute type: NMTOKENS. Default value: none.

The refname attribute contains an internal reference to the name attribute of another element. On a target element, refname indicates an indirect target which may resolve to either an internal or external reference. refname is used by the target, reference, footnote_reference, citation_reference, and substitution_reference elements (via the %refname.att; and %reference.atts; parameter entities).

refuri

Attribute type: CDATA. Default value: none.

The refuri attribute contains an external reference to a URI/URL. It is used by the target, reference, footnote_reference, and citation_reference elements (via the %reference.atts; parameter entity).

source

Attribute type: CDATA. Default value: none.

The source attribute is used to store the path or URL to the source text that was used to produce the document tree. It is one of the common attributes, shared by all Docutils elements.

start

Attribute type: %number;. Default value: none.

The start attribute is used in the enumerated_list element.

suffix

Attribute type: CDATA. Default value: none.

The suffix attribute is used in the enumerated_list element.

xml:space

Attribute type: one of "default" or "preserve". Default value: "preserve" (fixed).

The xml:space attribute is a standard XML attribute for whitespace-preserving elements. It is used by the literal_block, line_block, doctest_block, comment, and raw elements (via the %fixedspace.att; parameter entity). It is a fixed attribute, meant to communicate to an XML parser that the element contains significant whitespace. The attribute value should not be set in a document instance.

Parameter Entity Reference

  • %anonymous.att;
  • %auto.att;
  • %backrefs.att;
  • %basic.atts;
  • %bibliographic.elements;
  • %body.elements;
  • %fixedspace.att;
  • %inline.elements;
  • %reference.atts;
  • %refid.att;
  • %refname.att;
  • %refuri.att;
  • %section.elements;
  • %structure.model;
  • %text.model;

Parameter entities are used to simplify the DTD (to share definitions and reduce duplication) and to allow the DTD to be customized by wrapper DTDs (external client DTDs that use or import the Docutils DTD). Parameter entities may be overridden by wrapper DTDs, replacing the definitions below with custom definitions. Parameter entities whose names begin with "additional" are meant to allow easy extension by wrapper DTDs.

%anonymous.att;

The %anonymous.att; parameter entity contains the anonymous attribute, used for unnamed hyperlinks.

Entity definition:

anonymous %yesorno; #IMPLIED

The reference and target elements directly employ the %anonymous.att; parameter entity in their attribute lists.

%auto.att;

The %auto.att; parameter entity contains the auto attribute, used to indicate an automatically-numbered footnote or title.

Entity definition:

auto     CDATA     #IMPLIED

The footnote, footnote_reference, and title elements directly employ the %auto.att; parameter entity in their attribute lists.

%backrefs.att;

The %backrefs.att; parameter entity contains the backrefs attribute, a space-separated list of id references, for backlinks.

Entity definition:

backrefs  IDREFS    #IMPLIED

The citation, footnote, and system_message elements directly employ the %backrefs.att; parameter entity in their attribute lists.

%basic.atts;

The %basic.atts; parameter entity lists attributes common to all Docutils elements. See Common Attributes.

Entity definition:

id       ID        #IMPLIED
name     NMTOKENS  #IMPLIED
dupname  NMTOKENS  #IMPLIED
source   CDATA     #IMPLIED
class    NMTOKENS  #IMPLIED
%additional.basic.atts;

The %additional.basic.atts; parameter entity can be used by wrapper DTDs to extend %basic.atts;.

%bibliographic.elements;

The %bibliographic.elements; parameter entity contains an OR-list of all bibliographic elements.

Entity definition:

author | authors | organization | contact | address
| version | revision | status | date | copyright
| field
%additional.bibliographic.elements;

The %additional.bibliographic.elements; parameter entity can be used by wrapper DTDs to extend %bibliographic.elements;.

Only the docinfo element directly employs the %bibliographic.elements; parameter entity in its content model.

%body.elements;

The %body.elements; parameter entity contains an OR-list of all body elements. %body.elements; is itself contained within the %structure.model; parameter entity.

Entity definition:

paragraph | literal_block | doctest_block | line_block
| block_quote | table | figure | image | footnote | citation
| rubric
| bullet_list | enumerated_list | definition_list | field_list
| option_list
| attention | caution | danger | error | hint | important
| note | tip | warning | admonition
| target | substitution_definition | comment | pending
| system_message | raw
%additional.body.elements;

The %additional.body.elements; parameter entity can be used by wrapper DTDs to extend %body.elements;.

The %body.elements; parameter entity is directly employed in the content models of the following elements: admonition, attention, block_quote, caution, citation, danger, definition, description, entry, error, field_body, footer, footnote, header, hint, important, legend, list_item, note, sidebar, system_message, tip, topic, warning

Via %structure.model;, the %body.elements; parameter entity is indirectly employed in the content models of the document and section elements.

%fixedspace.att;

The %fixedspace.att; parameter entity contains the xml:space attribute, a standard XML attribute for whitespace-preserving elements.

Entity definition:

xml:space (default | preserve) #FIXED 'preserve'

The %fixedspace.att; parameter entity is directly employed in the attribute lists of the following elements: address, comment, doctest_block, line_block, literal_block, raw

%inline.elements;

The %inline.elements; parameter entity contains an OR-list of all inline elements.

Entity definition:

emphasis | strong | literal
| reference | footnote_reference | citation_reference
| substitution_reference | title_reference
| abbreviation | acronym | subscript | superscript
| inline | problematic | generated
| target | image | raw
%additional.inline.elements;

The %additional.inline.elements; parameter entity can be used by wrapper DTDs to extend %inline.elements;.

Via %text.model;, the %inline.elements; parameter entity is indirectly employed in the content models of the following elements: abbreviation, acronym, address, attribution, author, caption, classifier, contact, copyright, date, doctest_block, emphasis, generated, inline, line_block, literal_block, organization, paragraph, problematic, raw, reference, revision, rubric, status, strong, subscript, substitution_definition, substitution_reference, subtitle, superscript, target, term, title, title_reference, version

%reference.atts;

The %reference.atts; parameter entity groups together the refuri, refid, and refname attributes.

Entity definition:

%refuri.att;
%refid.att;
%refname.att;
%additional.reference.atts;

The %additional.reference.atts; parameter entity can be used by wrapper DTDs to extend %additional.reference.atts;.

The citation_reference, footnote_reference, reference, and target elements directly employ the %reference.att; parameter entity in their attribute lists.

%refid.att;

The %refid.att; parameter entity contains the refid attribute, an internal reference to the id attribute of another element.

Entity definition:

refid   CDATA     #IMPLIED

The title and problematic elements directly employ the %refid.att; parameter entity in their attribute lists.

Via %reference.atts;, the %refid.att; parameter entity is indirectly employed in the attribute lists of the citation_reference, footnote_reference, reference, and target elements.

%refname.att;

The %refname.att; parameter entity contains the refname attribute, an internal reference to the name attribute of another element. On a target element, refname indicates an indirect target which may resolve to either an internal or external reference.

Entity definition:

refname  NMTOKENS  #IMPLIED

The substitution_reference element directly employs the %refname.att; parameter entity in its attribute list.

Via %reference.atts;, the %refname.att; parameter entity is indirectly employed in the attribute lists of the citation_reference, footnote_reference, reference, and target elements.

%refuri.att;

The %refuri.att; parameter entity contains the refuri attribute, an external reference to a URI/URL.

Entity definition:

refuri   CDATA     #IMPLIED

Via %reference.atts;, the %refuri.att; parameter entity is indirectly employed in the attribute lists of the citation_reference, footnote_reference, reference, and target elements.

%section.elements;

The %section.elements; parameter entity contains an OR-list of all section-equivalent elements. %section.elements; is itself contained within the %structure.model; parameter entity.

Entity definition:

section
%additional.section.elements;

The %additional.section.elements; parameter entity can be used by wrapper DTDs to extend %section.elements;.

Via %structure.model;, the %section.elements; parameter entity is indirectly employed in the content models of the document and section elements.

%structure.model;

The %structure.model; parameter entity encapsulates the hierarchical structure of a document and of its constituent parts. See the discussion of the element hierarchy above.

Entity definition:

( ( (%body.elements; | topic | sidebar)+,
    (transition, (%body.elements; | topic | sidebar)+ )*,
    (%section.elements;)* )
| (%section.elements;)+ )

Each document or section contains either:

  • multiple body elements, topics, and/or sidebars, optionally interspersed with transitions (but transitions cannot occur at the beginning or end, nor may there be two transitions in a row), followed by zero or more sections; or
  • one or more sections (whose contents are recursively the same as this model).

The %structure.model; parameter entity is directly employed in the content models of the document and section elements.

%text.model;

The %text.model; parameter entity is used by many elements to represent text data mixed with inline elements.

Entity definition:

(#PCDATA | %inline.elements;)*

The %text.model; parameter entity is directly employed in the content models of the following elements: abbreviation, acronym, address, author, caption, classifier, contact, copyright, date, doctest_block, emphasis, field_name, generated, line_block, literal_block, organization, paragraph, problematic, raw, reference, revision, status, strong, substitution_definition, substitution_reference, subtitle, target, term, title, version

View document source. Generated on: 2004-07-01 22:46 UTC. Generated by Docutils from reStructuredText source.


Docutils Transforms

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.7
Date: | 2004-06-03
Copyright: | This document has been placed in the public domain.

Contents

  • Transforms Listed in Priority Order
  • Transform Priority Range Categories

For background about transforms and the Transformer object, see PEP 258.

Transforms Listed in Priority Order

Transform: module.Class | Added By | Priority
misc.class | "class" (d/p) | 210
references.Substitutions | standalone (r), pep (r) | 220
frontmatter.DocTitle | standalone (r) | 320
frontmatter.DocInfo | standalone (r) | 340
peps.Headers | pep (r) | 360
peps.Contents | pep (r) | 380
references.ChainedTargets | standalone (r), pep (r) | 420
references.AnonymousHyperlinks | standalone (r), pep (r) | 440
references.IndirectHyperlinks | standalone (r), pep (r) | 460
peps.TargetNotes | pep (r) | 520
references.TargetNotes | peps.TargetNotes (t/p) | 0
misc.CallBack | peps.TargetNotes (t/p) | 1
references.TargetNotes | "target-notes" (d/p) | 540
references.Footnotes | standalone (r), pep (r) | 620
references.ExternalTargets | standalone (r), pep (r) | 640
references.InternalTargets | standalone (r), pep (r) | 660
parts.SectNum | "sectnum" (d/p) | 710
parts.Contents | "contents" (d/p), peps.Contents (t/p) | 720
peps.PEPZero | peps.Headers (t/p) | 760
components.Filter | "meta" (d/p) | 780
universal.Decorations | Transformer | 820
universal.FinalChecks | Transformer | 840
universal.Messages | Transformer | 860
universal.FilterMessages | Transformer | 870
universal.TestMessages | DocutilsTestSupport | 880
misc.CallBack | n/a | 990

Key:

  • (r): Reader
  • (d): Directive
  • (t): Transform
  • (/p): Via a "pending" node

Transform Priority Range Categories

Priority |  |
From | To | Category
0 | 99 | immediate execution (added by another transform)
100 | 199 | very early (non-standard)
200 | 299 | very early
300 | 399 | early
400 | 699 | main
700 | 799 | late
800 | 899 | very late
900 | 999 | very late (non-standard)
View document source. Generated on: 2004-07-01 22:48 UTC. Generated by Docutils from reStructuredText source.


reStructuredText Directives

Author: | David Goodger
Contact: | goodger@python.org
Revision: | 1.55
Date: | 2004-06-19
Copyright: | This document has been placed in the public domain.

Contents

  • Admonitions
    • Specific Admonitions
    • Generic Admonition
  • Images
    • Image
    • Figure
  • Body Elements
    • Topic
    • Sidebar
    • Line Block
    • Parsed Literal Block
    • Rubric
    • Epigraph
    • Highlights
    • Pull-Quote
  • Tables
    • Table
    • CSV Table
  • Document Parts
    • Table of Contents
    • Automatic Section Numbering
  • References
    • Target Footnotes
    • Footnotes
    • Citations
  • HTML-Specific
    • Meta
    • Imagemap
  • Miscellaneous
    • Including an External Document Fragment
    • Raw Data Pass-Through
    • Replacement Text
    • Unicode Character Codes
    • Class
    • Custom Interpreted Text Roles
    • Restructuredtext-Test-Directive

This document describes the directives implemented in the reference reStructuredText parser.

Directives have the following syntax:

+-------+-------------------------------+
| ".. " | directive type "::" directive |
+-------+ block                         |
        |                               |
        +-------------------------------+

Directives begin with an explicit markup start (two periods and a space), followed by the directive type and two colons (collectively, the "directive marker"). The directive block begins immediately after the directive marker, and includes all subsequent indented lines. The directive block is divided into arguments, options (a field list), and content (in that order), any of which may appear. See the Directives section in the reStructuredText Markup Specification for syntax details.

Descriptions below list "doctree elements" (document tree element names; XML DTD generic identifiers) corresponding to individual directives. For details on the hierarchy of elements, please see The Docutils Document Tree and the Docutils Generic DTD XML document type definition. For directive implementation details, see Creating reStructuredText Directives.

Admonitions

Specific Admonitions

Directive Types:
 | "attention", "caution", "danger", "error", "hint", "important", "note", "tip", "warning", "admonition"
Doctree Elements:
 | attention, caution, danger, error, hint, important, note, tip, warning, admonition, title
Directive Arguments:
 | None.
Directive Options:
 | None.
Directive Content:
 | Interpreted as body elements.

Admonitions are specially marked "topics" that can appear anywhere an ordinary body element can. They contain arbitrary body elements. Typically, an admonition is rendered as an offset block in a document, sometimes outlined or shaded, with a title matching the admonition type. For example:

.. DANGER::
   Beware killer rabbits!

This directive might be rendered something like this:

+------------------------+
|        !DANGER!        |
|                        |
| Beware killer rabbits! |
+------------------------+

The following admonition directives have been implemented:

  • attention
  • caution
  • danger
  • error
  • hint
  • important
  • note
  • tip
  • warning

Any text immediately following the directive indicator (on the same line and/or indented on following lines) is interpreted as a directive block and is parsed for normal body elements. For example, the following "note" admonition directive contains one paragraph and a bullet list consisting of two list items:

.. note:: This is a note admonition.
   This is the second line of the first paragraph.

   - The note contains all indented body elements
     following.
   - It includes this bullet list.

Generic Admonition

Directive Type: | "admonition"
Doctree Elements:
 | admonition, title
Directive Arguments:
 | One, required (admonition title)
Directive Options:
 | Possible.
Directive Content:
 | Interpreted as body elements.

This is a generic, titled admonition. The title may be anything the author desires.

The author-supplied title is also used as a "class" attribute value after being converted into a valid identifier form (down-cased; non-alphanumeric characters converted to single hyphens; "admonition-" prefixed). For example, this admonition:

.. admonition:: And, by the way...

   You can make up your own admonition too.

becomes the following document tree (pseudo-XML):

<document source="test data">
    <admonition class="admonition-and-by-the-way">
        <title>
            And, by the way...
        <paragraph>
            You can make up your own admonition too.

The following option is recognized:

class : text
Override the computed "class" attribute value. See the class directive below.

Images

There are two image directives: "image" and "figure".

Image

Directive Type: | "image"
Doctree Element:
 | image
Directive Arguments:
 | One, required (image URI).
Directive Options:
 | Possible.
Directive Content:
 | None.

An "image" is a simple picture:

.. image:: picture.png

The URI for the image source file is specified in the directive argument. As with hyperlink targets, the image URI may begin on the same line as the explicit markup start and target name, or it may begin in an indented text block immediately following, with no intervening blank lines. If there are multiple lines in the link block, they are stripped of leading and trailing whitespace and joined together.

Optionally, the image link block may contain a flat field list, the image options. For example:

.. image:: picture.jpeg
   :height: 100
   :width: 200
   :scale: 50
   :alt: alternate text
   :align: right

The following options are recognized:

alt : text
Alternate text: a short description of the image, displayed by applications that cannot display images, or spoken by applications for visually impaired users.
height : integer
The desired height of the image in pixels, used to reserve space or scale the image vertically. When the "scale" option is also specified, they are combined. For example, a height of 200 and a scale of 50 is equivalent to a height of 100 with no scale.
width : integer
The width of the image in pixels, used to reserve space or scale the image horizontally. As with "height" above, when the "scale" option is also specified, they are combined.
scale : integer

The uniform scaling factor of the image, a percentage (but no "%" symbol is required or allowed). "100" means full-size, and is equivalent to omitting a "scale" option.

If no "height" or "width" options are specified, PIL 1 may be used to determine them, if PIL is installed and the image file is available.

align : "top", "middle", "bottom", "left", "center", or "right"
The alignment of the image, equivalent to the HTML <img> tag's "align" attribute. The values "top", "middle", and "bottom" control an image's vertical alignment (relative to the text baseline); they are only useful for inline images (substitutions). The values "left", "center", and "right" control an image's horizontal alignment, allowing the image to float and have the text flow around it. The specific behavior depends upon the browser or rendering software used.
target : text (URI or reference name)
Makes the image into a hyperlink reference ("clickable"). The option argument may be a URI (relative or absolute), or a reference name with underscore suffix (e.g. name_).
class : text
Set a "class" attribute value on the image element. See the class directive below.

Figure

Directive Type: | "figure"
Doctree Elements:
 | figure, image, caption, legend
Directive Arguments:
 | One, required (image URI).
Directive Options:
 | Possible.
Directive Content:
 | Interpreted as the figure caption and an optional legend.

A "figure" consists of image data (including image options), an optional caption (a single paragraph), and an optional legend (arbitrary body elements):

.. figure:: picture.png
   :scale: 50
   :alt: map to buried treasure

   This is the caption of the figure (a simple paragraph).

   The legend consists of all elements after the caption.  In this
   case, the legend consists of this paragraph and the following
   table:

   +-----------------------+-----------------------+
   | Symbol                | Meaning               |
   +=======================+=======================+
   | .. image:: tent.png   | Campground            |
   +-----------------------+-----------------------+
   | .. image:: waves.png  | Lake                  |
   +-----------------------+-----------------------+
   | .. image:: peak.png   | Mountain              |
   +-----------------------+-----------------------+

There must be blank lines before the caption paragraph and before the legend. To specify a legend without a caption, use an empty comment ("..") in place of the caption.

The "figure" directive supports all of the options of the "image" directive (see image options above). In addition, the following options are recognized:

figwidth : integer or "image"

The width of the figure in pixels, to limit the horizontal space used. A special value of "image" is allowed, in which case the included image's actual width is used (requires PIL 1). If the image file is not found or the required software is unavailable, this option is ignored.

Sets the "width" attribute of the "figure" doctree element.

This option does not scale the included image; use the "width" image option for that.

+---------------------------+
|        figure             |
|                           |
|<------ figwidth --------->|
|                           |
|  +---------------------+  |
|  |     image           |  |
|  |                     |  |
|  |<--- width --------->|  |
|  +---------------------+  |
|                           |
|The figure's caption should|
|wrap at this width.        |
+---------------------------+
figclass : text
Set a "class" attribute value on the figure element. See the class directive below.
[1] (1, 2) Python Imaging Library.

Body Elements

Topic

Directive Type: | "topic"
Doctree Element:
 | topic
Directive Arguments:
 | 1, required (topic title).
Directive Options:
 | Possible.
Directive Content:
 | Interpreted as the topic body.

A topic is like a block quote with a title, or a self-contained section with no subsections. Use the "topic" directive to indicate a self-contained idea that is separate from the flow of the document. Topics may occur anywhere a section or transition may occur. Body elements (including topics) may not contain nested topics.

The directive's sole argument is interpreted as the topic title; the next line must be blank. All subsequent lines make up the topic body, interpreted as body elements. For example:

.. topic:: Topic Title

    Subsequent indented lines comprise
    the body of the topic, and are
    interpreted as body elements.

The following option is recognized:

class : text
Set a "class" attribute value on the topic element. See the class directive below.

Sidebar

Directive Type: | "sidebar"
Doctree Element:
 | sidebar
Directive Arguments:
 | One, required (sidebar title).
Directive Options:
 | Possible.
Directive Content:
 | Interpreted as the sidebar body.

Sidebars are like miniature, parallel documents that occur inside other documents, providing related or reference material. A sidebar is typically offset by a border and "floats" to the side of the page; the document's main text may flow around it. Sidebars can also be likened to super-footnotes; their content is outside of the flow of the document's main text.

Sidebars may occur anywhere a section or transition may occur. Body elements (including sidebars) may not contain nested sidebars.

The directive's sole argument is interpreted as the sidebar title, which may be followed by a subtitle option (see below); the next line must be blank. All subsequent lines make up the sidebar body, interpreted as body elements. For example:

.. sidebar:: Sidebar Title
   :subtitle: Optional Sidebar Subtitle

   Subsequent indented lines comprise
   the body of the sidebar, and are
   interpreted as body elements.

The following options are recognized:

subtitle : text
The sidebar's subtitle.
class : text
Set a "class" attribute value on the sidebar element. See the class directive below.

Line Block

Directive Type: | "line-block"
Doctree Element:
 | line_block
Directive Arguments:
 | None.
Directive Options:
 | Possible.
Directive Content:
 | Becomes the body of the line block.

The "line-block" directive constructs an element where whitespace (including linebreaks) is significant and inline markup is supported. It is equivalent to a parsed literal block with different rendering: typically in an ordinary serif typeface instead of a typewriter/monospaced face, and not automatically indented. (Have the line-block directive begin a block quote to get an indented line block.) Line blocks are useful for address blocks and verse (poetry, song lyrics), where the structure of lines is significant. For example, here's a classic:

"To Ma Own Beloved Lassie: A Poem on her 17th Birthday", by
Ewan McTeagle (for Lassie O'Shea):

    .. line-block::

        Lend us a couple of bob till Thursday.
        I'm absolutely skint.
        But I'm expecting a postal order and I can pay you back
            as soon as it comes.
        Love, Ewan.

The following option is recognized:

class : text
Set a "class" attribute value on the line_block element. See the class directive below.

Parsed Literal Block

Directive Type: | "parsed-literal"
Doctree Element:
 | literal_block
Directive Arguments:
 | None.
Directive Options:
 | Possible.
Directive Content:
 | Becomes the body of the literal block.

Unlike an ordinary literal block, the "parsed-literal" directive constructs a literal block where the text is parsed for inline markup. It is equivalent to a line block with different rendering: typically in a typewriter/monospaced typeface, like an ordinary literal block. Parsed literal blocks are useful for adding hyperlinks to code examples.

However, care must be taken with the text, because inline markup is recognized and there is no protection from parsing. Backslash-escapes may be necessary to prevent unintended parsing. And because the markup characters are removed by the parser, care must also be taken with vertical alignment. Parsed "ASCII art" is tricky, and extra whitespace may be necessary.

For example, all the element names in this content model are links:

.. parsed-literal::

    ((title_,
      subtitle_?)?,
     docinfo_?,
     decoration_?,
     `%structure.model;`_)

The following option is recognized:

class : text
Set a "class" attribute value on the literal_block element. See the class directive below.

Rubric

Directive Type: | "rubric"
Doctree Element:
 | rubric
Directive Arguments:
 | 1, required (rubric text).
Directive Options:
 | Possible.
Directive Content:
 | None. rubric n. 1. a title, heading, or the like, in a manuscript, book, statute, etc., written or printed in red or otherwise distinguished from the rest of the text. ... -- Random House Webster's College Dictionary, 1991

The "rubric" directive inserts a "rubric" element into the document tree. A rubric is like an informal heading that doesn't correspond to the document's structure.

The following option is recognized:

class : text
Set a "class" attribute value on the rubric element. See the class directive below.

Epigraph

Directive Type: | "epigraph"
Doctree Element:
 | block_quote
Directive Arguments:
 | None.
Directive Options:
 | None.
Directive Content:
 | Interpreted as the body of the block quote.

Produces an epigraph-class block quote. For example, this input:

.. epigraph::

   No matter where you go, there you are.

   -- Buckaroo Banzai

becomes this document tree fragment:

<block_quote class="epigraph">
    <paragraph>
        No matter where you go, there you are.
    <attribution>
        Buckaroo Banzai

Highlights

Directive Type: | "highlights"
Doctree Element:
 | block_quote
Directive Arguments:
 | None.
Directive Options:
 | None.
Directive Content:
 | Interpreted as the body of the block quote.

Produces a highlights-class block quote. See Epigraph above.

Pull-Quote

Directive Type: | "pull-quote"
Doctree Element:
 | block_quote
Directive Arguments:
 | None.
Directive Options:
 | None.
Directive Content:
 | Interpreted as the body of the block quote.

Produces a pull-quote-class block quote. See Epigraph above.

Tables

Formal tables need more structure than the reStructuredText syntax supplies. Tables may be given titles with the table directive. Sometimes reStructuredText tables are inconvenient to write, or table data in a standard format is readily available. The csv-table directive supports CSV data.

Table

Directive Type: | "table"
Doctree Element:
 | table
Directive Arguments:
 | 1, optional (table title).
Directive Options:
 | Possible.
Directive Content:
 | A normal reStructuredText table.

(New in Docutils 0.3.1)

The "table" directive is used to create a titled table, to associate a title with a table:

.. table:: Truth table for "not"

   =====  =====
     A    not A
   =====  =====
   False  True
   True   False
   =====  =====

The following option is recognized:

class : text
Set a "class" attribute value on the table element. See the class directive below.

CSV Table

Directive Type: | "csv-table"
Doctree Element:
 | table
Directive Arguments:
 | 1, optional (table title).
Directive Options:
 | Possible.
Directive Content:
 | A CSV (comma-separated values) table.

Warning

The "csv-table" directive requires the csv.py module of the Python standard library, which was added in Python 2.3. It will not work with earlier versions of Python. Using the "csv-table" directive in a document will make the document incompatible with systems using Python 2.1 or 2.2.

(New in Docutils 0.3.4)

The "csv-table" directive is used to create a table from CSV (comma-separated values) data. CSV is a common data format generated by spreadsheet applications and commercial databases. The data may be internal (an integral part of the document) or external (a separate file).

Example:

.. csv-table:: Frozen Delights!
   :headers: "Treat", "Quantity", "Description"
   :widths: 15, 10, 30

   "Albatross", 2.99, "On a stick!"
   "Crunchy Frog", 1.49, "If we took the bones out, it wouldn't be
   crunchy, now would it?"
   "Gannet Ripple", 1.99, "On a stick!"

Block markup and inline markup within cells is supported. Line ends are recognized within cells.

Working limitations:

  • Whitespace delimiters are supported only for external CSV files.

  • There is no support for checking that the number of columns in each row is the same. However, this directive supports CSV generators that do not insert "empty" entries at the end of short rows, by automatically adding empty entries.

  • Due to limitations of the CSV parser, this directive is not Unicode compatible. It may also have problems with ASCII NUL characters. Accordingly, CSV tables should be ASCII-printable safe.

The following options are recognized:

class : text
Set a "class" attribute value on the table element. See the class directive below.
widths : integer [, integer...]
A comma-separated list of relative column widths. The default is equal-width columns (100%/#columns).
header-rows : integer
The number of rows of CSV data to use in the table header. Defaults to 0.
header : CSV data
Supplemental data for the table header, added independently of and before any header-rows from the main CSV data. Must use the same CSV format as the main CSV data.
file | url : path
Path or URL to CSV file.
delim : char | "tab" | "space"
A one-character string used to separate fields. Defaults to , (comma). May be specified as a Unicode code point; see the unicode directive for syntax details.
quote : char
A one-character string used to quote elements containing the delimiter or which start with the quote character. Defaults to " (quote). May be specified as a Unicode code point; see the unicode directive for syntax details.
keepspace : flag
Treat whitespace immediately following the delimiter as significant. The default is to ignore such whitespace.
escape : char

A one-character string used to escape the delimiter or quote characters. May be specified as a Unicode code point; see the unicode directive for syntax details. Used when the delimiter is used in an unquoted field, or when quote characters are used within a field. The default is to double-up the character, e.g. "He said, ""Hi!"""

Document Parts

Table of Contents

Directive Type: | "contents"
Doctree Elements:
 | pending, topic
Directive Arguments:
 | One, optional: title.
Directive Options:
 | Possible.
Directive Content:
 | None.

The "contents" directive inserts a table of contents (TOC) in two passes: initial parse and transform. During the initial parse, a "pending" element is generated which acts as a placeholder, storing the TOC title and any options internally. At a later stage in the processing, the "pending" element is replaced by a "topic" element, a title and the table of contents proper.

The directive in its simplest form:

.. contents::

Language-dependent boilerplate text will be used for the title. The English default title text is "Contents".

An explicit title, may be specified:

.. contents:: Table of Contents

The title may span lines, although it is not recommended:

.. contents:: Here's a very long Table of
   Contents title

Options may be specified for the directive, using a field list:

.. contents:: Table of Contents
   :depth: 2

If the default title is to be used, the options field list may begin on the same line as the directive marker:

.. contents:: :depth: 2

The following options are recognized:

depth : integer
The number of section levels that are collected in the table of contents. The default is unlimited depth.
local : flag (empty)
Generate a local table of contents. Entries will only include subsections of the section in which the directive is given. If no explicit title is given, the table of contents will not be titled.
backlinks : "entry" or "top" or "none"
Generate links from section headers back to the table of contents entries, the table of contents itself, or generate no backlinks.
class : text
Set a "class" attribute value on the topic element. See the class directive below.

Automatic Section Numbering

Directive Type: | "sectnum" or "section-autonumbering" (synonyms)
Doctree Elements:
 | pending, generated
Directive Arguments:
 | None.
Directive Options:
 | Possible.
Directive Content:
 | None.

The "sectnum" (or "section-autonumbering") directive automatically numbers sections and subsections in a document. Section numbers are of the "multiple enumeration" form, where each level has a number, separated by periods. For example, the title of section 1, subsection 2, subsubsection 3 would have "1.2.3" prefixed.

The "sectnum" directive does its work in two passes: the initial parse and a transform. During the initial parse, a "pending" element is generated which acts as a placeholder, storing any options internally. At a later stage in the processing, the "pending" element triggers a transform, which adds section numbers to titles. Section numbers are enclosed in a "generated" element, and titles have their "auto" attribute set to "1".

The following options are recognized:

depth : integer
The number of section levels that are numbered by this directive. The default is unlimited depth.

References

Target Footnotes

Directive Type: | "target-notes"
Doctree Elements:
 | pending, footnote, footnote_reference
Directive Arguments:
 | None.
Directive Options:
 | None.
Directive Content:
 | None.

The "target-notes" directive creates a footnote for each external target in the text, and corresponding footnote references after each reference. For every explicit target (of the form, .. _target name: URL) in the text, a footnote will be generated containing the visible URL as content.

Footnotes

NOT IMPLEMENTED YET

Directive Type: | "footnotes"
Doctree Elements:
 | pending, topic
Directive Arguments:
 | None?
Directive Options:
 | Possible?
Directive Content:
 | None.

@@@

Citations

NOT IMPLEMENTED YET

Directive Type: | "citations"
Doctree Elements:
 | pending, topic
Directive Arguments:
 | None?
Directive Options:
 | Possible?
Directive Content:
 | None.

@@@

HTML-Specific

Meta

Directive Type: | "meta"
Doctree Element:
 | meta (non-standard)
Directive Arguments:
 | None.
Directive Options:
 | None.
Directive Content:
 | Must contain a flat field list.

The "meta" directive is used to specify HTML metadata stored in HTML META tags. "Metadata" is data about data, in this case data about web pages. Metadata is used to describe and classify web pages in the World Wide Web, in a form that is easy for search engines to extract and collate.

Within the directive block, a flat field list provides the syntax for metadata. The field name becomes the contents of the "name" attribute of the META tag, and the field body (interpreted as a single string without inline markup) becomes the contents of the "content" attribute. For example:

.. meta::
   :description: The reStructuredText plaintext markup language
   :keywords: plaintext, markup language

This would be converted to the following HTML:

<meta name="description"
    content="The reStructuredText plaintext markup language">
<meta name="keywords" content="plaintext, markup language">

Support for other META attributes ("http-equiv", "scheme", "lang", "dir") are provided through field arguments, which must be of the form "attr=value":

.. meta::
   :description lang=en: An amusing story
   :description lang=fr: Un histoire amusant

And their HTML equivalents:

<meta name="description" lang="en" content="An amusing story">
<meta name="description" lang="fr" content="Un histoire amusant">

Some META tags use an "http-equiv" attribute instead of the "name" attribute. To specify "http-equiv" META tags, simply omit the name:

.. meta::
   :http-equiv=Content-Type: text/html; charset=ISO-8859-1

HTML equivalent:

<meta http-equiv="Content-Type"
     content="text/html; charset=ISO-8859-1">

Imagemap

NOT IMPLEMENTED YET

Non-standard element: imagemap.

Miscellaneous

Including an External Document Fragment

Directive Type: | "include"
Doctree Elements:
 | depend on data being included
Directive Arguments:
 | One, required.
Directive Options:
 | Possible.
Directive Content:
 | None.

The "include" directive reads a reStructuredText-formatted text file and parses it in the current document's context at the point of the directive. The directive argument is the path to the file to be included, relative to the document containing the directive. For example:

This first example will be parsed at the document level, and can
thus contain any construct, including section headers.

.. include:: inclusion.txt

Back in the main document.

    This second example will be parsed in a block quote context.
    Therefore it may only contain body elements.  It may not
    contain section headers.

    .. include:: inclusion.txt

If an included document fragment contains section structure, the title adornments must match those of the master document.

The text encoding of the master input source is used for included files.

The following options are recognized:

literal : flag (empty)
The entire included text is inserted into the document as a single literal block (useful for program listings).

Raw Data Pass-Through

Directive Type: | "raw"
Doctree Element:
 | raw
Directive Arguments:
 | One, required (output format type).
Directive Options:
 | Possible.
Directive Content:
 | Stored verbatim, uninterpreted. None (empty) if a "file" or "url" option given.

The "raw" directive indicates non-reStructuredText data that is to be passed untouched to the Writer. The name of the output format is given in the first argument. The interpretation of the raw data is up to the Writer. A Writer may ignore any raw output not matching its format.

For example, the following input would be passed untouched by an HTML Writer:

.. raw:: html

   <hr width=50 size=10>

A LaTeX Writer could insert the following raw content into its output stream:

.. raw:: latex

   \setlength{\parindent}{0pt}

Raw data can also be read from an external file, specified in a directive option. In this case, the content block must be empty. For example:

.. raw:: html
   :file: inclusion.html

The following options are recognized:

file : string
The local filesystem path of a raw data file to be included.
url : string
An Internet URL reference to a raw data file to be included.

Replacement Text

Directive Type: | "replace"
Doctree Element:
 | Text & inline elements
Directive Arguments:
 | None.
Directive Options:
 | None.
Directive Content:
 | A single paragraph; may contain inline markup.

The "replace" directive is used to indicate replacement text for a substitution reference. It may be used within substitution definitions only. For example, this directive can be used to expand abbreviations:

.. |reST| replace:: reStructuredText

Yes, |reST| is a long word, so I can't blame anyone for wanting to
abbreviate it.

As reStructuredText doesn't support nested inline markup, the only way to create a reference with styled text is to use substitutions with the "replace" directive:

I recommend you try |Python|_.

.. |Python| replace:: Python, *the* best language around
.. _Python: http://www.python.org/

Unicode Character Codes

Directive Type: | "unicode"
Doctree Element:
 | Text
Directive Arguments:
 | One or more, required (Unicode character codes, optional text, and comments).
Directive Options:
 | None.
Directive Content:
 | None.

The "unicode" directive converts Unicode character codes (numerical values) to characters, and may be used in substitution definitions only. Codes may be decimal numbers, hexadecimal numbers (prefixed by 0x, x, \x, U+, u, or \u), or XML-style numeric character entities (e.g. &#x1a2b;). Codes are case-insensitive.

Text following " .. " is a comment and is ignored. Spaces are ignored, and any other text remains as-is.

Example:

Copyright |copy| 2003, |BogusMegaCorp (TM)|.

.. |copy| unicode:: 0xA9 .. copyright sign
.. |BogusMegaCorp (TM)| unicode:: BogusMegaCorp U+2122
   .. with trademark sign

Class

Directive Type: | "class"
Doctree Element:
 | pending
Directive Arguments:
 | One, required (class name / attribute value).
Directive Options:
 | None.
Directive Content:
 | None.

The "class" directive sets a "class" attribute value on the first immediately following non-comment element 2. For details of the "class" attribute, see its entry in The Docutils Document Tree. The "class" attribute value is converted to lowercase and all non-alphanumeric characters are converted to hyphens in order to match this regular expression: [a-z](-?[a-z0-9]+)*. See the docstring from docutils.nodes.make_id for the rationale.

Examples:

.. class:: special

This is a "special" paragraph.

.. class:: exceptional

An Exceptional Section
======================

This is an ordinary paragraph.

The text above is parsed and transformed into this doctree fragment:

<paragraph class="special">
    This is a "special" paragraph.
<section class="exceptional">
    <title>
        An Exceptional Section
    <paragraph>
        This is an ordinary paragraph.
[2]

To set a "class" attribute value on a block quote, the "class" directive must be followed by an empty comment:

.. class:: highlights
..

    Block quote text.

The directive doesn't allow content, therefore an empty comment is required to terminate the directive. Without the empty comment, the block quote text would be interpreted as the "class" directive's content, and the parser would complain.

Custom Interpreted Text Roles

Directive Type: | "role"
Doctree Element:
 | None; affects subsequent parsing.
Directive Arguments:
 | Two; one required (role name), one optional (base role name, in parentheses).
Directive Options:
 | Possible (depends on base role).
Directive Content:
 | depends on base role.

(New in Docutils 0.3.2)

The "role" directive dynamically creates a custom interpreted text role and registers it with the parser. This means that after declaring a role like this:

.. role:: custom

the document may use the new "custom" role:

An example of using :custom:`interpreted text`

This will be parsed into the following document tree fragment:

<paragraph>
    An example of using
    <inline class="custom">
        interpreted text

The role must be declared in a document before it can be used.

The new role may be based on an existing role, specified as a second argument in parentheses (whitespace optional):

.. role:: custom(emphasis)

:custom:`text`

The parsed result is as follows:

<paragraph>
    <emphasis class="custom">
        text

If no base role is explicitly specified, a generic custom role is automatically used. Subsequent interpreted text will produce an "inline" element with a "class" attribute, as in the first example above.

With most roles, the ":class:" option can be used to set a "class" attribute that is different from the role name. For example:

.. role:: custom
   :class: special

:custom:`interpreted text`

This is the parsed result:

<paragraph>
    <inline class="special">
        interpreted text

The following option is recognized by the "role" directive for most base roles:

class : text
Set a "class" attribute value on the element produced (inline, or element associated with a base class) when the custom interpreted text role is used. If no directive options are specified, a "class" option with the directive argument (role name) as the value is implied. See the class directive above.

Specific roles may support other options and/or directive content. See the reStructuredText Interpreted Text Roles document for details.

Restructuredtext-Test-Directive

Directive Type: | "restructuredtext-test-directive"
Doctree Element:
 | system_warning
Directive Arguments:
 | None.
Directive Options:
 | None.
Directive Content:
 | Interpreted as a literal block.

This directive is provided for test purposes only. (Nobody is expected to type in a name that long!) It is converted into a level-1 (info) system message showing the directive data, possibly followed by a literal block containing the rest of the directive block.

View document source. Generated on: 2004-07-01 22:48 UTC. Generated by Docutils from reStructuredText source.


An Introduction to reStructuredText

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.8
Date: | 2004-06-03
Copyright: | This document has been placed in the public domain.

reStructuredText is an easy-to-read, what-you-see-is-what-you-get plaintext markup syntax and parser system. It is useful for inline program documentation (such as Python docstrings), for quickly creating simple web pages, and for standalone documents. reStructuredText is a proposed revision and reinterpretation of the StructuredText and Setext lightweight markup systems.

reStructuredText is designed for extensibility for specific application domains. Its parser is a component of Docutils.

This document defines the goals of reStructuredText and provides a history of the project. It is written using the reStructuredText markup, and therefore serves as an example of its use. For a gentle introduction to using reStructuredText, please read A ReStructuredText Primer. The Quick reStructuredText user reference is also useful. The reStructuredText Markup Specification is the definitive reference. There is also an analysis of the Problems With StructuredText.

ReStructuredText's web page is http://docutils.sourceforge.net/rst.html.

Goals

The primary goal of reStructuredText is to define a markup syntax for use in Python docstrings and other documentation domains, that is readable and simple, yet powerful enough for non-trivial use. The intended purpose of the reStructuredText markup is twofold:

  • the establishment of a set of standard conventions allowing the expression of structure within plaintext, and
  • the conversion of such documents into useful structured data formats.

The secondary goal of reStructuredText is to be accepted by the Python community (by way of being blessed by PythonLabs and the BDFL 1) as a standard for Python inline documentation (possibly one of several standards, to account for taste).

[1] Python's creator and "Benevolent Dictator For Life", Guido van Rossum.

To clarify the primary goal, here are specific design goals, in order, beginning with the most important:

  1. Readable. The marked-up text must be easy to read without any prior knowledge of the markup language. It should be as easily read in raw form as in processed form.
  2. Unobtrusive. The markup that is used should be as simple and unobtrusive as possible. The simplicity of markup constructs should be roughly proportional to their frequency of use. The most common constructs, with natural and obvious markup, should be the simplest and most unobtrusive. Less common constructs, for which there is no natural or obvious markup, should be distinctive.
  3. Unambiguous. The rules for markup must not be open for interpretation. For any given input, there should be one and only one possible output (including error output).
  4. Unsurprising. Markup constructs should not cause unexpected output upon processing. As a fallback, there must be a way to prevent unwanted markup processing when a markup construct is used in a non-markup context (for example, when documenting the markup syntax itself).
  5. Intuitive. Markup should be as obvious and easily remembered as possible, for the author as well as for the reader. Constructs should take their cues from such naturally occurring sources as plaintext email messages, newsgroup postings, and text documentation such as README.txt files.
  6. Easy. It should be easy to mark up text using any ordinary text editor.
  7. Scalable. The markup should be applicable regardless of the length of the text.
  8. Powerful. The markup should provide enough constructs to produce a reasonably rich structured document.
  9. Language-neutral. The markup should apply to multiple natural (as well as artificial) languages, not only English.
  10. Extensible. The markup should provide a simple syntax and interface for adding more complex general markup, and custom markup.
  11. Output-format-neutral. The markup will be appropriate for processing to multiple output formats, and will not be biased toward any particular format.

The design goals above were used as criteria for accepting or rejecting syntax, or selecting between alternatives.

It is emphatically not the goal of reStructuredText to define docstring semantics, such as docstring contents or docstring length. These issues are orthogonal to the markup syntax and beyond the scope of this specification.

Also, it is not the goal of reStructuredText to maintain compatibility with StructuredText or Setext. reStructuredText shamelessly steals their great ideas and ignores the not-so-great.

Author's note:

Due to the nature of the problem we're trying to solve (or, perhaps, due to the nature of the proposed solution), the above goals unavoidably conflict. I have tried to extract and distill the wisdom accumulated over the years in the Python Doc-SIG mailing list and elsewhere, to come up with a coherent and consistent set of syntax rules, and the above goals by which to measure them.

There will inevitably be people who disagree with my particular choices. Some desire finer control over their markup, others prefer less. Some are concerned with very short docstrings, others with full-length documents. This specification is an effort to provide a reasonably rich set of markup constructs in a reasonably simple form, that should satisfy a reasonably large group of reasonable people.

David Goodger (goodger@users.sourceforge.net), 2001-04-20

History

reStructuredText, the specification, is based on StructuredText and Setext. StructuredText was developed by Jim Fulton of Zope Corporation (formerly Digital Creations) and first released in 1996. It is now released as a part of the open-source "Z Object Publishing Environment" (ZOPE). Ian Feldman's and Tony Sanders' earlier Setext specification was either an influence on StructuredText or, by their similarities, at least evidence of the correctness of this approach.

I discovered StructuredText in late 1999 while searching for a way to document the Python modules in one of my projects. Version 1.1 of StructuredText was included in Daniel Larsson's pythondoc. Although I was not able to get pythondoc to work for me, I found StructuredText to be almost ideal for my needs. I joined the Python Doc-SIG (Documentation Special Interest Group) mailing list and found an ongoing discussion of the shortcomings of the StructuredText "standard". This discussion has been going on since the inception of the mailing list in 1996, and possibly predates it.

I decided to modify the original module with my own extensions and some suggested by the Doc-SIG members. I soon realized that the module was not written with extension in mind, so I embarked upon a general reworking, including adapting it to the "re" regular expression module (the original inspiration for the name of this project). Soon after I completed the modifications, I discovered that StructuredText.py was up to version 1.23 in the ZOPE distribution. Implementing the new syntax extensions from version 1.23 proved to be an exercise in frustration, as the complexity of the module had become overwhelming.

In 2000, development on StructuredTextNG ("Next Generation") began at Zope Corporation (then Digital Creations). It seems to have many improvements, but still suffers from many of the problems of classic StructuredText.

I decided that a complete rewrite was in order, and even started a reStructuredText SourceForge project (now inactive). My motivations (the "itches" I aim to "scratch") are as follows:

  • I need a standard format for inline documentation of the programs I write. This inline documentation has to be convertible to other useful formats, such as HTML. I believe many others have the same need.
  • I believe in the Setext/StructuredText idea and want to help formalize the standard. However, I feel the current specifications and implementations have flaws that desperately need fixing.
  • reStructuredText could form part of the foundation for a documentation extraction and processing system, greatly benefitting Python. But it is only a part, not the whole. reStructuredText is a markup language specification and a reference parser implementation, but it does not aspire to be the entire system. I don't want reStructuredText or a hypothetical Python documentation processor to die stillborn because of over-ambition.
  • Most of all, I want to help ease the documentation chore, the bane of many a programmer.

Unfortunately I was sidetracked and stopped working on this project. In November 2000 I made the time to enumerate the problems of StructuredText and possible solutions, and complete the first draft of a specification. This first draft was posted to the Doc-SIG in three parts:

  • A Plan for Structured Text
  • Problems With StructuredText
  • reStructuredText: Revised Structured Text Specification

In March 2001 a flurry of activity on the Doc-SIG spurred me to further revise and refine my specification, the result of which you are now reading. An offshoot of the reStructuredText project has been the realization that a single markup scheme, no matter how well thought out, may not be enough. In order to tame the endless debates on Doc-SIG, a flexible Docstring Processing System framework needed to be constructed. This framework has become the more important of the two projects; reStructuredText has found its place as one possible choice for a single component of the larger framework.

The project web site and the first project release were rolled out in June 2001, including posting the second draft of the spec 2 and the first draft of PEPs 256, 257, and 258 3 to the Doc-SIG. These documents and the project implementation proceeded to evolve at a rapid pace. Implementation history details can be found in the project history file.

In November 2001, the reStructuredText parser was nearing completion. Development of the parser continued with the addition of small convenience features, improvements to the syntax, the filling in of gaps, and bug fixes. After a long holiday break, in early 2002 most development moved over to the other Docutils components, the "Readers", "Writers", and "Transforms". A "standalone" reader (processes standalone text file documents) was completed in February, and a basic HTML writer (producing HTML 4.01, using CSS-1) was completed in early March.

PEP 287, "reStructuredText Standard Docstring Format", was created to formally propose reStructuredText as a standard format for Python docstrings, PEPs, and other files. It was first posted to comp.lang.python and the Python-dev mailing list on 2002-04-02.

Version 0.4 of the reStructuredText and Docstring Processing System projects were released in April 2002. The two projects were immediately merged, renamed to "Docutils", and a 0.1 release soon followed.

[2]

The second draft of the spec:

  • An Introduction to reStructuredText
  • Problems With StructuredText
  • reStructuredText Markup Specification
  • Python Extensions to the reStructuredText Markup Specification
[3]

First drafts of the PEPs:

  • PEP 256: Docstring Processing System Framework
  • PEP 258: DPS Generic Implementation Details
  • PEP 257: Docstring Conventions

Current working versions of the PEPs can be found in http://docutils.sourceforge.net/docs/peps/, and official versions can be found in the master PEP repository.
View document source. Generated on: 2004-07-01 22:48 UTC. Generated by Docutils from reStructuredText source.


reStructuredText Markup Specification

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.63
Date: | 2004-06-28
Copyright: | This document has been placed in the public domain.

Note

This document is a detailed technical specification; it is not a tutorial or a primer. If this is your first exposure to reStructuredText, please read A ReStructuredText Primer and the Quick reStructuredText user reference first.

reStructuredText is plaintext that uses simple and intuitive constructs to indicate the structure of a document. These constructs are equally easy to read in raw and processed forms. This document is itself an example of reStructuredText (raw, if you are reading the text file, or processed, if you are reading an HTML document, for example). The reStructuredText parser is a component of Docutils.

Simple, implicit markup is used to indicate special constructs, such as section headings, bullet lists, and emphasis. The markup used is as minimal and unobtrusive as possible. Less often-used constructs and extensions to the basic reStructuredText syntax may have more elaborate or explicit markup.

reStructuredText is applicable to documents of any length, from the very small (such as inline program documentation fragments, e.g. Python docstrings) to the quite large (this document).

The first section gives a quick overview of the syntax of the reStructuredText markup by example. A complete specification is given in the Syntax Details section.

Literal blocks (in which no markup processing is done) are used for examples throughout this document, to illustrate the plaintext markup.

Contents

  • Quick Syntax Overview
  • Syntax Details
    • Whitespace
      • Blank Lines
      • Indentation
    • Escaping Mechanism
    • Reference Names
    • Document Structure
      • Document
      • Sections
      • Transitions
    • Body Elements
      • Paragraphs
      • Bullet Lists
      • Enumerated Lists
      • Definition Lists
      • Field Lists
        • Bibliographic Fields
        • RCS Keywords
      • Option Lists
      • Literal Blocks
        • Indented Literal Blocks
        • Quoted Literal Blocks
      • Block Quotes
      • Doctest Blocks
      • Tables
        • Grid Tables
        • Simple Tables
      • Explicit Markup Blocks
        • Footnotes
          • Auto-Numbered Footnotes
          • Auto-Symbol Footnotes
          • Mixed Manual and Auto-Numbered Footnotes
        • Citations
        • Hyperlink Targets
          • Anonymous Hyperlinks
        • Directives
        • Substitution Definitions
        • Comments
    • Implicit Hyperlink Targets
    • Inline Markup
      • Character-Level Inline Markup
      • Emphasis
      • Strong Emphasis
      • Interpreted Text
      • Inline Literals
      • Hyperlink References
        • Embedded URIs
      • Inline Internal Targets
      • Footnote References
      • Citation References
      • Substitution References
      • Standalone Hyperlinks
  • Error Handling

Quick Syntax Overview

A reStructuredText document is made up of body or block-level elements, and may be structured into sections. Sections are indicated through title style (underlines & optional overlines). Sections contain body elements and/or subsections. Some body elements contain further elements, such as lists containing list items, which in turn may contain paragraphs and other body elements. Others, such as paragraphs, contain text and inline markup elements.

Here are examples of body elements:

  • Paragraphs (and inline markup):

    Paragraphs contain text and may contain inline markup:
*emphasis*, **strong emphasis**, `interpreted text`, ``inline
literals``, standalone hyperlinks (http://www.python.org),
external hyperlinks (Python_), internal cross-references
(example_), footnote references ([1]_), citation references
([CIT2002]_), substitution references (|example|), and _`inline
internal targets`.

Paragraphs are separated by blank lines and are left-aligned.

  • Five types of lists:

    1. Bullet lists:

      - This is a bullet list.

- Bullets can be "-", "*", or "+".

    2. Enumerated lists:

      1. This is an enumerated list.

2. Enumerators may be arabic numbers, letters, or roman
   numerals.

    3. Definition lists:

      what
    Definition lists associate a term with a definition.

how
    The term is a one-line phrase, and the definition is one
    or more paragraphs or body elements, indented relative to
    the term.

    4. Field lists:

      :what: Field lists map field names to field bodies, like
       database records.  They are often part of an extension
       syntax.

:how: The field marker is a colon, the field name, and a
      colon.

      The field body may contain one or more body elements,
      indented relative to the field marker.

    5. Option lists, for listing command-line options:

      -a            command-line option "a"
-b file       options can have arguments
              and long descriptions
--long        options can be long also
--input=file  long options can also have
              arguments
/V            DOS/VMS-style options too

      There must be at least two spaces between the option and the description.

  • Literal blocks:

    Literal blocks are either indented or line-prefix-quoted blocks,
and indicated with a double-colon ("::") at the end of the
preceding paragraph (right here -->)::

    if literal_block:
        text = 'is left as-is'
        spaces_and_linebreaks = 'are preserved'
        markup_processing = None

  • Block quotes:

    Block quotes consist of indented body elements:

    This theory, that is mine, is mine.

    -- Anne Elk (Miss)

  • Doctest blocks:

    >>> print 'Python-specific usage examples; begun with ">>>"'
Python-specific usage examples; begun with ">>>"
>>> print '(cut and pasted from interactive Python sessions)'
(cut and pasted from interactive Python sessions)

  • Two syntaxes for tables:

    1. Grid tables; complete, but complex and verbose:

      +------------------------+------------+----------+
| Header row, column 1   | Header 2   | Header 3 |
+========================+============+==========+
| body row 1, column 1   | column 2   | column 3 |
+------------------------+------------+----------+
| body row 2             | Cells may span        |
+------------------------+-----------------------+

    2. Simple tables; easy and compact, but limited:

      ====================  ==========  ==========
Header row, column 1  Header 2    Header 3
====================  ==========  ==========
body row 1, column 1  column 2    column 3
body row 2            Cells may span columns
====================  ======================

  • Explicit markup blocks all begin with an explicit block marker, two periods and a space:

    • Footnotes:

      .. [1] A footnote contains body elements, consistently
   indented by at least 3 spaces.

    • Citations:

      .. [CIT2002] Just like a footnote, except the label is
   textual.

    • Hyperlink targets:

      .. _Python: http://www.python.org

.. _example:

The "_example" target above points to this paragraph.

    • Directives:

      .. image:: mylogo.png

    • Substitution definitions:

      .. |symbol here| image:: symbol.png

    • Comments:

      .. Comments begin with two dots and a space.  Anything may
   follow, except for the syntax of footnotes/citations,
   hyperlink targets, directives, or substitution definitions.

Syntax Details

Descriptions below list "doctree elements" (document tree element names; XML DTD generic identifiers) corresponding to syntax constructs. For details on the hierarchy of elements, please see The Docutils Document Tree and the Docutils Generic DTD XML document type definition.

Whitespace

Spaces are recommended for indentation, but tabs may also be used. Tabs will be converted to spaces. Tab stops are at every 8th column.

Other whitespace characters (form feeds [chr(12)] and vertical tabs [chr(11)]) are converted to single spaces before processing.

Blank Lines

Blank lines are used to separate paragraphs and other elements. Multiple successive blank lines are equivalent to a single blank line, except within literal blocks (where all whitespace is preserved). Blank lines may be omitted when the markup makes element separation unambiguous, in conjunction with indentation. The first line of a document is treated as if it is preceded by a blank line, and the last line of a document is treated as if it is followed by a blank line.

Indentation

Indentation is used to indicate, and is only significant in indicating:

  • multi-line contents of list items,
  • multiple body elements within a list item (including nested lists),
  • the definition part of a definition list item,
  • block quotes,
  • the extent of literal blocks, and
  • the extent of explicit markup blocks.

Any text whose indentation is less than that of the current level (i.e., unindented text or "dedents") ends the current level of indentation.

Since all indentation is significant, the level of indentation must be consistent. For example, indentation is the sole markup indicator for block quotes:

This is a top-level paragraph.

    This paragraph belongs to a first-level block quote.

    Paragraph 2 of the first-level block quote.

Multiple levels of indentation within a block quote will result in more complex structures:

This is a top-level paragraph.

    This paragraph belongs to a first-level block quote.

        This paragraph belongs to a second-level block quote.

Another top-level paragraph.

        This paragraph belongs to a second-level block quote.

    This paragraph belongs to a first-level block quote.  The
    second-level block quote above is inside this first-level
    block quote.

When a paragraph or other construct consists of more than one line of text, the lines must be left-aligned:

This is a paragraph.  The lines of
this paragraph are aligned at the left.

    This paragraph has problems.  The
lines are not left-aligned.  In addition
  to potential misinterpretation, warning
    and/or error messages will be generated
  by the parser.

Several constructs begin with a marker, and the body of the construct must be indented relative to the marker. For constructs using simple markers (bullet lists, enumerated lists, footnotes, citations, hyperlink targets, directives, and comments), the level of indentation of the body is determined by the position of the first line of text, which begins on the same line as the marker. For example, bullet list bodies must be indented by at least two columns relative to the left edge of the bullet:

- This is the first line of a bullet list
  item's paragraph.  All lines must align
  relative to the first line.  [1]_

      This indented paragraph is interpreted
      as a block quote.

Because it is not sufficiently indented,
this paragraph does not belong to the list
item.

.. [1] Here's a footnote.  The second line is aligned
   with the beginning of the footnote label.  The ".."
   marker is what determines the indentation.

For constructs using complex markers (field lists and option lists), where the marker may contain arbitrary text, the indentation of the first line after the marker determines the left edge of the body. For example, field lists may have very long markers (containing the field names):

:Hello: This field has a short field name, so aligning the field
        body with the first line is feasible.

:Number-of-African-swallows-required-to-carry-a-coconut: It would
    be very difficult to align the field body with the left edge
    of the first line.  It may even be preferable not to begin the
    body on the same line as the marker.

Escaping Mechanism

The character set universally available to plaintext documents, 7-bit ASCII, is limited. No matter what characters are used for markup, they will already have multiple meanings in written text. Therefore markup characters will sometimes appear in text without being intended as markup. Any serious markup system requires an escaping mechanism to override the default meaning of the characters used for the markup. In reStructuredText we use the backslash, commonly used as an escaping character in other domains.

A backslash followed by any character (except whitespace characters) escapes that character. The escaped character represents the character itself, and is prevented from playing a role in any markup interpretation. The backslash is removed from the output. A literal backslash is represented by two backslashes in a row (the first backslash "escapes" the second, preventing it being interpreted in an "escaping" role).

Backslash-escaped whitespace characters are removed from the document. This allows for character-level inline markup.

There are two contexts in which backslashes have no special meaning: literal blocks and inline literals. In these contexts, a single backslash represents a literal backslash, without having to double up.

Please note that the reStructuredText specification and parser do not address the issue of the representation or extraction of text input (how and in what form the text actually reaches the parser). Backslashes and other characters may serve a character-escaping purpose in certain contexts and must be dealt with appropriately. For example, Python uses backslashes in strings to escape certain characters, but not others. The simplest solution when backslashes appear in Python docstrings is to use raw docstrings:

r"""This is a raw docstring.  Backslashes (\) are not touched."""

Reference Names

Simple reference names are single words consisting of alphanumerics plus isolated (no two adjacent) internal hyphens, underscores, and periods; no whitespace or other characters are allowed. Footnote labels (Footnotes & Footnote References), citation labels (Citations & Citation References), interpreted text roles, and some hyperlink references use the simple reference name syntax.

Reference names using punctuation or whose names are phrases (two or more space-separated words) are called "phrase-references". Phrase-references are expressed by enclosing the phrase in backquotes and treating the backquoted text as a reference name:

Want to learn about `my favorite programming language`_?

.. _my favorite programming language: http://www.python.org

Simple reference names may also optionally use backquotes.

Reference names are whitespace-neutral and case-insensitive. When resolving reference names internally:

  • whitespace is normalized (one or more spaces, horizontal or vertical tabs, newlines, carriage returns, or form feeds, are interpreted as a single space), and
  • case is normalized (all alphabetic characters are converted to lowercase).

For example, the following hyperlink references are equivalent:

- `A HYPERLINK`_
- `a    hyperlink`_
- `A
  Hyperlink`_

Hyperlinks, footnotes, and citations all share the same namespace for reference names. The labels of citations (simple reference names) and manually-numbered footnotes (numbers) are entered into the same database as other hyperlink names. This means that a footnote (defined as ".. [1]") which can be referred to by a footnote reference ([1]_), can also be referred to by a plain hyperlink reference (1). Of course, each type of reference (hyperlink, footnote, citation) may be processed and rendered differently. Some care should be taken to avoid reference name conflicts.

Document Structure

Document

Doctree element: document.

The top-level element of a parsed reStructuredText document is the "document" element. After initial parsing, the document element is a simple container for a document fragment, consisting of body elements, transitions, and sections, but lacking a document title or other bibliographic elements. The code that calls the parser may choose to run one or more optional post-parse transforms, rearranging the document fragment into a complete document with a title and possibly other metadata elements (author, date, etc.; see Bibliographic Fields).

Specifically, there is no way to indicate a document title and subtitle explicitly in reStructuredText. Instead, a lone top-level section title (see Sections below) can be treated as the document title. Similarly, a lone second-level section title immediately after the "document title" can become the document subtitle. The rest of the sections are then lifted up a level or two. See the DocTitle transform for details.

Sections

Doctree elements: section, title.

Sections are identified through their titles, which are marked up with adornment: "underlines" below the title text, or underlines and matching "overlines" above the title. An underline/overline is a single repeated punctuation character that begins in column 1 and forms a line extending at least as far as the right edge of the title text. Specifically, an underline/overline character may be any non-alphanumeric printable 7-bit ASCII character 1. When an overline is used, the length and character used must match the underline. Underline-only adornment styles are distinct from overline-and-underline styles that use the same character. There may be any number of levels of section titles, although some output formats may have limits (HTML has 6 levels).

[1]

The following are all valid section title adornment characters:

! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~

Some characters are more suitable than others. The following are recommended:

= - ` : . ' " ~ ^ _ * + #

Rather than imposing a fixed number and order of section title adornment styles, the order enforced will be the order as encountered. The first style encountered will be an outermost title (like HTML H1), the second style will be a subtitle, the third will be a subsubtitle, and so on.

Below are examples of section title styles:

===============
 Section Title
===============

---------------
 Section Title
---------------

Section Title
=============

Section Title
-------------

Section Title
`````````````

Section Title
'''''''''''''

Section Title
.............

Section Title
~~~~~~~~~~~~~

Section Title
*************

Section Title
+++++++++++++

Section Title
^^^^^^^^^^^^^

When a title has both an underline and an overline, the title text may be inset, as in the first two examples above. This is merely aesthetic and not significant. Underline-only title text may not be inset.

A blank line after a title is optional. All text blocks up to the next title of the same or higher level are included in a section (or subsection, etc.).

All section title styles need not be used, nor need any specific section title style be used. However, a document must be consistent in its use of section titles: once a hierarchy of title styles is established, sections must use that hierarchy.

Each section title automatically generates a hyperlink target pointing to the section. The text of the hyperlink target (the "reference name") is the same as that of the section title. See Implicit Hyperlink Targets for a complete description.

Sections may contain body elements, transitions, and nested sections.

Transitions

Doctree element: transition.

Instead of subheads, extra space or a type ornament between paragraphs may be used to mark text divisions or to signal changes in subject or emphasis.

(The Chicago Manual of Style, 14th edition, section 1.80)

Transitions are commonly seen in novels and short fiction, as a gap spanning one or more lines, with or without a type ornament such as a row of asterisks. Transitions separate other body elements. A transition should not begin or end a section or document, nor should two transitions be immediately adjacent.

The syntax for a transition marker is a horizontal line of 4 or more repeated punctuation characters. The syntax is the same as section title underlines without title text. Transition markers require blank lines before and after:

Para.

----------

Para.

Unlike section title underlines, no hierarchy of transition markers is enforced, nor do differences in transition markers accomplish anything. It is recommended that a single consistent style be used.

The processing system is free to render transitions in output in any way it likes. For example, horizontal rules (<hr>) in HTML output would be an obvious choice.

Body Elements

Paragraphs

Doctree element: paragraph.

Paragraphs consist of blocks of left-aligned text with no markup indicating any other body element. Blank lines separate paragraphs from each other and from other body elements. Paragraphs may contain inline markup.

Syntax diagram:

+------------------------------+
| paragraph                    |
|                              |
+------------------------------+

+------------------------------+
| paragraph                    |
|                              |
+------------------------------+

Bullet Lists

Doctree elements: bullet_list, list_item.

A text block which begins with a "-", "*", or "+", followed by whitespace, is a bullet list item (a.k.a. "unordered" list item). List item bodies must be left-aligned and indented relative to the bullet; the text immediately after the bullet determines the indentation. For example:

- This is the first bullet list item.  The blank line above the
  first list item is required; blank lines between list items
  (such as below this paragraph) are optional.

- This is the first paragraph in the second item in the list.

  This is the second paragraph in the second item in the list.
  The blank line above this paragraph is required.  The left edge
  of this paragraph lines up with the paragraph above, both
  indented relative to the bullet.

  - This is a sublist.  The bullet lines up with the left edge of
    the text blocks above.  A sublist is a new list so requires a
    blank line above and below.

- This is the third item of the main list.

This paragraph is not part of the list.

Here are examples of incorrectly formatted bullet lists:

- This first line is fine.
A blank line is required between list items and paragraphs.
(Warning)

- The following line appears to be a new sublist, but it is not:
  - This is a paragraph continuation, not a sublist (since there's
    no blank line).  This line is also incorrectly indented.
  - Warnings may be issued by the implementation.

Syntax diagram:

+------+-----------------------+
| "- " | list item             |
+------| (body elements)+      |
       +-----------------------+

Enumerated Lists

Doctree elements: enumerated_list, list_item.

Enumerated lists (a.k.a. "ordered" lists) are similar to bullet lists, but use enumerators instead of bullets. An enumerator consists of an enumeration sequence member and formatting, followed by whitespace. The following enumeration sequences are recognized:

  • arabic numerals: 1, 2, 3, ... (no upper limit).
  • uppercase alphabet characters: A, B, C, ..., Z.
  • lower-case alphabet characters: a, b, c, ..., z.
  • uppercase Roman numerals: I, II, III, IV, ..., MMMMCMXCIX (4999).
  • lowercase Roman numerals: i, ii, iii, iv, ..., mmmmcmxcix (4999).

The following formatting types are recognized:

  • suffixed with a period: "1.", "A.", "a.", "I.", "i.".
  • surrounded by parentheses: "(1)", "(A)", "(a)", "(I)", "(i)".
  • suffixed with a right-parenthesis: "1)", "A)", "a)", "I)", "i)".

While parsing an enumerated list, a new list will be started whenever:

  • An enumerator is encountered which does not have the same format and sequence type as the current list (e.g. "1.", "(a)" produces two separate lists).
  • The enumerators are not in sequence (e.g., "1.", "3." produces two separate lists).

It is recommended that the enumerator of the first list item be ordinal-1 ("1", "A", "a", "I", or "i"). Although other start-values will be recognized, they may not be supported by the output format. A level-1 [info] system message will be generated for any list beginning with a non-ordinal-1 enumerator.

Lists using Roman numerals must begin with "I"/"i" or a multi-character value, such as "II" or "XV". Any other single-character Roman numeral ("V", "X", "L", "C", "D", "M") will be interpreted as a letter of the alphabet, not as a Roman numeral. Likewise, lists using letters of the alphabet may not begin with "I"/"i", since these are recognized as Roman numeral 1.

The second line of each enumerated list item is checked for validity. This is to prevent ordinary paragraphs from being mistakenly interpreted as list items, when they happen to begin with text identical to enumerators. For example, this text is parsed as an ordinary paragraph:

A. Einstein was a really
smart dude.

However, ambiguity cannot be avoided if the paragraph consists of only one line. This text is parsed as an enumerated list item:

A. Einstein was a really smart dude.

If a single-line paragraph begins with text identical to an enumerator ("A.", "1.", "(b)", "I)", etc.), the first character will have to be escaped in order to have the line parsed as an ordinary paragraph:

\A. Einstein was a really smart dude.

Nested enumerated lists must be created with indentation. For example:

1. Item 1.

   a) Item 1a.
   b) Item 1b.

Example syntax diagram:

+-------+----------------------+
| "1. " | list item            |
+-------| (body elements)+     |
        +----------------------+

Definition Lists

Doctree elements: definition_list, definition_list_item, term, classifier, definition.

Each definition list item contains a term, an optional classifier, and a definition. A term is a simple one-line word or phrase. An optional classifier may follow the term on the same line, after an inline " : " (space, colon, space). A definition is a block indented relative to the term, and may contain multiple paragraphs and other body elements. There may be no blank line between a term line and a definition block (this distinguishes definition lists from block quotes). Blank lines are required before the first and after the last definition list item, but are optional in-between. For example:

term 1
    Definition 1.

term 2
    Definition 2, paragraph 1.

    Definition 2, paragraph 2.

term 3 : classifier
    Definition 3.

Inline markup is parsed in the term line before the term/classifier delimiter (" : ") is recognized. The delimiter will only be recognized if it appears outside of any inline markup.

A definition list may be used in various ways, including:

  • As a dictionary or glossary. The term is the word itself, a classifier may be used to indicate the usage of the term (noun, verb, etc.), and the definition follows.
  • To describe program variables. The term is the variable name, a classifier may be used to indicate the type of the variable (string, integer, etc.), and the definition describes the variable's use in the program. This usage of definition lists supports the classifier syntax of Grouch, a system for describing and enforcing a Python object schema.

Syntax diagram:

+---------------------------+
| term [ " : " classifier ] |
+--+------------------------+--+
   | definition                |
   | (body elements)+          |
   +---------------------------+

Field Lists

Doctree elements: field_list, field, field_name, field_body.

Field lists are used as part of an extension syntax, such as options for directives, or database-like records meant for further processing. They may also be used for two-column table-like structures resembling database records (label & data pairs). Applications of reStructuredText may recognize field names and transform fields or field bodies in certain contexts. For examples, see Bibliographic Fields below, or the "image" and "meta" directives in reStructuredText Directives.

Field lists are mappings from field names to field bodies, modeled on RFC822 headers. A field name is made up of one or more letters, numbers, whitespace, and punctuation, except colons (":"). Inline markup is parsed in field names. Field names are case-insensitive when further processed or transformed. The field name, along with a single colon prefix and suffix, together form the field marker. The field marker is followed by whitespace and the field body. The field body may contain multiple body elements, indented relative to the field marker. The first line after the field name marker determines the indentation of the field body. For example:

:Date: 2001-08-16
:Version: 1
:Authors: - Me
          - Myself
          - I
:Indentation: Since the field marker may be quite long, the second
   and subsequent lines of the field body do not have to line up
   with the first line, but they must be indented relative to the
   field name marker, and they must line up with each other.
:Parameter i: integer

The interpretation of individual words in a multi-word field name is up to the application. The application may specify a syntax for the field name. For example, second and subsequent words may be treated as "arguments", quoted phrases may be treated as a single argument, and direct support for the "name=value" syntax may be added.

Standard RFC822 headers cannot be used for this construct because they are ambiguous. A word followed by a colon at the beginning of a line is common in written text. However, in well-defined contexts such as when a field list invariably occurs at the beginning of a document (PEPs and email messages), standard RFC822 headers could be used.

Syntax diagram (simplified):

+--------------------+----------------------+
| ":" field name ":" | field body           |
+-------+------------+                      |
        | (body elements)+                  |
        +-----------------------------------+

Bibliographic Fields

Doctree elements: docinfo, author, authors, organization, contact, version, status, date, copyright, field, topic.

When a field list is the first non-comment element in a document (after the document title, if there is one), it may have its fields transformed to document bibliographic data. This bibliographic data corresponds to the front matter of a book, such as the title page and copyright page.

Certain registered field names (listed below) are recognized and transformed to the corresponding doctree elements, most becoming child elements of the "docinfo" element. No ordering is required of these fields, although they may be rearranged to fit the document structure, as noted. Unless otherwise indicated below, each of the bibliographic elements' field bodies may contain a single paragraph only. Field bodies may be checked for RCS keywords and cleaned up. Any unrecognized fields will remain as generic fields in the docinfo element.

The registered bibliographic field names and their corresponding doctree elements are as follows:

  • Field name "Author": author element.
  • "Authors": authors.
  • "Organization": organization.
  • "Contact": contact.
  • "Address": address.
  • "Version": version.
  • "Status": status.
  • "Date": date.
  • "Copyright": copyright.
  • "Dedication": topic.
  • "Abstract": topic.

The "Authors" field may contain either: a single paragraph consisting of a list of authors, separated by ";" or ","; or a bullet list whose elements each contain a single paragraph per author. ";" is checked first, so "Doe, Jane; Doe, John" will work. In some languages (e.g. Swedish), there is no singular/plural distinction between "Author" and "Authors", so only an "Authors" field is provided, and a single name is interpreted as an "Author". If a single name contains a comma, end it with a semicolon to disambiguate: ":Authors: Doe, Jane;".

The "Address" field is for a multi-line surface mailing address. A specialized form of line block`_ (see reStructuredText Directives), newlines and whitespace will be preserved.

The "Dedication" and "Abstract" fields may contain arbitrary body elements. Only one of each is allowed. They become topic elements with "Dedication" or "Abstract" titles (or language equivalents) immediately following the docinfo element.

This field-name-to-element mapping can be replaced for other languages. See the DocInfo transform implementation documentation for details.

Unregistered/generic fields may contain one or more paragraphs or arbitrary body elements.

RCS Keywords

Bibliographic fields recognized by the parser are normally checked for RCS 2 keywords and cleaned up 3. RCS keywords may be entered into source files as "$keyword$", and once stored under RCS or CVS 4, they are expanded to "$keyword: expansion text $". For example, a "Status" field will be transformed to a "status" element:

:Status: $keyword: expansion text $
[2] Revision Control System.
[3] RCS keyword processing can be turned off (unimplemented).
[4] Concurrent Versions System. CVS uses the same keywords as RCS.

Processed, the "status" element's text will become simply "expansion text". The dollar sign delimiters and leading RCS keyword name are removed.

The RCS keyword processing only kicks in when both of these conditions hold:

  1. The field list is in bibliographic context (first non-comment construct in the document, after a document title if there is one).
  2. The field name is a recognized bibliographic field name.

Option Lists

Doctree elements: option_list, option_list_item, option_group, option, option_string, option_argument, description.

Option lists are two-column lists of command-line options and descriptions, documenting a program's options. For example:

-a         Output all.
-b         Output both (this description is
           quite long).
-c arg     Output just arg.
--long     Output all day long.

-p         This option has two paragraphs in the description.
           This is the first.

           This is the second.  Blank lines may be omitted between
           options (as above) or left in (as here and below).

--very-long-option  A VMS-style option.  Note the adjustment for
                    the required two spaces.

--an-even-longer-option
           The description can also start on the next line.

-2, --two  This option has two variants.

-f FILE, --file=FILE  These two options are synonyms; both have
                      arguments.

/V         A VMS/DOS-style option.

There are several types of options recognized by reStructuredText:

  • Short POSIX options consist of one dash and an option letter.
  • Long POSIX options consist of two dashes and an option word; some systems use a single dash.
  • Old GNU-style "plus" options consist of one plus and an option letter ("plus" options are deprecated now, their use discouraged).
  • DOS/VMS options consist of a slash and an option letter or word.

Please note that both POSIX-style and DOS/VMS-style options may be used by DOS or Windows software. These and other variations are sometimes used mixed together. The names above have been chosen for convenience only.

The syntax for short and long POSIX options is based on the syntax supported by Python's getopt.py module, which implements an option parser similar to the GNU libc getopt_long() function but with some restrictions. There are many variant option systems, and reStructuredText option lists do not support all of them.

Although long POSIX and DOS/VMS option words may be allowed to be truncated by the operating system or the application when used on the command line, reStructuredText option lists do not show or support this with any special syntax. The complete option word should be given, supported by notes about truncation if and when applicable.

Options may be followed by an argument placeholder, whose role and syntax should be explained in the description text. Either a space or an equals sign may be used as a delimiter between options and option argument placeholders; short options ("-" or "+" prefix only) may omit the delimiter. Option arguments may take one of two forms:

  • Begins with a letter ([a-zA-Z]) and subsequently consists of letters, numbers, underscores and hyphens ([a-zA-Z0-9_-]).
  • Begins with an open-angle-bracket (<) and ends with a close-angle-bracket (>); any characters except angle brackets are allowed internally.

Multiple option "synonyms" may be listed, sharing a single description. They must be separated by comma-space.

There must be at least two spaces between the option(s) and the description. The description may contain multiple body elements. The first line after the option marker determines the indentation of the description. As with other types of lists, blank lines are required before the first option list item and after the last, but are optional between option entries.

Syntax diagram (simplified):

+----------------------------+-------------+
| option [" " argument] "  " | description |
+-------+--------------------+             |
        | (body elements)+                 |
        +----------------------------------+

Literal Blocks

Doctree element: literal_block.

A paragraph consisting of two colons ("::") signifies that the following text block(s) comprise a literal block. The literal block must either be indented or quoted (see below). No markup processing is done within a literal block. It is left as-is, and is typically rendered in a monospaced typeface:

This is a typical paragraph.  An indented literal block follows.

::

    for a in [5,4,3,2,1]:   # this is program code, shown as-is
        print a
    print "it's..."
    # a literal block continues until the indentation ends

This text has returned to the indentation of the first paragraph,
is outside of the literal block, and is therefore treated as an
ordinary paragraph.

The paragraph containing only "::" will be completely removed from the output; no empty paragraph will remain.

As a convenience, the "::" is recognized at the end of any paragraph. If immediately preceded by whitespace, both colons will be removed from the output (this is the "partially minimized" form). When text immediately precedes the "::", one colon will be removed from the output, leaving only one colon visible (i.e., "::" will be replaced by ":"; this is the "fully minimized" form).

In other words, these are all equivalent (please pay attention to the colons after "Paragraph"):

  1. Expanded form:

    Paragraph:

::

    Literal block

  2. Partially minimized form:

    Paragraph: ::

    Literal block

  3. Fully minimized form:

    Paragraph::

    Literal block

All whitespace (including line breaks, but excluding minimum indentation for indented literal blocks) is preserved. Blank lines are required before and after a literal block, but these blank lines are not included as part of the literal block.

Indented Literal Blocks

Indented literal blocks are indicated by indentation relative to the surrounding text (leading whitespace on each line). The minimum indentation will be removed from each line of an indented literal block. The literal block need not be contiguous; blank lines are allowed between sections of indented text. The literal block ends with the end of the indentation.

Syntax diagram:

+------------------------------+
| paragraph                    |
| (ends with "::")             |
+------------------------------+
   +---------------------------+
   | indented literal block    |
   +---------------------------+

Quoted Literal Blocks

Quoted literal blocks are unindented contiguous blocks of text where each line begins with the same non-alphanumeric printable 7-bit ASCII character 5. A blank line ends a quoted literal block. The quoting characters are preserved in the processed document.

[5]

The following are all valid quoting characters:

! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~

Note that these are the same characters as are valid for title adornment of sections.

Possible uses include literate programming in Haskell and email quoting:

John Doe wrote::

>> Great idea!
>
> Why didn't I think of that?

You just did!  ;-)

Syntax diagram:

+------------------------------+
| paragraph                    |
| (ends with "::")             |
+------------------------------+
+------------------------------+
| ">" per-line-quoted          |
| ">" contiguous literal block |
+------------------------------+

Block Quotes

Doctree element: block_quote, attribution.

A text block that is indented relative to the preceding text, without markup indicating it to be a literal block, is a block quote. All markup processing (for body elements and inline markup) continues within the block quote:

This is an ordinary paragraph, introducing a block quote.

    "It is my business to know things.  That is my trade."

    -- Sherlock Holmes

If the final block of a block quote begins with "--", "---", or a true em-dash (flush left within the block quote), it is interpreted as an attribution. If the attribution consists of multiple lines, the left edges of the second and subsequent lines must align.

Blank lines are required before and after a block quote, but these blank lines are not included as part of the block quote.

Syntax diagram:

+------------------------------+
| (current level of            |
| indentation)                 |
+------------------------------+
   +---------------------------+
   | block quote               |
   | (body elements)+          |
   |                           |
   | -- attribution text       |
   |    (optional)             |
   +---------------------------+

Doctest Blocks

Doctree element: doctest_block.

Doctest blocks are interactive Python sessions cut-and-pasted into docstrings. They are meant to illustrate usage by example, and provide an elegant and powerful testing environment via the doctest module in the Python standard library.

Doctest blocks are text blocks which begin with ">>> ", the Python interactive interpreter main prompt, and end with a blank line. Doctest blocks are treated as a special case of literal blocks, without requiring the literal block syntax. If both are present, the literal block syntax takes priority over Doctest block syntax:

This is an ordinary paragraph.

>>> print 'this is a Doctest block'
this is a Doctest block

The following is a literal block::

    >>> This is not recognized as a doctest block by
    reStructuredText.  It *will* be recognized by the doctest
    module, though!

Indentation is not required for doctest blocks.

Tables

Doctree elements: table, tgroup, colspec, thead, tbody, row, entry.

ReStructuredText provides two syntaxes for delineating table cells: Grid Tables and Simple Tables.

As with other body elements, blank lines are required before and after tables. Tables' left edges should align with the left edge of preceding text blocks; if indented, the table is considered to be part of a block quote.

Once isolated, each table cell is treated as a miniature document; the top and bottom cell boundaries act as delimiting blank lines. Each cell contains zero or more body elements. Cell contents may include left and/or right margins, which are removed before processing.

Grid Tables

Grid tables provide a complete table representation via grid-like "ASCII art". Grid tables allow arbitrary cell contents (body elements), and both row and column spans. However, grid tables can be cumbersome to produce, especially for simple data sets. The Emacs table mode is a tool that allows easy editing of grid tables, in Emacs. See Simple Tables for a simpler (but limited) representation.

Grid tables are described with a visual grid made up of the characters "-", "=", "|", and "+". The hyphen ("-") is used for horizontal lines (row separators). The equals sign ("=") may be used to separate optional header rows from the table body (not supported by the Emacs table mode). The vertical bar ("|") is used for vertical lines (column separators). The plus sign ("+") is used for intersections of horizontal and vertical lines. Example:

+------------------------+------------+----------+----------+
| Header row, column 1   | Header 2   | Header 3 | Header 4 |
| (header rows optional) |            |          |          |
+========================+============+==========+==========+
| body row 1, column 1   | column 2   | column 3 | column 4 |
+------------------------+------------+----------+----------+
| body row 2             | Cells may span columns.          |
+------------------------+------------+---------------------+
| body row 3             | Cells may  | - Table cells       |
+------------------------+ span rows. | - contain           |
| body row 4             |            | - body elements.    |
+------------------------+------------+---------------------+

Some care must be taken with grid tables to avoid undesired interactions with cell text in rare cases. For example, the following table contains a cell in row 2 spanning from column 2 to column 4:

+--------------+----------+-----------+-----------+
| row 1, col 1 | column 2 | column 3  | column 4  |
+--------------+----------+-----------+-----------+
| row 2        |                                  |
+--------------+----------+-----------+-----------+
| row 3        |          |           |           |
+--------------+----------+-----------+-----------+

If a vertical bar is used in the text of that cell, it could have unintended effects if accidentally aligned with column boundaries:

+--------------+----------+-----------+-----------+
| row 1, col 1 | column 2 | column 3  | column 4  |
+--------------+----------+-----------+-----------+
| row 2        | Use the command ``ls | more``.   |
+--------------+----------+-----------+-----------+
| row 3        |          |           |           |
+--------------+----------+-----------+-----------+

Several solutions are possible. All that is needed is to break the continuity of the cell outline rectangle. One possibility is to shift the text by adding an extra space before:

+--------------+----------+-----------+-----------+
| row 1, col 1 | column 2 | column 3  | column 4  |
+--------------+----------+-----------+-----------+
| row 2        |  Use the command ``ls | more``.  |
+--------------+----------+-----------+-----------+
| row 3        |          |           |           |
+--------------+----------+-----------+-----------+

Another possibility is to add an extra line to row 2:

+--------------+----------+-----------+-----------+
| row 1, col 1 | column 2 | column 3  | column 4  |
+--------------+----------+-----------+-----------+
| row 2        | Use the command ``ls | more``.   |
|              |                                  |
+--------------+----------+-----------+-----------+
| row 3        |          |           |           |
+--------------+----------+-----------+-----------+

Simple Tables

Simple tables provide a compact and easy to type but limited row-oriented table representation for simple data sets. Cell contents are typically single paragraphs, although arbitrary body elements may be represented in most cells. Simple tables allow multi-line rows (in all but the first column) and column spans, but not row spans. See Grid Tables above for a complete table representation.

Simple tables are described with horizontal borders made up of "=" and "-" characters. The equals sign ("=") is used for top and bottom table borders, and to separate optional header rows from the table body. The hyphen ("-") is used to indicate column spans in a single row by underlining the joined columns.

A simple table begins with a top border of equals signs with one or more spaces at each column boundary (two or more spaces recommended). Regardless of spans, the top border must fully describe all table columns. There must be at least two columns in the table (to differentiate it from section headers). The last of the optional header rows is underlined with '=', again with spaces at column boundaries. There may not be a blank line below the header row separator; it would be interpreted as the bottom border of the table. The bottom boundary of the table consists of '=' underlines, also with spaces at column boundaries. For example, here is a truth table, a three-column table with one header row and four body rows:

=====  =====  =======
  A      B    A and B
=====  =====  =======
False  False  False
True   False  False
False  True   False
True   True   True
=====  =====  =======

Underlines of '-' may be used to indicate column spans by "filling in" column margins to join adjacent columns. Column span underlines must be complete (they must cover all columns) and align with established column boundaries. Text lines containing column span underlines may not contain any other text. A column span underline applies only to one row immediately above it. For example, here is a table with a column span in the header:

=====  =====  ======
   Inputs     Output
------------  ------
  A      B    A or B
=====  =====  ======
False  False  False
True   False  True
False  True   True
True   True   True
=====  =====  ======

Each line of text must contain spaces at column boundaries, except where cells have been joined by column spans. Each line of text starts a new row, except when there is a blank cell in the first column. In that case, that line of text is parsed as a continuation line. For this reason, cells in the first column of new rows (not continuation lines) must contain some text; blank cells would lead to a misinterpretation. An empty comment ("..") is sufficient and will be omitted from the processed output (see Comments below). Also, this mechanism limits cells in the first column to only one line of text. Use grid tables if this limitation is unacceptable.

Underlines of '-' may also be used to visually separate rows, even if there are no column spans. This is especially useful in long tables, where rows are many lines long.

Blank lines are permitted within simple tables. Their interpretation depends on the context. Blank lines between rows are ignored. Blank lines within multi-line rows may separate paragraphs or other body elements within cells.

The rightmost column is unbounded; text may continue past the edge of the table (as indicated by the table borders). However, it is recommended that borders be made long enough to contain the entire text.

The following example illustrates continuation lines (row 2 consists of two lines of text, and four lines for row 3), a blank line separating paragraphs (row 3, column 2), and text extending past the right edge of the table:

=====  =====
col 1  col 2
=====  =====
1      Second column of row 1.
2      Second column of row 2.
       Second line of paragraph.
3      - Second column of row 3.

       - Second item in bullet
         list (row 3, column 2).
=====  =====

Explicit Markup Blocks

An explicit markup block is a text block:

  • whose first line begins with ".." followed by whitespace (the "explicit markup start"),
  • whose second and subsequent lines (if any) are indented relative to the first, and
  • which ends before an unindented line.

Explicit markup blocks are analogous to bullet list items, with ".." as the bullet. The text immediately after the explicit markup start determines the indentation of the block body. Blank lines are required between explicit markup blocks and other elements, but are optional between explicit markup blocks where unambiguous.

The explicit markup syntax is used for footnotes, citations, hyperlink targets, directives, substitution definitions, and comments.

Footnotes

Doctree elements: footnote, label.

Each footnote consists of an explicit markup start (".. "), a left square bracket, the footnote label, a right square bracket, and whitespace, followed by indented body elements. A footnote label can be:

  • a whole decimal number consisting of one or more digits,
  • a single "#" (denoting auto-numbered footnotes),
  • a "#" followed by a simple reference name (an autonumber label), or
  • a single "*" (denoting auto-symbol footnotes).

If the first body element within a footnote is a simple paragraph, it may begin on the same line as the footnote label. Other elements must begin on a new line, consistently indented (by at least 3 spaces) and left-aligned.

Footnotes may occur anywhere in the document, not only at the end. Where or how they appear in the processed output depends on the processing system.

Here is a manually numbered footnote:

.. [1] Body elements go here.

Each footnote automatically generates a hyperlink target pointing to itself. The text of the hyperlink target name is the same as that of the footnote label. Auto-numbered footnotes generate a number as their footnote label and reference name. See Implicit Hyperlink Targets for a complete description of the mechanism.

Syntax diagram:

+-------+-------------------------+
| ".. " | "[" label "]" footnote  |
+-------+                         |
        | (body elements)+        |
        +-------------------------+
Auto-Numbered Footnotes

A number sign ("#") may be used as the first character of a footnote label to request automatic numbering of the footnote or footnote reference.

The first footnote to request automatic numbering is assigned the label "1", the second is assigned the label "2", and so on (assuming there are no manually numbered footnotes present; see Mixed Manual and Auto-Numbered Footnotes below). A footnote which has automatically received a label "1" generates an implicit hyperlink target with name "1", just as if the label was explicitly specified.

A footnote may specify a label explicitly while at the same time requesting automatic numbering: [#label]. These labels are called autonumber labels. Autonumber labels do two things:

  • On the footnote itself, they generate a hyperlink target whose name is the autonumber label (doesn't include the "#").

  • They allow an automatically numbered footnote to be referred to more than once, as a footnote reference or hyperlink reference. For example:

    If [#note]_ is the first footnote reference, it will show up as
"[1]".  We can refer to it again as [#note]_ and again see
"[1]".  We can also refer to it as note_ (an ordinary internal
hyperlink reference).

.. [#note] This is the footnote labeled "note".

The numbering is determined by the order of the footnotes, not by the order of the references. For footnote references without autonumber labels ([#]_), the footnotes and footnote references must be in the same relative order but need not alternate in lock-step. For example:

[#]_ is a reference to footnote 1, and [#]_ is a reference to
footnote 2.

.. [#] This is footnote 1.
.. [#] This is footnote 2.
.. [#] This is footnote 3.

[#]_ is a reference to footnote 3.

Special care must be taken if footnotes themselves contain auto-numbered footnote references, or if multiple references are made in close proximity. Footnotes and references are noted in the order they are encountered in the document, which is not necessarily the same as the order in which a person would read them.

Auto-Symbol Footnotes

An asterisk ("*") may be used for footnote labels to request automatic symbol generation for footnotes and footnote references. The asterisk may be the only character in the label. For example:

Here is a symbolic footnote reference: [*]_.

.. [*] This is the footnote.

A transform will insert symbols as labels into corresponding footnotes and footnote references. The number of references must be equal to the number of footnotes. One symbol footnote cannot have multiple references.

The standard Docutils system uses the following symbols for footnote marks 6:

  • asterisk/star ("*")
  • dagger (HTML character entity "&dagger;", Unicode U+02020)
  • double dagger ("&Dagger;"/U+02021)
  • section mark ("&sect;"/U+000A7)
  • pilcrow or paragraph mark ("&para;"/U+000B6)
  • number sign ("#")
  • spade suit ("&spades;"/U+02660)
  • heart suit ("&hearts;"/U+02665)
  • diamond suit ("&diams;"/U+02666)
  • club suit ("&clubs;"/U+02663)
[6] This list was inspired by the list of symbols for "Note Reference Marks" in The Chicago Manual of Style, 14th edition, section 12.51. "Parallels" ("||") were given in CMoS instead of the pilcrow. The last four symbols (the card suits) were added arbitrarily.

If more than ten symbols are required, the same sequence will be reused, doubled and then tripled, and so on ("**" etc.).

Note

When using auto-symbol footnotes, the choice of output encoding is important. Many of the symbols used are not encodable in certain common text encodings such as Latin-1 (ISO 8859-1). The use of UTF-8 for the output encoding is recommended. An alternative for HTML and XML output is to use the "xmlcharrefreplace" output encoding error handler (available in Python 2.3 & later).
Mixed Manual and Auto-Numbered Footnotes

Manual and automatic footnote numbering may both be used within a single document, although the results may not be expected. Manual numbering takes priority. Only unused footnote numbers are assigned to auto-numbered footnotes. The following example should be illustrative:

[2]_ will be "2" (manually numbered),
[#]_ will be "3" (anonymous auto-numbered), and
[#label]_ will be "1" (labeled auto-numbered).

.. [2] This footnote is labeled manually, so its number is fixed.

.. [#label] This autonumber-labeled footnote will be labeled "1".
   It is the first auto-numbered footnote and no other footnote
   with label "1" exists.  The order of the footnotes is used to
   determine numbering, not the order of the footnote references.

.. [#] This footnote will be labeled "3".  It is the second
   auto-numbered footnote, but footnote label "2" is already used.

Citations

Citations are identical to footnotes except that they use only non-numeric labels such as [note] or [GVR2001]. Citation labels are simple reference names (case-insensitive single words consisting of alphanumerics plus internal hyphens, underscores, and periods; no whitespace). Citations may be rendered separately and differently from footnotes. For example:

Here is a citation reference: [CIT2002]_.

.. [CIT2002] This is the citation.  It's just like a footnote,
   except the label is textual.

Hyperlink Targets

Doctree element: target.

These are also called explicit hyperlink targets, to differentiate them from implicit hyperlink targets defined below.

Hyperlink targets identify a location within or outside of a document, which may be linked to by hyperlink references.

Hyperlink targets may be named or anonymous. Named hyperlink targets consist of an explicit markup start (".. "), an underscore, the reference name (no trailing underscore), a colon, whitespace, and a link block:

.. _hyperlink-name: link-block

Reference names are whitespace-neutral and case-insensitive. See Reference Names for details and examples.

Anonymous hyperlink targets consist of an explicit markup start (".. "), two underscores, a colon, whitespace, and a link block; there is no reference name:

.. __: anonymous-hyperlink-target-link-block

An alternate syntax for anonymous hyperlinks consists of two underscores, a space, and a link block:

__ anonymous-hyperlink-target-link-block

See Anonymous Hyperlinks below.

There are three types of hyperlink targets: internal, external, and indirect.

  1. Internal hyperlink targets have empty link blocks. They provide an end point allowing a hyperlink to connect one place to another within a document. An internal hyperlink target points to the element following the target. For example:

    Clicking on this internal hyperlink will take us to the target_
below.

.. _target:

The hyperlink target above points to this paragraph.

    Internal hyperlink targets may be "chained". Multiple adjacent internal hyperlink targets all point to the same element:

    .. _target1:
.. _target2:

The targets "target1" and "target2" are synonyms; they both
point to this paragraph.

    If the element "pointed to" is an external hyperlink target (with a URI in its link block; see #2 below) the URI from the external hyperlink target is propagated to the internal hyperlink targets; they will all "point to" the same URI. There is no need to duplicate a URI. For example, all three of the following hyperlink targets refer to the same URI:

    .. _Python DOC-SIG mailing list archive:
.. _archive:
.. _Doc-SIG: http://mail.python.org/pipermail/doc-sig/

    An inline form of internal hyperlink target is available; see Inline Internal Targets.

  2. External hyperlink targets have an absolute or relative URI or email address in their link blocks. For example, take the following input:

    See the Python_ home page for info.

`Write to me`_ with your questions.

.. _Python: http://www.python.org
.. _Write to me: jdoe@example.com

    After processing into HTML, the hyperlinks might be expressed as:

    See the <a href="http://www.python.org">Python</a> home page
for info.

<a href="mailto:jdoe@example.com">Write to me</a> with your
questions.

    An external hyperlink's URI may begin on the same line as the explicit markup start and target name, or it may begin in an indented text block immediately following, with no intervening blank lines. If there are multiple lines in the link block, they are stripped of leading and trailing whitespace and concatenated. The following external hyperlink targets are equivalent:

    .. _one-liner: http://docutils.sourceforge.net/rst.html

.. _starts-on-this-line: http://
   docutils.sourceforge.net/rst.html

.. _entirely-below:
   http://docutils.
   sourceforge.net/rst.html

    If an external hyperlink target's URI contains an underscore as its last character, it must be escaped to avoid being mistaken for an indirect hyperlink target:

    This link_ refers to a file called ``underscore_``.

.. _link: underscore\_

    It is possible (although not generally recommended) to include URIs directly within hyperlink references. See Embedded URIs below.

  3. Indirect hyperlink targets have a hyperlink reference in their link blocks. In the following example, target "one" indirectly references whatever target "two" references, and target "two" references target "three", an internal hyperlink target. In effect, all three reference the same thing:

    .. _one: two_
.. _two: three_
.. _three:

    Just as with hyperlink references anywhere else in a document, if a phrase-reference is used in the link block it must be enclosed in backquotes. As with external hyperlink targets, the link block of an indirect hyperlink target may begin on the same line as the explicit markup start or the next line. It may also be split over multiple lines, in which case the lines are joined with whitespace before being normalized.

    For example, the following indirect hyperlink targets are equivalent:

    .. _one-liner: `A HYPERLINK`_
.. _entirely-below:
   `a    hyperlink`_
.. _split: `A
   Hyperlink`_

If a reference name contains a colon followed by whitespace, either:

  • the phrase must be enclosed in backquotes:

    .. _`FAQTS: Computers: Programming: Languages: Python`:
   http://python.faqts.com/

  • or the colon(s) must be backslash-escaped in the link target:

    .. _Chapter One\: "Tadpole Days":

It's not easy being green...

See Implicit Hyperlink Targets below for the resolution of duplicate reference names.

Syntax diagram:

+-------+----------------------+
| ".. " | "_" name ":" link    |
+-------+ block                |
        |                      |
        +----------------------+
Anonymous Hyperlinks

The World Wide Web Consortium recommends in its HTML Techniques for Web Content Accessibility Guidelines that authors should "clearly identify the target of each link." Hyperlink references should be as verbose as possible, but duplicating a verbose hyperlink name in the target is onerous and error-prone. Anonymous hyperlinks are designed to allow convenient verbose hyperlink references, and are analogous to Auto-Numbered Footnotes. They are particularly useful in short or one-off documents. However, this feature is easily abused and can result in unreadable plaintext and/or unmaintainable documents. Caution is advised.

Anonymous hyperlink references are specified with two underscores instead of one:

See `the web site of my favorite programming language`__.

Anonymous targets begin with ".. __:"; no reference name is required or allowed:

.. __: http://www.python.org

As a convenient alternative, anonymous targets may begin with "__" only:

__ http://www.python.org

The reference name of the reference is not used to match the reference to its target. Instead, the order of anonymous hyperlink references and targets within the document is significant: the first anonymous reference will link to the first anonymous target. The number of anonymous hyperlink references in a document must match the number of anonymous targets. For readability, it is recommended that targets be kept close to references. Take care when editing text containing anonymous references; adding, removing, and rearranging references require attention to the order of corresponding targets.

Directives

Doctree elements: depend on the directive.

Directives are an extension mechanism for reStructuredText, a way of adding support for new constructs without adding new primary syntax (directives may support additional syntax locally). All standard directives (those implemented and registered in the reference reStructuredText parser) are described in the reStructuredText Directives document, and are always available. Any other directives are domain-specific, and may require special action to make them available when processing the document.

For example, here's how an image may be placed:

.. image:: mylogo.jpeg

A figure (a graphic with a caption) may placed like this:

.. figure:: larch.png

   The larch.

An admonition (note, caution, etc.) contains other body elements:

.. note:: This is a paragraph

   - Here is a bullet list.

Directives are indicated by an explicit markup start (".. ") followed by the directive type, two colons, and whitespace (together called the "directive marker"). Directive types are case-insensitive single words (alphanumerics plus internal hyphens, underscores, and periods; no whitespace). Two colons are used after the directive type for these reasons:

  • Two colons are distinctive, and unlikely to be used in common text.

  • Two colons avoids clashes with common comment text like:

    .. Danger: modify at your own risk!

  • If an implementation of reStructuredText does not recognize a directive (i.e., the directive-handler is not installed), a level-3 (error) system message is generated, and the entire directive block (including the directive itself) will be included as a literal block. Thus "::" is a natural choice.

The directive block is consists of any text on the first line of the directive after the directive marker, and any subsequent indented text. The interpretation of the directive block is up to the directive code. There are three logical parts to the directive block:

  1. Directive arguments.
  2. Directive options.
  3. Directive content.

Individual directives can employ any combination of these parts. Directive arguments can be filesystem paths, URLs, title text, etc. Directive options are indicated using field lists; the field names and contents are directive-specific. Arguments and options must form a contiguous block beginning on the first or second line of the directive; a blank line indicates the beginning of the directive content block. If either arguments and/or options are employed by the directive, a blank line must separate them from the directive content. The "figure" directive employs all three parts:

.. figure:: larch.png
   :scale: 50

   The larch.

Simple directives may not require any content. If a directive that does not employ a content block is followed by indented text anyway, it is an error. If a block quote should immediately follow a directive, use an empty comment in-between (see Comments below).

Actions taken in response to directives and the interpretation of text in the directive content block or subsequent text block(s) are directive-dependent. See reStructuredText Directives for details.

Directives are meant for the arbitrary processing of their contents, which can be transformed into something possibly unrelated to the original text. It may also be possible for directives to be used as pragmas, to modify the behavior of the parser, such as to experiment with alternate syntax. There is no parser support for this functionality at present; if a reasonable need for pragma directives is found, they may be supported.

Directives do not generate "directive" elements; they are a parser construct only, and have no intrinsic meaning outside of reStructuredText. Instead, the parser will transform recognized directives into (possibly specialized) document elements. Unknown directives will trigger level-3 (error) system messages.

Syntax diagram:

+-------+-------------------------------+
| ".. " | directive type "::" directive |
+-------+ block                         |
        |                               |
        +-------------------------------+

Substitution Definitions

Doctree element: substitution_definition.

Substitution definitions are indicated by an explicit markup start (".. ") followed by a vertical bar, the substitution text, another vertical bar, whitespace, and the definition block. Substitution text may not begin or end with whitespace. A substitution definition block contains an embedded inline-compatible directive (without the leading ".. "), such as an image. For example:

The |biohazard| symbol must be used on containers used to
dispose of medical waste.

.. |biohazard| image:: biohazard.png

It is an error for a substitution definition block to directly or indirectly contain a circular substitution reference.

Substitution references are replaced in-line by the processed contents of the corresponding definition (linked by matching substitution text). Matches are case-sensitive but forgiving; if no exact match is found, a case-insensitive comparison is attempted.

Substitution definitions allow the power and flexibility of block-level directives to be shared by inline text. They are a way to include arbitrarily complex inline structures within text, while keeping the details out of the flow of text. They are the equivalent of SGML/XML's named entities or programming language macros.

Without the substitution mechanism, every time someone wants an application-specific new inline structure, they would have to petition for a syntax change. In combination with existing directive syntax, any inline structure can be coded without new syntax (except possibly a new directive).

Syntax diagram:

+-------+-----------------------------------------------------+
| ".. " | "|" substitution text "| " directive type "::" data |
+-------+ directive block                                     |
        |                                                     |
        +-----------------------------------------------------+

Following are some use cases for the substitution mechanism. Please note that most of the embedded directives shown are examples only and have not been implemented.

Objects

Substitution references may be used to associate ambiguous text with a unique object identifier.

For example, many sites may wish to implement an inline "user" directive:

|Michael| and |Jon| are our widget-wranglers.

.. |Michael| user:: mjones
.. |Jon|     user:: jhl

Depending on the needs of the site, this may be used to index the document for later searching, to hyperlink the inline text in various ways (mailto, homepage, mouseover Javascript with profile and contact information, etc.), or to customize presentation of the text (include username in the inline text, include an icon image with a link next to the text, make the text bold or a different color, etc.).

The same approach can be used in documents which frequently refer to a particular type of objects with unique identifiers but ambiguous common names. Movies, albums, books, photos, court cases, and laws are possible. For example:

|The Transparent Society| offers a fascinating alternate view
on privacy issues.

.. |The Transparent Society| book:: isbn=0738201448

Classes or functions, in contexts where the module or class names are unclear and/or interpreted text cannot be used, are another possibility:

4XSLT has the convenience method |runString|, so you don't
have to mess with DOM objects if all you want is the
transformed output.

.. |runString| function:: module=xml.xslt class=Processor
Images

Images are a common use for substitution references:

West led the |H| 3, covered by dummy's |H| Q, East's |H| K,
and trumped in hand with the |S| 2.

.. |H| image:: /images/heart.png
   :height: 11
   :width: 11
.. |S| image:: /images/spade.png
   :height: 11
   :width: 11

* |Red light| means stop.
* |Green light| means go.
* |Yellow light| means go really fast.

.. |Red light|    image:: red_light.png
.. |Green light|  image:: green_light.png
.. |Yellow light| image:: yellow_light.png

|-><-| is the official symbol of POEE_.

.. |-><-| image:: discord.png
.. _POEE: http://www.poee.org/

The "image" directive has been implemented.

Styles 7

Substitution references may be used to associate inline text with an externally defined presentation style:

Even |the text in Texas| is big.

.. |the text in Texas| style:: big

The style name may be meaningful in the context of some particular output format (CSS class name for HTML output, LaTeX style name for LaTeX, etc), or may be ignored for other output formats (such as plaintext).

[7] There may be sufficient need for a "style" mechanism to warrant simpler syntax such as an extension to the interpreted text role syntax. The substitution mechanism is cumbersome for simple text styling.
Templates

Inline markup may be used for later processing by a template engine. For example, a Zope author might write:

Welcome back, |name|!

.. |name| tal:: replace user/getUserName

After processing, this ZPT output would result:

Welcome back,
<span tal:replace="user/getUserName">name</span>!

Zope would then transform this to something like "Welcome back, David!" during a session with an actual user.

Replacement text

The substitution mechanism may be used for simple macro substitution. This may be appropriate when the replacement text is repeated many times throughout one or more documents, especially if it may need to change later. A short example is unavoidably contrived:

|RST| is a little annoying to type over and over, especially
when writing about |RST| itself, and spelling out the
bicapitalized word |RST| every time isn't really necessary for
|RST| source readability.

.. |RST| replace:: reStructuredText_
.. _reStructuredText: http://docutils.sourceforge.net/rst.html

Substitution is also appropriate when the replacement text cannot be represented using other inline constructs, or is obtrusively long:

But still, that's nothing compared to a name like
|j2ee-cas|__.

.. |j2ee-cas| replace::
   the Java `TM`:super: 2 Platform, Enterprise Edition Client
   Access Services
__ http://developer.java.sun.com/developer/earlyAccess/
   j2eecas/

Comments

Doctree element: comment.

Arbitrary indented text may follow the explicit markup start and will be processed as a comment element. No further processing is done on the comment block text; a comment contains a single "text blob". Depending on the output formatter, comments may be removed from the processed output. The only restriction on comments is that they not use the same syntax as any of the other explicit markup constructs: substitution definitions, directives, footnotes, citations, or hyperlink targets. To ensure that none of the other explicit markup constructs is recognized, leave the ".." on a line by itself:

.. This is a comment
..
   _so: is this!
..
   [and] this!
..
   this:: too!
..
   |even| this:: !

A explicit markup start followed by a blank line and nothing else (apart from whitespace) is an "empty comment". It serves to terminate a preceding construct, and does not consume any indented text following. To have a block quote follow a list or any indented construct, insert an unindented empty comment in-between.

Syntax diagram:

+-------+----------------------+
| ".. " | comment              |
+-------+ block                |
        |                      |
        +----------------------+

Implicit Hyperlink Targets

Implicit hyperlink targets are generated by section titles, footnotes, and citations, and may also be generated by extension constructs. Implicit hyperlink targets otherwise behave identically to explicit hyperlink targets.

Problems of ambiguity due to conflicting duplicate implicit and explicit reference names are avoided by following this procedure:

  1. Explicit hyperlink targets override any implicit targets having the same reference name. The implicit hyperlink targets are removed, and level-1 (info) system messages are inserted.
  2. Duplicate implicit hyperlink targets are removed, and level-1 (info) system messages inserted. For example, if two or more sections have the same title (such as "Introduction" subsections of a rigidly-structured document), there will be duplicate implicit hyperlink targets.
  3. Duplicate explicit hyperlink targets are removed, and level-2 (warning) system messages are inserted. Exception: duplicate external hyperlink targets (identical hyperlink names and referenced URIs) do not conflict, and are not removed.

System messages are inserted where target links have been removed. See "Error Handling" in PEP 258.

The parser must return a set of unique hyperlink targets. The calling software (such as the Docutils) can warn of unresolvable links, giving reasons for the messages.

Inline Markup

In reStructuredText, inline markup applies to words or phrases within a text block. The same whitespace and punctuation that serves to delimit words in written text is used to delimit the inline markup syntax constructs. The text within inline markup may not begin or end with whitespace. Arbitrary character-level inline markup is supported although not encouraged. Inline markup cannot be nested.

There are nine inline markup constructs. Five of the constructs use identical start-strings and end-strings to indicate the markup:

  • emphasis: "*"
  • strong emphasis: "**"
  • interpreted text: "`"
  • inline literals: "``"
  • substitution references: "|"

Three constructs use different start-strings and end-strings:

  • inline internal targets: "_`" and "`"
  • footnote references: "[" and "]_"
  • hyperlink references: "`" and "`_" (phrases), or just a trailing "_" (single words)

Standalone hyperlinks are recognized implicitly, and use no extra markup.

The inline markup start-string and end-string recognition rules are as follows. If any of the conditions are not met, the start-string or end-string will not be recognized or processed.

  1. Inline markup start-strings must start a text block or be immediately preceded by whitespace or one of:

    ' " ( [ { < - / :

  2. Inline markup start-strings must be immediately followed by non-whitespace.

  3. Inline markup end-strings must be immediately preceded by non-whitespace.

  4. Inline markup end-strings must end a text block or be immediately followed by whitespace or one of:

    ' " ) ] } > - / : . , ; ! ? \

  5. If an inline markup start-string is immediately preceded by a single or double quote, "(", "[", "{", or "<", it must not be immediately followed by the corresponding single or double quote, ")", "]", "}", or ">".

  6. An inline markup end-string must be separated by at least one character from the start-string.

  7. An unescaped backslash preceding a start-string or end-string will disable markup recognition, except for the end-string of inline literals. See Escaping Mechanism above for details.

For example, none of the following are recognized as containing inline markup start-strings:

  • asterisks: * "*" '*' (*) (* [*] {*} 1*x BOM32_*
  • double asterisks: ** a**b O(N**2) etc.
  • backquotes: ` `` etc.
  • underscores: _ __ __init__ __init__() etc.
  • vertical bars: | || etc.

It may be desirable to use inline literals for some of these anyhow, especially if they represent code snippets. It's a judgment call.

These cases do require either literal-quoting or escaping to avoid misinterpretation:

*4, class_, *args, **kwargs, `TeX-quoted', *ML, *.txt

The inline markup recognition rules were devised intentionally to allow 90% of non-markup uses of "*", "`", "_", and "|" without resorting to backslashes. For 9 of the remaining 10%, use inline literals or literal blocks:

"``\*``" -> "\*" (possibly in another font or quoted)

Only those who understand the escaping and inline markup rules should attempt the remaining 1%. ;-)

Inline markup delimiter characters are used for multiple constructs, so to avoid ambiguity there must be a specific recognition order for each character. The inline markup recognition order is as follows:

  • Asterisks: Strong emphasis ("**") is recognized before emphasis ("*").
  • Backquotes: Inline literals ("``"), inline internal targets (leading "_`", trailing "`"), are mutually independent, and are recognized before phrase hyperlink references (leading "`", trailing "`_") and interpreted text ("`").
  • Trailing underscores: Footnote references ("[" + label + "]_") and simple hyperlink references (name + trailing "_") are mutually independent.
  • Vertical bars: Substitution references ("|") are independently recognized.
  • Standalone hyperlinks are the last to be recognized.

Character-Level Inline Markup

It is possible to mark up individual characters within a word with backslash escapes (see Escaping Mechanism above). Backslash escapes can be used to allow arbitrary text to immediately follow inline markup:

Python ``list``\s use square bracket syntax.

The backslash will disappear from the processed document. The word "list" will appear as inline literal text, and the letter "s" will immediately follow it as normal text, with no space in-between.

Arbitrary text may immediately precede inline markup using backslash-escaped whitespace:

Possible in *re*\ ``Structured``\ *Text*, though not encouraged.

The backslashes and spaces separating "re", "Structured", and "Text" above will disappear from the processed document.

Caution!

The use of backslash-escapes for character-level inline markup is not encouraged. Such use is ugly and detrimental to the unprocessed document's readability. Please use this feature sparingly and only where absolutely necessary.

Emphasis

Doctree element: emphasis.

Start-string = end-string = "*".

Text enclosed by single asterisk characters is emphasized:

This is *emphasized text*.

Emphasized text is typically displayed in italics.

Strong Emphasis

Doctree element: strong.

Start-string = end-string = "**".

Text enclosed by double-asterisks is emphasized strongly:

This is **strong text**.

Strongly emphasized text is typically displayed in boldface.

Interpreted Text

Doctree element: depends on the explicit or implicit role and processing.

Start-string = end-string = "`".

Interpreted text is text that is meant to be related, indexed, linked, summarized, or otherwise processed, but the text itself is typically left alone. Interpreted text is enclosed by single backquote characters:

This is `interpreted text`.

The "role" of the interpreted text determines how the text is interpreted. The role may be inferred implicitly (as above; the "default role" is used) or indicated explicitly, using a role marker. A role marker consists of a colon, the role name, and another colon. A role name is a single word consisting of alphanumerics plus internal hyphens, underscores, and periods; no whitespace or other characters are allowed. A role marker is either a prefix or a suffix to the interpreted text, whichever reads better; it's up to the author:

:role:`interpreted text`

`interpreted text`:role:

Interpreted text allows extensions to the available inline descriptive markup constructs. To emphasis, strong emphasis, inline literals, and hyperlink references, we can add "title reference", "index entry", "acronym", "class", "red", "blinking" or anything else we want. Only pre-determined roles are recognized; unknown roles will generate errors. A core set of standard roles is implemented in the reference parser; see reStructuredText Interpreted Text Roles for individual descriptions. In addition, applications may support specialized roles.

Inline Literals

Doctree element: literal.

Start-string = end-string = "``".

Text enclosed by double-backquotes is treated as inline literals:

This text is an example of ``inline literals``.

Inline literals may contain any characters except two adjacent backquotes in an end-string context (according to the recognition rules above). No markup interpretation (including backslash-escape interpretation) is done within inline literals.

Line breaks are not preserved in inline literals. Although a reStructuredText parser will preserve runs of spaces in its output, the final representation of the processed document is dependent on the output formatter, thus the preservation of whitespace cannot be guaranteed. If the preservation of line breaks and/or other whitespace is important, literal blocks should be used.

Inline literals are useful for short code snippets. For example:

The regular expression ``[+-]?(\d+(\.\d*)?|\.\d+)`` matches
floating-point numbers (without exponents).

Hyperlink References

Doctree element: reference.

  • Named hyperlink references:
    • Start-string = "" (empty string), end-string = "_".
    • Start-string = "`", end-string = "`_". (Phrase references.)
  • Anonymous hyperlink references:
    • Start-string = "" (empty string), end-string = "__".
    • Start-string = "`", end-string = "`__". (Phrase references.)

Hyperlink references are indicated by a trailing underscore, "_", except for standalone hyperlinks which are recognized independently. The underscore can be thought of as a right-pointing arrow. The trailing underscores point away from hyperlink references, and the leading underscores point toward hyperlink targets.

Hyperlinks consist of two parts. In the text body, there is a source link, a reference name with a trailing underscore (or two underscores for anonymous hyperlinks):

See the Python_ home page for info.

A target link with a matching reference name must exist somewhere else in the document. See Hyperlink Targets for a full description).

Anonymous hyperlinks (which see) do not use reference names to match references to targets, but otherwise behave similarly to named hyperlinks.

Embedded URIs

A hyperlink reference may directly embed a target URI inline, within angle brackets ("<...>") as follows:

See the `Python home page <http://www.python.org>`_ for info.

This is exactly equivalent to:

See the `Python home page`_ for info.

.. _Python home page: http://www.python.org

The bracketed URI must be preceded by whitespace and be the last text before the end string. With a single trailing underscore, the reference is named and the same target URI may be referred to again.

With two trailing underscores, the reference and target are both anonymous, and the target cannot be referred to again. These are "one-off" hyperlinks. For example:

`RFC 2396 <http://www.rfc-editor.org/rfc/rfc2396.txt>`__ and `RFC
2732 <http://www.rfc-editor.org/rfc/rfc2732.txt>`__ together
define the syntax of URIs.

Equivalent to:

`RFC 2396`__ and `RFC 2732`__ together define the syntax of URIs.

__ http://www.rfc-editor.org/rfc/rfc2396.txt
__ http://www.rfc-editor.org/rfc/rfc2732.txt

If reference text happens to end with angle-bracketed text that is not a URI, the open-angle-bracket needs to be backslash-escaped. For example, here is a reference to a title describing a tag:

See `HTML Element: \<a>`_ below.

The reference text may also be omitted, in which case the URI will be duplicated for use as the reference text. This is useful for relative URIs where the address or file name is also the desired reference text:

See `<a_named_relative_link>`_ or `<an_anonymous_relative_link>`__
for details.

Caution!

This construct offers easy authoring and maintenance of hyperlinks at the expense of general readability. Inline URIs, especially long ones, inevitably interrupt the natural flow of text. For documents meant to be read in source form, the use of independent block-level hyperlink targets is strongly recommended. The embedded URI construct is most suited to documents intended only to be read in processed form.

Inline Internal Targets

Doctree element: target.

Start-string = "_`", end-string = "`".

Inline internal targets are the equivalent of explicit internal hyperlink targets, but may appear within running text. The syntax begins with an underscore and a backquote, is followed by a hyperlink name or phrase, and ends with a backquote. Inline internal targets may not be anonymous.

For example, the following paragraph contains a hyperlink target named "Norwegian Blue":

Oh yes, the _`Norwegian Blue`.  What's, um, what's wrong with it?

See Implicit Hyperlink Targets for the resolution of duplicate reference names.

Footnote References

Doctree element: footnote_reference.

Start-string = "[", end-string = "]_".

Each footnote reference consists of a square-bracketed label followed by a trailing underscore. Footnote labels are one of:

  • one or more digits (i.e., a number),
  • a single "#" (denoting auto-numbered footnotes),
  • a "#" followed by a simple reference name (an autonumber label), or
  • a single "*" (denoting auto-symbol footnotes).

For example:

Please RTFM [1]_.

.. [1] Read The Fine Manual

Citation References

Doctree element: citation_reference.

Start-string = "[", end-string = "]_".

Each citation reference consists of a square-bracketed label followed by a trailing underscore. Citation labels are simple reference names (case-insensitive single words, consisting of alphanumerics plus internal hyphens, underscores, and periods; no whitespace).

For example:

Here is a citation reference: [CIT2002]_.

See Citations for the citation itself.

Substitution References

Doctree element: substitution_reference, reference.

Start-string = "|", end-string = "|" (optionally followed by "_" or "__").

Vertical bars are used to bracket the substitution reference text. A substitution reference may also be a hyperlink reference by appending a "_" (named) or "__" (anonymous) suffix; the substitution text is used for the reference text in the named case.

The processing system replaces substitution references with the processed contents of the corresponding substitution definitions (which see for the definition of "correspond"). Substitution definitions produce inline-compatible elements.

Examples:

This is a simple |substitution reference|.  It will be replaced by
the processing system.

This is a combination |substitution and hyperlink reference|_.  In
addition to being replaced, the replacement text or element will
refer to the "substitution and hyperlink reference" target.

Standalone Hyperlinks

Doctree element: reference.

Start-string = end-string = "" (empty string).

A URI (absolute URI 8 or standalone email address) within a text block is treated as a general external hyperlink with the URI itself as the link's text. For example:

See http://www.python.org for info.

would be marked up in HTML as:

See <a href="http://www.python.org">http://www.python.org</a> for
info.

Two forms of URI are recognized:

  1. Absolute URIs. These consist of a scheme, a colon (":"), and a scheme-specific part whose interpretation depends on the scheme.

    The scheme is the name of the protocol, such as "http", "ftp", "mailto", or "telnet". The scheme consists of an initial letter, followed by letters, numbers, and/or "+", "-", ".". Recognition is limited to known schemes, per the Official IANA Registry of URI Schemes and the W3C's Retired Index of WWW Addressing Schemes.

    The scheme-specific part of the resource identifier may be either hierarchical or opaque:

    • Hierarchical identifiers begin with one or two slashes and may use slashes to separate hierarchical components of the path. Examples are web pages and FTP sites:

      http://www.python.org

ftp://ftp.python.org/pub/python

    • Opaque identifiers do not begin with slashes. Examples are email addresses and newsgroups:

      mailto:someone@somewhere.com

news:comp.lang.python

    With queries, fragments, and %-escape sequences, URIs can become quite complicated. A reStructuredText parser must be able to recognize any absolute URI, as defined in RFC2396 and RFC2732.

  2. Standalone email addresses, which are treated as if they were absolute URIs with a "mailto:" scheme. Example:

    someone@somewhere.com

Punctuation at the end of a URI is not considered part of the URI, unless the URI is terminated by a closing angle bracket (">"). Backslashes may be used in URIs to escape markup characters, specifically asterisks ("*") and underscores ("_") which are vaid URI characters (see Escaping Mechanism above).

[8] Uniform Resource Identifier. URIs are a general form of URLs (Uniform Resource Locators). For the syntax of URIs see RFC2396 and RFC2732.

Error Handling

Doctree element: system_message, problematic.

Markup errors are handled according to the specification in PEP 258.

View document source. Generated on: 2004-07-01 22:48 UTC. Generated by Docutils from reStructuredText source.


reStructuredText Interpreted Text Roles

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.14
Date: | 2004-06-03
Copyright: | This document has been placed in the public domain.

This document describes the interpreted text roles implemented in the reference reStructuredText parser.

Interpreted text uses backquotes (`) around the text. An explicit role marker may optionally appear before or after the text, delimited with colons. For example:

This is `interpreted text` using the default role.

This is :title:`interpreted text` using an explicit role.

A default role may be defined by applications of reStructuredText; it is used if no explicit :role: prefix or suffix is given. The "default default role" is :title-reference:.

See the Interpreted Text section in the reStructuredText Markup Specification for syntax details. For details on the hierarchy of elements, please see The Docutils Document Tree and the Docutils Generic DTD XML document type definition. For interpreted text role implementation details, see Creating reStructuredText Interpreted Text Roles.

Contents

  • Customization
  • Standard Interpreted Text Roles
    • :emphasis:
    • :literal:
    • :pep-reference:
    • :rfc-reference:
    • :strong:
    • :subscript:
    • :superscript:
    • :title-reference:

Customization

Custom interpreted text roles may be defined in a document with the "role" directive. Customization details are listed with each role.

The class option is recognized by the "role" directive for most interpreted text roles. A description is provided by "role" directive documentation.

Standard Interpreted Text Roles

:emphasis:

Aliases: None
DTD Element: emphasis
Customization:
Options: | class.
Content: | None.

Implements emphasis. These are equivalent:

*text*
:emphasis:`text`

:literal:

Aliases: None
DTD Element: literal
Customization:
Options: | class.
Content: | None.

Implements inline literal text. These are equivalent:

``text``
:literal:`text`

Care must be taken with backslash-escapes though. These are not equivalent:

``text \ and \ backslashes``
:literal:`text \ and \ backslashes`

The backslashes in the first line are preserved (and do nothing), whereas the backslashes in the second line escape the following spaces.

:pep-reference:

Aliases: :PEP:
DTD Element: reference
Customization:
Options: | class.
Content: | None.

The :pep-reference: role is used to create an HTTP reference to a PEP (Python Enhancement Proposal). The :PEP: alias is usually used. For example:

See :PEP:`287` for more information about reStructuredText.

This is equivalent to:

See `PEP 287`__ for more information about reStructuredText.

__ http://www.python.org/peps/pep-0287.html

:rfc-reference:

Aliases: :RFC:
DTD Element: reference
Customization:
Options: | class.
Content: | None.

The :rfc-reference: role is used to create an HTTP reference to an RFC (Internet Request for Comments). The :RFC: alias is usually used. For example:

See :RFC:`2822` for information about email headers.

This is equivalent to:

See `RFC 2822`__ for information about email headers.

__ http://www.faqs.org/rfcs/rfc2822.html

:strong:

Aliases: None
DTD Element: strong
Customization:
Options: | class.
Content: | None.

Implements strong emphasis. These are equivalent:

**text**
:strong:`text`

:subscript:

Aliases: :sub:
DTD Element: subscript
Customization:
Options: | class.
Content: | None.

Implements subscripts.

Tip

Whitespace or punctuation is required around interpreted text, but often not desired with subscripts & superscripts. Backslash-escaped whitespace can be used; the whitespace will be removed from the processed document:

H\ :sub:`2`\ O
E = mc\ :sup:`2`

In such cases, readability of the plain text can be greatly improved with substitutions:

The chemical formula for pure water is |H2O|.

.. |H2O| replace:: H\ :sub:`2`\ O

See the reStructuredText spec for further information on character-level markup and the substitution mechanism.

:superscript:

Aliases: :sup:
DTD Element: superscript
Customization:
Options: | class.
Content: | None.

Implements superscripts. See the tip in :subscript: above.

:title-reference:

Aliases: :title:, :t:.
DTD Element: title_reference
Customization:
Options: | class.
Content: | None.

The :title-reference: role is used to describe the titles of books, periodicals, and other materials. It is the equivalent of the HTML "cite" element, and it is expected that HTML writers will typically render "title_reference" elements using "cite".

Since title references are typically rendered with italics, they are often marked up using *emphasis*, which is misleading and vague. The "title_reference" element provides accurate and unambiguous descriptive markup.

Let's assume :title-reference: is the default interpreted text role (see below) for this example:

`Design Patterns` [GoF95]_ is an excellent read.

The following document fragment (pseudo-XML) will result from processing:

<paragraph>
    <title_reference>
        Design Patterns

    <citation_reference refname="gof95">
        GoF95
     is an excellent read.

:title-reference: is the default interpreted text role in the standard reStructuredText parser. This means that no explicit role is required. Applications of reStructuredText may designate a different default role, in which case the explicit :title-reference: role must be used to obtain a title_reference element.

View document source. Generated on: 2004-07-01 22:50 UTC. Generated by Docutils from reStructuredText source.


PEP 256 -- Docstring Processing System Framework

PEP: | 256
Title: | Docstring Processing System Framework
Version: | 1.9
Last-Modified: | 2004/06/05 02:55:11
Author: | David Goodger <goodger at users.sourceforge.net>
Discussions-To: | <doc-sig at python.org>
Status: | Draft
Type: | Standards Track
Content-Type: | text/x-rst
Created: | 01-Jun-2001
Post-History: | 13-Jun-2001

Contents

  • Abstract
  • Road Map to the Docstring PEPs
  • Rationale
    • PyDoc & Other Existing Systems
  • Specification
  • Project Web Site
  • References and Footnotes
  • Copyright
  • Acknowledgements

Abstract

Python lends itself to inline documentation. With its built-in docstring syntax, a limited form of Literate Programming [4] is easy to do in Python. However, there are no satisfactory standard tools for extracting and processing Python docstrings. The lack of a standard toolset is a significant gap in Python's infrastructure; this PEP aims to fill the gap.

The issues surrounding docstring processing have been contentious and difficult to resolve. This PEP proposes a generic Docstring Processing System (DPS) framework, which separates out the components (program and conceptual), enabling the resolution of individual issues either through consensus (one solution) or through divergence (many). It promotes standard interfaces which will allow a variety of plug-in components (input context readers, markup parsers, and output format writers) to be used.

The concepts of a DPS framework are presented independently of implementation details.

Road Map to the Docstring PEPs

There are many aspects to docstring processing. The "Docstring PEPs" have broken up the issues in order to deal with each of them in isolation, or as close as possible. The individual aspects and associated PEPs are as follows:

  • Docstring syntax. PEP 287, "reStructuredText Docstring Format" [1], proposes a syntax for Python docstrings, PEPs, and other uses.
  • Docstring semantics consist of at least two aspects:
    • Conventions: the high-level structure of docstrings. Dealt with in PEP 257, "Docstring Conventions" [2].
    • Methodology: rules for the informational content of docstrings. Not addressed.
  • Processing mechanisms. This PEP (PEP 256) outlines the high-level issues and specification of an abstract docstring processing system (DPS). PEP 258, "Docutils Design Specification" [3], is an overview of the design and implementation of one DPS under development.
  • Output styles: developers want the documentation generated from their source code to look good, and there are many different ideas about what that means. PEP 258 touches on "Stylist Transforms". This aspect of docstring processing has yet to be fully explored.

By separating out the issues, we can form consensus more easily (smaller fights ;-), and accept divergence more readily.

Rationale

There are standard inline documentation systems for some other languages. For example, Perl has POD [5] ("Plain Old Documentation") and Java has Javadoc [6], but neither of these mesh with the Pythonic way. POD syntax is very explicit, but takes after Perl in terms of readability. Javadoc is HTML-centric; except for "@field" tags, raw HTML is used for markup. There are also general tools such as Autoduck [7] and Web [8] (Tangle & Weave), useful for multiple languages.

There have been many attempts to write auto-documentation systems for Python (not an exhaustive list):

  • Marc-Andre Lemburg's doc.py [9]
  • Daniel Larsson's pythondoc [10] & gendoc [10]
  • Doug Hellmann's HappyDoc [11]
  • Laurence Tratt's Crystal (no longer available on the web)
  • Ka-Ping Yee's pydoc [12] (pydoc.py is now part of the Python standard library; see below)
  • Tony Ibbs' docutils [13] (Tony has donated this name to the Docutils project [14])
  • Edward Loper's STminus [15] formalization and related efforts

These systems, each with different goals, have had varying degrees of success. A problem with many of the above systems was over-ambition combined with inflexibility. They provided a self-contained set of components: a docstring extraction system, a markup parser, an internal processing system and one or more output format writers with a fixed style. Inevitably, one or more aspects of each system had serious shortcomings, and they were not easily extended or modified, preventing them from being adopted as standard tools.

It has become clear (to this author, at least) that the "all or nothing" approach cannot succeed, since no monolithic self-contained system could possibly be agreed upon by all interested parties. A modular component approach designed for extension, where components may be multiply implemented, may be the only chance for success. Standard inter-component APIs will make the DPS components comprehensible without requiring detailed knowledge of the whole, lowering the barrier for contributions, and ultimately resulting in a rich and varied system.

Each of the components of a docstring processing system should be developed independently. A "best of breed" system should be chosen, either merged from existing systems, and/or developed anew. This system should be included in Python's standard library.

PyDoc & Other Existing Systems

PyDoc became part of the Python standard library as of release 2.1. It extracts and displays docstrings from within the Python interactive interpreter, from the shell command line, and from a GUI window into a web browser (HTML). Although a very useful tool, PyDoc has several deficiencies, including:

  • In the case of the GUI/HTML, except for some heuristic hyperlinking of identifier names, no formatting of the docstrings is done. They are presented within <p><small><tt> tags to avoid unwanted line wrapping. Unfortunately, the result is not attractive.
  • PyDoc extracts docstrings and structural information (class identifiers, method signatures, etc.) from imported module objects. There are security issues involved with importing untrusted code. Also, information from the source is lost when importing, such as comments, "additional docstrings" (string literals in non-docstring contexts; see PEP 258 [3]), and the order of definitions.

The functionality proposed in this PEP could be added to or used by PyDoc when serving HTML pages. The proposed docstring processing system's functionality is much more than PyDoc needs in its current form. Either an independent tool will be developed (which PyDoc may or may not use), or PyDoc could be expanded to encompass this functionality and become the docstring processing system (or one such system). That decision is beyond the scope of this PEP.

Similarly for other existing docstring processing systems, their authors may or may not choose compatibility with this framework. However, if this framework is accepted and adopted as the Python standard, compatibility will become an important consideration in these systems' future.

Specification

The docstring processing system framework is broken up as follows:

  1. Docstring conventions. Documents issues such as:

    • What should be documented where.
    • First line is a one-line synopsis.

    PEP 257 [2] documents some of these issues.

  2. Docstring processing system design specification. Documents issues such as:

    • High-level spec: what a DPS does.
    • Command-line interface for executable script.
    • System Python API.
    • Docstring extraction rules.
    • Readers, which encapsulate the input context.
    • Parsers.
    • Document tree: the intermediate internal data structure. The output of the Parser and Reader, and the input to the Writer all share the same data structure.
    • Transforms, which modify the document tree.
    • Writers for output formats.
    • Distributors, which handle output management (one file, many files, or objects in memory).

    These issues are applicable to any docstring processing system implementation. PEP 258 [3] documents these issues.

  3. Docstring processing system implementation.

  4. Input markup specifications: docstring syntax. PEP 287 [1] proposes a standard syntax.

  5. Input parser implementations.

  6. Input context readers ("modes": Python source code, PEP, standalone text file, email, etc.) and implementations.

  7. Stylists: certain input context readers may have associated stylists which allow for a variety of output document styles.

  8. Output formats (HTML, XML, TeX, DocBook, info, etc.) and writer implementations.

Components 1, 2/3/5, and 4 are the subject of individual companion PEPs. If there is another implementation of the framework or syntax/parser, additional PEPs may be required. Multiple implementations of each of components 6 and 7 will be required; the PEP mechanism may be overkill for these components.

Project Web Site

A SourceForge project has been set up for this work at http://docutils.sourceforge.net/.

References and Footnotes

[1] (1, 2) PEP 287, reStructuredText Docstring Format, Goodger (http://www.python.org/peps/pep-0287.html)
[2] (1, 2) PEP 257, Docstring Conventions, Goodger, Van Rossum (http://www.python.org/peps/pep-0257.html)
[3] (1, 2, 3) PEP 258, Docutils Design Specification, Goodger (http://www.python.org/peps/pep-0258.html)
[4] http://www.literateprogramming.com/
[5] http://www.perldoc.com/perl5.6/pod/perlpod.html
[6] http://java.sun.com/j2se/javadoc/
[7] http://www.helpmaster.com/hlp-developmentaids-autoduck.htm
[8] http://www-cs-faculty.stanford.edu/~knuth/cweb.html
[9] http://www.egenix.com/files/python/SoftwareDescriptions.html#doc.py
[10] (1, 2) http://starship.python.net/crew/danilo/pythondoc/
[11] http://happydoc.sourceforge.net/
[12] http://www.python.org/doc/current/lib/module-pydoc.html
[13] http://www.tibsnjoan.co.uk/docutils.html
[14] http://docutils.sourceforge.net/
[15] http://www.cis.upenn.edu/~edloper/pydoc/
[16] http://www.python.org/sigs/doc-sig/

Copyright

This document has been placed in the public domain.

Acknowledgements

This document borrows ideas from the archives of the Python Doc-SIG [16]. Thanks to all members past & present.

View document source. Generated on: 2004-07-01 23:07 UTC. Generated by Docutils from reStructuredText source.


PEP 257 -- Docstring Conventions

PEP: | 257
Title: | Docstring Conventions
Version: | 1.7
Last-Modified: | 2002/11/30 01:59:35
Author: | David Goodger <goodger at users.sourceforge.net>, Guido van Rossum <guido at python.org>
Discussions-To: | doc-sig at python.org
Status: | Active
Type: | Informational
Content-Type: | text/x-rst
Created: | 29-May-2001
Post-History: | 13-Jun-2001

Contents

  • Abstract
  • Rationale
  • Specification
    • What is a Docstring?
    • One-line Docstrings
    • Multi-line Docstrings
    • Handling Docstring Indentation
  • References and Footnotes
  • Copyright
  • Acknowledgements

Abstract

This PEP documents the semantics and conventions associated with Python docstrings.

Rationale

The aim of this PEP is to standardize the high-level structure of docstrings: what they should contain, and how to say it (without touching on any markup syntax within docstrings). The PEP contains conventions, not laws or syntax.

"A universal convention supplies all of maintainability, clarity, consistency, and a foundation for good programming habits too. What it doesn't do is insist that you follow it against your will. That's Python!"

—Tim Peters on comp.lang.python, 2001-06-16

If you violate these conventions, the worst you'll get is some dirty looks. But some software (such as the Docutils [4] docstring processing system [1] [2]) will be aware of the conventions, so following them will get you the best results.

Specification

What is a Docstring?

A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition. Such a docstring becomes the __doc__ special attribute of that object.

All modules should normally have docstrings, and all functions and classes exported by a module should also have docstrings. Public methods (including the __init__ constructor) should also have docstrings. A package may be documented in the module docstring of the __init__.py file in the package directory.

String literals occurring elsewhere in Python code may also act as documentation. They are not recognized by the Python bytecode compiler and are not accessible as runtime object attributes (i.e. not assigned to __doc__), but two types of extra docstrings may be extracted by software tools:

  1. String literals occurring immediately after a simple assignment at the top level of a module, class, or __init__ method are called "attribute docstrings".
  2. String literals occurring immediately after another docstring are called "additional docstrings".

Please see PEP 258, "Docutils Design Specification" [2], for a detailed description of attribute and additional docstrings.

XXX Mention docstrings of 2.2 properties.

For consistency, always use """triple double quotes""" around docstrings. Use r"""raw triple double quotes""" if you use any backslashes in your docstrings. For Unicode docstrings, use u"""Unicode triple-quoted strings""".

There are two forms of docstrings: one-liners and multi-line docstrings.

One-line Docstrings

One-liners are for really obvious cases. They should really fit on one line. For example:

def kos_root():
    """Return the pathname of the KOS root directory."""
    global _kos_root
    if _kos_root: return _kos_root
    ...

Notes:

  • Triple quotes are used even though the string fits on one line. This makes it easy to later expand it.

  • The closing quotes are on the same line as the opening quotes. This looks better for one-liners.

  • There's no blank line either before or after the docstring.

  • The docstring is a phrase ending in a period. It prescribes the function or method's effect as a command ("Do this", "Return that"), not as a description; e.g. don't write "Returns the pathname ...".

  • The one-line docstring should NOT be a "signature" reiterating the function/method parameters (which can be obtained by introspection). Don't do:

    def function(a, b):
    """function(a, b) -> list"""

    This type of docstring is only appropriate for C functions (such as built-ins), where introspection is not possible. However, the nature of the return value cannot be determined by introspection, so it should be mentioned. The preferred form for such a docstring would be something like:

    def function(a, b):
    """Do X and return a list."""

    (Of course "Do X" should be replaced by a useful description!)

Multi-line Docstrings

Multi-line docstrings consist of a summary line just like a one-line docstring, followed by a blank line, followed by a more elaborate description. The summary line may be used by automatic indexing tools; it is important that it fits on one line and is separated from the rest of the docstring by a blank line. The summary line may be on the same line as the opening quotes or on the next line. The entire docstring is indented the same as the quotes at its first line (see example below).

Insert a blank line before and after all docstrings (one-line or multi-line) that document a class -- generally speaking, the class's methods are separated from each other by a single blank line, and the docstring needs to be offset from the first method by a blank line; for symmetry, put a blank line between the class header and the docstring. Docstrings documenting functions or methods generally don't have this requirement, unless the function or method's body is written as a number of blank-line separated sections -- in this case, treat the docstring as another section, and precede it with a blank line.

The docstring of a script (a stand-alone program) should be usable as its "usage" message, printed when the script is invoked with incorrect or missing arguments (or perhaps with a "-h" option, for "help"). Such a docstring should document the script's function and command line syntax, environment variables, and files. Usage messages can be fairly elaborate (several screens full) and should be sufficient for a new user to use the command properly, as well as a complete quick reference to all options and arguments for the sophisticated user.

The docstring for a module should generally list the classes, exceptions and functions (and any other objects) that are exported by the module, with a one-line summary of each. (These summaries generally give less detail than the summary line in the object's docstring.) The docstring for a package (i.e., the docstring of the package's __init__.py module) should also list the modules and subpackages exported by the package.

The docstring for a function or method should summarize its behavior and document its arguments, return value(s), side effects, exceptions raised, and restrictions on when it can be called (all if applicable). Optional arguments should be indicated. It should be documented whether keyword arguments are part of the interface.

The docstring for a class should summarize its behavior and list the public methods and instance variables. If the class is intended to be subclassed, and has an additional interface for subclasses, this interface should be listed separately (in the docstring). The class constructor should be documented in the docstring for its __init__ method. Individual methods should be documented by their own docstring.

If a class subclasses another class and its behavior is mostly inherited from that class, its docstring should mention this and summarize the differences. Use the verb "override" to indicate that a subclass method replaces a superclass method and does not call the superclass method; use the verb "extend" to indicate that a subclass method calls the superclass method (in addition to its own behavior).

Do not use the Emacs convention of mentioning the arguments of functions or methods in upper case in running text. Python is case sensitive and the argument names can be used for keyword arguments, so the docstring should document the correct argument names. It is best to list each argument on a separate line. For example:

def complex(real=0.0, imag=0.0):
    """Form a complex number.

    Keyword arguments:
    real -- the real part (default 0.0)
    imag -- the imaginary part (default 0.0)

    """
    if imag == 0.0 and real == 0.0: return complex_zero
    ...

The BDFL [3] recommends inserting a blank line between the last paragraph in a multi-line docstring and its closing quotes, placing the closing quotes on a line by themselves. This way, Emacs' fill-paragraph command can be used on it.

Handling Docstring Indentation

Docstring processing tools will strip a uniform amount of indentation from the second and further lines of the docstring, equal to the minimum indentation of all non-blank lines after the first line. Any indentation in the first line of the docstring (i.e., up to the first newline) is insignificant and removed. Relative indentation of later lines in the docstring is retained. Blank lines should be removed from the beginning and end of the docstring.

Since code is much more precise than words, here is an implementation of the algorithm:

def trim(docstring):
    if not docstring:
        return ''
    # Convert tabs to spaces (following the normal Python rules)
    # and split into a list of lines:
    lines = docstring.expandtabs().splitlines()
    # Determine minimum indentation (first line doesn't count):
    indent = sys.maxint
    for line in lines[1:]:
        stripped = line.lstrip()
        if stripped:
            indent = min(indent, len(line) - len(stripped))
    # Remove indentation (first line is special):
    trimmed = [lines[0].strip()]
    if indent < sys.maxint:
        for line in lines[1:]:
            trimmed.append(line[indent:].rstrip())
    # Strip off trailing and leading blank lines:
    while trimmed and not trimmed[-1]:
        trimmed.pop()
    while trimmed and not trimmed[0]:
        trimmed.pop(0)
    # Return a single string:
    return '\n'.join(trimmed)

The docstring in this example contains two newline characters and is therefore 3 lines long. The first and last lines are blank:

def foo():
    """
    This is the second line of the docstring.
    """

To illustrate:

>>> print repr(foo.__doc__)
'\n    This is the second line of the docstring.\n    '
>>> foo.__doc__.splitlines()
['', '    This is the second line of the docstring.', '    ']
>>> trim(foo.__doc__)
'This is the second line of the docstring.'

Once trimmed, these docstrings are equivalent:

def foo():
    """A multi-line
    docstring.
    """

def bar():
    """
    A multi-line
    docstring.
    """

References and Footnotes

[1] PEP 256, Docstring Processing System Framework, Goodger (http://www.python.org/peps/pep-0256.html)
[2] (1, 2) PEP 258, Docutils Design Specification, Goodger (http://www.python.org/peps/pep-0258.html)
[3] Guido van Rossum, Python's creator and Benevolent Dictator For Life.
[4] http://docutils.sourceforge.net/
[5] http://www.python.org/doc/essays/styleguide.html
[6] http://www.python.org/sigs/doc-sig/

Copyright

This document has been placed in the public domain.

Acknowledgements

The "Specification" text comes mostly verbatim from the Python Style Guide [5] essay by Guido van Rossum.

This document borrows ideas from the archives of the Python Doc-SIG [6]. Thanks to all members past and present.

View document source. Generated on: 2004-07-01 23:07 UTC. Generated by Docutils from reStructuredText source.


PEP 258 -- Docutils Design Specification

PEP: | 258
Title: | Docutils Design Specification
Version: | 1.31
Last-Modified: | 2004/06/04 21:31:18
Author: | David Goodger <goodger at users.sourceforge.net>
Discussions-To: | <doc-sig at python.org>
Status: | Draft
Type: | Standards Track
Content-Type: | text/x-rst
Requires: | 256 257
Created: | 31-May-2001
Post-History: | 13-Jun-2001

Contents

  • Abstract
  • Specification
    • Docutils Project Model
      • Publisher
      • Readers
      • Parsers
      • Transformer
      • Writers
      • Input/Output
    • Docutils Package Structure
    • Front-End Tools
    • Document Tree
    • Error Handling
    • Python Source Reader
      • Processing Model
      • AST Mining
      • Docstring Extraction Rules
        • Attribute Docstrings
        • Additional Docstrings
      • Choice of Docstring Format
      • Identifier Cross-References
      • Stylist Transforms
  • References and Footnotes
  • Project Web Site
  • Copyright
  • Acknowledgements

Abstract

This PEP documents design issues and implementation details for Docutils, a Python Docstring Processing System (DPS). The rationale and high-level concepts of a DPS are documented in PEP 256, "Docstring Processing System Framework" [1]. Also see PEP 256 for a "Road Map to the Docstring PEPs".

Docutils is being designed modularly so that any of its components can be replaced easily. In addition, Docutils is not limited to the processing of Python docstrings; it processes standalone documents as well, in several contexts.

No changes to the core Python language are required by this PEP. Its deliverables consist of a package for the standard library and its documentation.

Specification

Docutils Project Model

Project components and data flow:

                 +---------------------------+
                 |        Docutils:          |
                 | docutils.core.Publisher,  |
                 | docutils.core.publish_*() |
                 +---------------------------+
                  /            |            \
                 /             |             \
        1,3,5   /        6     |              \ 7
       +--------+       +-------------+       +--------+
       | READER | ----> | TRANSFORMER | ====> | WRITER |
       +--------+       +-------------+       +--------+
        /     \\                                  |
       /       \\                                 |
 2    /      4  \\                             8  |
+-------+   +--------+                        +--------+
| INPUT |   | PARSER |                        | OUTPUT |
+-------+   +--------+                        +--------+

The numbers above each component indicate the path a document's data takes. Double-width lines between Reader & Parser and between Transformer & Writer indicate that data sent along these paths should be standard (pure & unextended) Docutils doc trees. Single-width lines signify that internal tree extensions or completely unrelated representations are possible, but they must be supported at both ends.

Publisher

The docutils.core module contains a "Publisher" facade class and several convenience functions: "publish_cmdline()" (for command-line front ends), "publish_file()" (for programmatic use with file-like I/O), and "publish_string()" (for programmatic use with string I/O). The Publisher class encapsulates the high-level logic of a Docutils system. The Publisher class has overall responsibility for processing, controlled by the Publisher.publish() method:

  1. Set up internal settings (may include config files & command-line options) and I/O objects.
  2. Call the Reader object to read data from the source Input object and parse the data with the Parser object. A document object is returned.
  3. Set up and apply transforms via the Transformer object attached to the document.
  4. Call the Writer object which translates the document to the final output format and writes the formatted data to the destination Output object. Depending on the Output object, the output may be returned from the Writer, and then from the publish() method.

Calling the "publish" function (or instantiating a "Publisher" object) with component names will result in default behavior. For custom behavior (customizing component settings), create custom component objects first, and pass them to the Publisher or publish_* convenience functions.

Readers

Readers understand the input context (where the data is coming from), send the whole input or discrete "chunks" to the parser, and provide the context to bind the chunks together back into a cohesive whole.

Each reader is a module or package exporting a "Reader" class with a "read" method. The base "Reader" class can be found in the docutils/readers/__init__.py module.

Most Readers will have to be told what parser to use. So far (see the list of examples below), only the Python Source Reader ("PySource"; still incomplete) will be able to determine the parser on its own.

Responsibilities:

  • Get input text from the source I/O.
  • Pass the input text to the parser, along with a fresh document tree root.

Examples:

  • Standalone (Raw/Plain): Just read a text file and process it. The reader needs to be told which parser to use.

    The "Standalone Reader" has been implemented in module docutils.readers.standalone.

  • Python Source: See Python Source Reader below. This Reader is currently in development in the Docutils sandbox.

  • Email: RFC-822 headers, quoted excerpts, signatures, MIME parts.

  • PEP: RFC-822 headers, "PEP xxxx" and "RFC xxxx" conversion to URIs. The "PEP Reader" has been implemented in module docutils.readers.pep; see PEP 287 and PEP 12.

  • Wiki: Global reference lookups of "wiki links" incorporated into transforms. (CamelCase only or unrestricted?) Lazy indentation?

  • Web Page: As standalone, but recognize meta fields as meta tags. Support for templates of some sort? (After <body>, before </body>?)

  • FAQ: Structured "question & answer(s)" constructs.

  • Compound document: Merge chapters into a book. Master manifest file?

Parsers

Parsers analyze their input and produce a Docutils document tree. They don't know or care anything about the source or destination of the data.

Each input parser is a module or package exporting a "Parser" class with a "parse" method. The base "Parser" class can be found in the docutils/parsers/__init__.py module.

Responsibilities: Given raw input text and a doctree root node, populate the doctree by parsing the input text.

Example: The only parser implemented so far is for the reStructuredText markup. It is implemented in the docutils/parsers/rst/ package.

The development and integration of other parsers is possible and encouraged.

Transformer

The Transformer class, in docutils/transforms/__init__.py, stores transforms and applies them to documents. A transformer object is attached to every new document tree. The Publisher calls Transformer.apply_transforms() to apply all stored transforms to the document tree. Transforms change the document tree from one form to another, add to the tree, or prune it. Transforms resolve references and footnote numbers, process interpreted text, and do other context-sensitive processing.

Some transforms are specific to components (Readers, Parser, Writers, Input, Output). Standard component-specific transforms are specified in the default_transforms attribute of component classes. After the Reader has finished processing, the Publisher calls Transformer.populate_from_components() with a list of components and all default transforms are stored.

Each transform is a class in a module in the docutils/transforms/ package, a subclass of docutils.tranforms.Transform. Transform classes each have a default_priority attribute which is used by the Transformer to apply transforms in order (low to high). The default priority can be overridden when adding transforms to the Transformer object.

Transformer responsibilities:

  • Apply transforms to the document tree, in priority order.
  • Store a mapping of component type name ('reader', 'writer', etc.) to component objects. These are used by certain transforms (such as "components.Filter") to determine suitability.

Transform responsibilities:

  • Modify a doctree in-place, either purely transforming one structure into another, or adding new structures based on the doctree and/or external data.

Examples of transforms (in the docutils/transforms/ package):

  • frontmatter.DocInfo: Conversion of document metadata (bibliographic information).
  • references.AnonymousHyperlinks: Resolution of anonymous references to corresponding targets.
  • parts.Contents: Generates a table of contents for a document.
  • document.Merger: Combining multiple populated doctrees into one. (Not yet implemented or fully understood.)
  • document.Splitter: Splits a document into a tree-structure of subdocuments, perhaps by section. It will have to transform references appropriately. (Neither implemented not remotely understood.)
  • components.Filter: Includes or excludes elements which depend on a specific Docutils component.

Writers

Writers produce the final output (HTML, XML, TeX, etc.). Writers translate the internal document tree structure into the final data format, possibly running Writer-specific transforms first.

By the time the document gets to the Writer, it should be in final form. The Writer's job is simply (and only) to translate from the Docutils doctree structure to the target format. Some small transforms may be required, but they should be local and format-specific.

Each writer is a module or package exporting a "Writer" class with a "write" method. The base "Writer" class can be found in the docutils/writers/__init__.py module.

Responsibilities:

  • Translate doctree(s) into specific output formats.
    • Transform references into format-native forms.
  • Write the translated output to the destination I/O.

Examples:

  • XML: Various forms, such as:
    • Docutils XML (an expression of the internal document tree, implemented as docutils.writers.docutils_xml).
    • DocBook (being implemented in the Docutils sandbox).
  • HTML (XHTML implemented as docutils.writers.html4css1).
  • PDF (a ReportLabs interface is being developed in the Docutils sandbox).
  • TeX (a LaTeX Writer is being implemented in the sandbox).
  • Docutils-native pseudo-XML (implemented as docutils.writers.pseudoxml, used for testing).
  • Plain text
  • reStructuredText?

Input/Output

I/O classes provide a uniform API for low-level input and output. Subclasses will exist for a variety of input/output mechanisms. However, they can be considered an implementation detail. Most applications should be satisfied using one of the convenience functions associated with the Publisher.

I/O classes are currently in the preliminary stages; there's a lot of work yet to be done. Issues:

  • How to represent multi-file input (files & directories) in the API?
  • How to represent multi-file output? Perhaps "Writer" variants, one for each output distribution type? Or Output objects with associated transforms?

Responsibilities:

  • Read data from the input source (Input objects) or write data to the output destination (Output objects).

Examples of input sources:

  • A single file on disk or a stream (implemented as docutils.io.FileInput).
  • Multiple files on disk (MultiFileInput?).
  • Python source files: modules and packages.
  • Python strings, as received from a client application (implemented as docutils.io.StringInput).

Examples of output destinations:

  • A single file on disk or a stream (implemented as docutils.io.FileOutput).
  • A tree of directories and files on disk.
  • A Python string, returned to a client application (implemented as docutils.io.StringOutput).
  • No output; useful for programmatic applications where only a portion of the normal output is to be used (implemented as docutils.io.NullOutput).
  • A single tree-shaped data structure in memory.
  • Some other set of data structures in memory.

Docutils Package Structure

  • Package "docutils".

    • Module "__init__.py" contains: class "Component", a base class for Docutils components; class "SettingsSpec", a base class for specifying runtime settings (used by docutils.frontend); and class "TransformSpec", a base class for specifying transforms.

    • Module "docutils.core" contains facade class "Publisher" and convenience functions. See Publisher above.

    • Module "docutils.frontend" provides runtime settings support, for programmatic use and front-end tools (including configuration file support, and command-line argument and option processing).

    • Module "docutils.io" provides a uniform API for low-level input and output. See Input/Output above.

    • Module "docutils.nodes" contains the Docutils document tree element class library plus tree-traversal Visitor pattern base classes. See Document Tree below.

    • Module "docutils.statemachine" contains a finite state machine specialized for regular-expression-based text filters and parsers. The reStructuredText parser implementation is based on this module.

    • Module "docutils.urischemes" contains a mapping of known URI schemes ("http", "ftp", "mail", etc.).

    • Module "docutils.utils" contains utility functions and classes, including a logger class ("Reporter"; see Error Handling below).

    • Package "docutils.parsers": markup parsers.

      • Function "get_parser_class(parser_name)" returns a parser module by name. Class "Parser" is the base class of specific parsers. (docutils/parsers/__init__.py)
      • Package "docutils.parsers.rst": the reStructuredText parser.
      • Alternate markup parsers may be added.

      See Parsers above.

    • Package "docutils.readers": context-aware input readers.

      • Function "get_reader_class(reader_name)" returns a reader module by name or alias. Class "Reader" is the base class of specific readers. (docutils/readers/__init__.py)
      • Module "docutils.readers.standalone" reads independent document files.
      • Module "docutils.readers.pep" reads PEPs (Python Enhancement Proposals).
      • Readers to be added for: Python source code (structure & docstrings), email, FAQ, and perhaps Wiki and others.

      See Readers above.

    • Package "docutils.writers": output format writers.

      • Function "get_writer_class(writer_name)" returns a writer module by name. Class "Writer" is the base class of specific writers. (docutils/writers/__init__.py)
      • Module "docutils.writers.html4css1" is a simple HyperText Markup Language document tree writer for HTML 4.01 and CSS1.
      • Module "docutils.writers.docutils_xml" writes the internal document tree in XML form.
      • Module "docutils.writers.pseudoxml" is a simple internal document tree writer; it writes indented pseudo-XML.
      • Writers to be added: HTML 3.2 or 4.01-loose, XML (various forms, such as DocBook), PDF, TeX, plaintext, reStructuredText, and perhaps others.

      See Writers above.

    • Package "docutils.transforms": tree transform classes.

      • Class "Transformer" stores transforms and applies them to document trees. (docutils/transforms/__init__.py)
      • Class "Transform" is the base class of specific transforms. (docutils/transforms/__init__.py)
      • Each module contains related transform classes.

      See Transforms above.

    • Package "docutils.languages": Language modules contain language-dependent strings and mappings. They are named for their language identifier (as defined in Choice of Docstring Format below), converting dashes to underscores.

      • Function "get_language(language_code)", returns matching language module. (docutils/languages/__init__.py)
      • Modules: en.py (English), de.py (German), fr.py (French), it.py (Italian), sk.py (Slovak), sv.py (Swedish).
      • Other languages to be added.

  • Third-party modules: "extras" directory. These modules are installed only if they're not already present in the Python installation.

    • extras/optparse.py and extras/textwrap.py provide option parsing and command-line help; from Greg Ward's http://optik.sf.net/ project, included for convenience.
    • extras/roman.py contains Roman numeral conversion routines.

Front-End Tools

The tools/ directory contains several front ends for common Docutils processing. See Docutils Front-End Tools [4] for details.

Document Tree

A single intermediate data structure is used internally by Docutils, in the interfaces between components; it is defined in the docutils.nodes module. It is not required that this data structure be used internally by any of the components, just between components as outlined in the diagram in the Docutils Project Model above.

Custom node types are allowed, provided that either (a) a transform converts them to standard Docutils nodes before they reach the Writer proper, or (b) the custom node is explicitly supported by certain Writers, and is wrapped in a filtered "pending" node. An example of condition (a) is the Python Source Reader (see below), where a "stylist" transform converts custom nodes. The HTML <meta> tag is an example of condition (b); it is supported by the HTML Writer but not by others. The reStructuredText "meta" directive creates a "pending" node, which contains knowledge that the embedded "meta" node can only be handled by HTML-compatible writers. The "pending" node is resolved by the docutils.transforms.components.Filter transform, which checks that the calling writer supports HTML; if it doesn't, the "pending" node (and enclosed "meta" node) is removed from the document.

The document tree data structure is similar to a DOM tree, but with specific node names (classes) instead of DOM's generic nodes. The schema is documented in an XML DTD (eXtensible Markup Language Document Type Definition), which comes in two parts:

  • the Docutils Generic DTD, docutils.dtd [5], and
  • the OASIS Exchange Table Model, soextbl.dtd [6].

The DTD defines a rich set of elements, suitable for many input and output formats. The DTD retains all information necessary to reconstruct the original input text, or a reasonable facsimile thereof.

See The Docutils Document Tree [7] for details (incomplete).

Error Handling

When the parser encounters an error in markup, it inserts a system message (DTD element "system_message"). There are five levels of system messages:

  • Level-0, "DEBUG": an internal reporting issue. There is no effect on the processing. Level-0 system messages are handled separately from the others.
  • Level-1, "INFO": a minor issue that can be ignored. There is little or no effect on the processing. Typically level-1 system messages are not reported.
  • Level-2, "WARNING": an issue that should be addressed. If ignored, there may be minor problems with the output. Typically level-2 system messages are reported but do not halt processing
  • Level-3, "ERROR": a major issue that should be addressed. If ignored, the output will contain unpredictable errors. Typically level-3 system messages are reported but do not halt processing
  • Level-4, "SEVERE": a critical error that must be addressed. Typically level-4 system messages are turned into exceptions which halt processing. If ignored, the output will contain severe errors.

Although the initial message levels were devised independently, they have a strong correspondence to VMS error condition severity levels [8]; the names in quotes for levels 1 through 4 were borrowed from VMS. Error handling has since been influenced by the log4j project [9].

Python Source Reader

The Python Source Reader ("PySource") is the Docutils component that reads Python source files, extracts docstrings in context, then parses, links, and assembles the docstrings into a cohesive whole. It is a major and non-trivial component, currently under experimental development in the Docutils sandbox. High-level design issues are presented here.

Processing Model

This model will evolve over time, incorporating experience and discoveries.

  1. The PySource Reader uses an Input class to read in Python packages and modules, into a tree of strings.
  2. The Python modules are parsed, converting the tree of strings into a tree of abstract syntax trees with docstring nodes.
  3. The abstract syntax trees are converted into an internal representation of the packages/modules. Docstrings are extracted, as well as code structure details. See AST Mining below. Namespaces are constructed for lookup in step 6.
  4. One at a time, the docstrings are parsed, producing standard Docutils doctrees.
  5. PySource assembles all the individual docstrings' doctrees into a Python-specific custom Docutils tree paralleling the package/module/class structure; this is a custom Reader-specific internal representation (see the Docutils Python Source DTD [10]). Namespaces must be merged: Python identifiers, hyperlink targets.
  6. Cross-references from docstrings (interpreted text) to Python identifiers are resolved according to the Python namespace lookup rules. See Identifier Cross-References below.
  7. A "Stylist" transform is applied to the custom doctree (by the Transformer), custom nodes are rendered using standard nodes as primitives, and a standard document tree is emitted. See Stylist Transforms below.
  8. Other transforms are applied to the standard doctree by the Transformer.
  9. The standard doctree is sent to a Writer, which translates the document into a concrete format (HTML, PDF, etc.).
  10. The Writer uses an Output class to write the resulting data to its destination (disk file, directories and files, etc.).

AST Mining

Abstract Syntax Tree mining code will be written (or adapted) that scans a parsed Python module, and returns an ordered tree containing the names, docstrings (including attribute and additional docstrings; see below), and additional info (in parentheses below) of all of the following objects:

  • packages
  • modules
  • module attributes (+ initial values)
  • classes (+ inheritance)
  • class attributes (+ initial values)
  • instance attributes (+ initial values)
  • methods (+ parameters & defaults)
  • functions (+ parameters & defaults)

(Extract comments too? For example, comments at the start of a module would be a good place for bibliographic field lists.)

In order to evaluate interpreted text cross-references, namespaces for each of the above will also be required.

See the python-dev/docstring-develop thread "AST mining", started on 2001-08-14.

Docstring Extraction Rules

  1. What to examine:

    1. If the "__all__" variable is present in the module being documented, only identifiers listed in "__all__" are examined for docstrings.
    2. In the absence of "__all__", all identifiers are examined, except those whose names are private (names begin with "_" but don't begin and end with "__").
    3. 1a and 1b can be overridden by runtime settings.

  2. Where:

    Docstrings are string literal expressions, and are recognized in the following places within Python modules:

    1. At the beginning of a module, function definition, class definition, or method definition, after any comments. This is the standard for Python __doc__ attributes.
    2. Immediately following a simple assignment at the top level of a module, class definition, or __init__ method definition, after any comments. See Attribute Docstrings below.
    3. Additional string literals found immediately after the docstrings in (a) and (b) will be recognized, extracted, and concatenated. See Additional Docstrings below.
    4. @@@ 2.2-style "properties" with attribute docstrings? Wait for syntax?

  3. How:

    Whenever possible, Python modules should be parsed by Docutils, not imported. There are several reasons:

    • Importing untrusted code is inherently insecure.
    • Information from the source is lost when using introspection to examine an imported module, such as comments and the order of definitions.
    • Docstrings are to be recognized in places where the byte-code compiler ignores string literal expressions (2b and 2c above), meaning importing the module will lose these docstrings.

    Of course, standard Python parsing tools such as the "parser" library module should be used.

    When the Python source code for a module is not available (i.e. only the .pyc file exists) or for C extension modules, to access docstrings the module can only be imported, and any limitations must be lived with.

Since attribute docstrings and additional docstrings are ignored by the Python byte-code compiler, no namespace pollution or runtime bloat will result from their use. They are not assigned to __doc__ or to any other attribute. The initial parsing of a module may take a slight performance hit.

Attribute Docstrings

(This is a simplified version of PEP 224 [2].)

A string literal immediately following an assignment statement is interpreted by the docstring extraction machinery as the docstring of the target of the assignment statement, under the following conditions:

  1. The assignment must be in one of the following contexts:

    1. At the top level of a module (i.e., not nested inside a compound statement such as a loop or conditional): a module attribute.
    2. At the top level of a class definition: a class attribute.
    3. At the top level of the "__init__" method definition of a class: an instance attribute. Instance attributes assigned in other methods are assumed to be implementation details. (@@@ __new__ methods?)
    4. A function attribute assignment at the top level of a module or class definition.

    Since each of the above contexts are at the top level (i.e., in the outermost suite of a definition), it may be necessary to place dummy assignments for attributes assigned conditionally or in a loop.

  2. The assignment must be to a single target, not to a list or a tuple of targets.

  3. The form of the target:

    1. For contexts 1a and 1b above, the target must be a simple identifier (not a dotted identifier, a subscripted expression, or a sliced expression).
    2. For context 1c above, the target must be of the form "self.attrib", where "self" matches the "__init__" method's first parameter (the instance parameter) and "attrib" is a simple identifier as in 3a.
    3. For context 1d above, the target must be of the form "name.attrib", where "name" matches an already-defined function or method name and "attrib" is a simple identifier as in 3a.

Blank lines may be used after attribute docstrings to emphasize the connection between the assignment and the docstring.

Examples:

g = 'module attribute (module-global variable)'
"""This is g's docstring."""

class AClass:

    c = 'class attribute'
    """This is AClass.c's docstring."""

    def __init__(self):
        """Method __init__'s docstring."""

        self.i = 'instance attribute'
        """This is self.i's docstring."""

def f(x):
    """Function f's docstring."""
    return x**2

f.a = 1
"""Function attribute f.a's docstring."""

Additional Docstrings

(This idea was adapted from PEP 216 [3].)

Many programmers would like to make extensive use of docstrings for API documentation. However, docstrings do take up space in the running program, so some programmers are reluctant to "bloat up" their code. Also, not all API documentation is applicable to interactive environments, where __doc__ would be displayed.

Docutils' docstring extraction tools will concatenate all string literal expressions which appear at the beginning of a definition or after a simple assignment. Only the first strings in definitions will be available as __doc__, and can be used for brief usage text suitable for interactive sessions; subsequent string literals and all attribute docstrings are ignored by the Python byte-code compiler and may contain more extensive API information.

Example:

def function(arg):
    """This is __doc__, function's docstring."""
    """
    This is an additional docstring, ignored by the byte-code
    compiler, but extracted by Docutils.
    """
    pass

Issue: from __future__ import

This would break "from __future__ import" statements introduced in Python 2.1 for multiple module docstrings (main docstring plus additional docstring(s)). The Python Reference Manual specifies:

A future statement must appear near the top of the module. The only lines that can appear before a future statement are:

  • the module docstring (if any),
  • comments,
  • blank lines, and
  • other future statements.

Resolution?

  1. Should we search for docstrings after a __future__ statement? Very ugly.
  2. Redefine __future__ statements to allow multiple preceding string literals?
  3. Or should we not even worry about this? There probably shouldn't be __future__ statements in production code, after all. Perhaps modules with __future__ statements will simply have to put up with the single-docstring limitation.

Choice of Docstring Format

Rather than force everyone to use a single docstring format, multiple input formats are allowed by the processing system. A special variable, __docformat__, may appear at the top level of a module before any function or class definitions. Over time or through decree, a standard format or set of formats should emerge.

A module's __docformat__ variable only applies to the objects defined in the module's file. In particular, the __docformat__ variable in a package's __init__.py file does not apply to objects defined in subpackages and submodules.

The __docformat__ variable is a string containing the name of the format being used, a case-insensitive string matching the input parser's module or package name (i.e., the same name as required to "import" the module or package), or a registered alias. If no __docformat__ is specified, the default format is "plaintext" for now; this may be changed to the standard format if one is ever established.

The __docformat__ string may contain an optional second field, separated from the format name (first field) by a single space: a case-insensitive language identifier as defined in RFC 1766. A typical language identifier consists of a 2-letter language code from ISO 639 [11] (3-letter codes used only if no 2-letter code exists; RFC 1766 is currently being revised to allow 3-letter codes). If no language identifier is specified, the default is "en" for English. The language identifier is passed to the parser and can be used for language-dependent markup features.

Identifier Cross-References

In Python docstrings, interpreted text is used to classify and mark up program identifiers, such as the names of variables, functions, classes, and modules. If the identifier alone is given, its role is inferred implicitly according to the Python namespace lookup rules. For functions and methods (even when dynamically assigned), parentheses ('()') may be included:

This function uses `another()` to do its work.

For class, instance and module attributes, dotted identifiers are used when necessary. For example (using reStructuredText markup):

class Keeper(Storer):

    """
    Extend `Storer`.  Class attribute `instances` keeps track
    of the number of `Keeper` objects instantiated.
    """

    instances = 0
    """How many `Keeper` objects are there?"""

    def __init__(self):
        """
        Extend `Storer.__init__()` to keep track of instances.

        Keep count in `Keeper.instances`, data in `self.data`.
        """
        Storer.__init__(self)
        Keeper.instances += 1

        self.data = []
        """Store data in a list, most recent last."""

    def store_data(self, data):
        """
        Extend `Storer.store_data()`; append new `data` to a
        list (in `self.data`).
        """
        self.data = data

Each of the identifiers quoted with backquotes ("`") will become references to the definitions of the identifiers themselves.

Stylist Transforms

Stylist transforms are specialized transforms specific to the PySource Reader. The PySource Reader doesn't have to make any decisions as to style; it just produces a logically constructed document tree, parsed and linked, including custom node types. Stylist transforms understand the custom nodes created by the Reader and convert them into standard Docutils nodes.

Multiple Stylist transforms may be implemented and one can be chosen at runtime (through a "--style" or "--stylist" command-line option). Each Stylist transform implements a different layout or style; thus the name. They decouple the context-understanding part of the Reader from the layout-generating part of processing, resulting in a more flexible and robust system. This also serves to "separate style from content", the SGML/XML ideal.

By keeping the piece of code that does the styling small and modular, it becomes much easier for people to roll their own styles. The "barrier to entry" is too high with existing tools; extracting the stylist code will lower the barrier considerably.

References and Footnotes

[1] PEP 256, Docstring Processing System Framework, Goodger (http://www.python.org/peps/pep-0256.html)
[2] PEP 224, Attribute Docstrings, Lemburg (http://www.python.org/peps/pep-0224.html)
[3] PEP 216, Docstring Format, Zadka (http://www.python.org/peps/pep-0216.html)
[4] http://docutils.sourceforge.net/docs/user/tools.html
[5] http://docutils.sourceforge.net/docs/ref/docutils.dtd
[6] http://docutils.sourceforge.net/docs/ref/soextblx.dtd
[7] http://docutils.sourceforge.net/docs/ref/doctree.html
[8] http://www.openvms.compaq.com:8000/73final/5841/841pro_027.html#error_cond_severity
[9] http://logging.apache.org/log4j/docs/index.html
[10] http://docutils.sourceforge.net/docs/dev/pysource.dtd
[11] http://lcweb.loc.gov/standards/iso639-2/englangn.html
[12] http://www.python.org/sigs/doc-sig/

Project Web Site

A SourceForge project has been set up for this work at http://docutils.sourceforge.net/.

Copyright

This document has been placed in the public domain.

Acknowledgements

This document borrows ideas from the archives of the Python Doc-SIG [12]. Thanks to all members past & present.

View document source. Generated on: 2004-07-01 23:07 UTC. Generated by Docutils from reStructuredText source.


PEP 287 -- reStructuredText Docstring Format

PEP: | 287
Title: | reStructuredText Docstring Format
Version: | 1.22
Last-Modified: | 2004/06/03 18:27:51
Author: | David Goodger <goodger at users.sourceforge.net>
Discussions-To: | <doc-sig at python.org>
Status: | Draft
Type: | Informational
Content-Type: | text/x-rst
Created: | 25-Mar-2002
Post-History: | 02-Apr-2002
Replaces: | 216

Contents

  • Abstract
  • Benefits
  • Goals
  • Rationale
  • Specification
  • Docstring-Significant Features
  • Questions & Answers
  • References & Footnotes
  • Copyright
  • Acknowledgements

Abstract

When plaintext hasn't been expressive enough for inline documentation, Python programmers have sought out a format for docstrings. This PEP proposes that the reStructuredText markup [5] be adopted as a standard markup format for structured plaintext documentation in Python docstrings, and for PEPs and ancillary documents as well. reStructuredText is a rich and extensible yet easy-to-read, what-you-see-is-what-you-get plaintext markup syntax.

Only the low-level syntax of docstrings is addressed here. This PEP is not concerned with docstring semantics or processing at all (see PEP 256 for a "Road Map to the Docstring PEPs"). Nor is it an attempt to deprecate pure plaintext docstrings, which are always going to be legitimate. The reStructuredText markup is an alternative for those who want more expressive docstrings.

Benefits

Programmers are by nature a lazy breed. We reuse code with functions, classes, modules, and subsystems. Through its docstring syntax, Python allows us to document our code from within. The "holy grail" of the Python Documentation Special Interest Group (Doc-SIG [6]) has been a markup syntax and toolset to allow auto-documentation, where the docstrings of Python systems can be extracted in context and processed into useful, high-quality documentation for multiple purposes.

Document markup languages have three groups of customers: the authors who write the documents, the software systems that process the data, and the readers, who are the final consumers and the most important group. Most markups are designed for the authors and software systems; readers are only meant to see the processed form, either on paper or via browser software. ReStructuredText is different: it is intended to be easily readable in source form, without prior knowledge of the markup. ReStructuredText is entirely readable in plaintext format, and many of the markup forms match common usage (e.g., *emphasis*), so it reads quite naturally. Yet it is rich enough to produce complex documents, and extensible so that there are few limits. Of course, to write reStructuredText documents some prior knowledge is required.

The markup offers functionality and expressivity, while maintaining easy readability in the source text. The processed form (HTML etc.) makes it all accessible to readers: inline live hyperlinks; live links to and from footnotes; automatic tables of contents (with live links!); tables; images for diagrams etc.; pleasant, readable styled text.

The reStructuredText parser is available now, part of the Docutils [24] project. Standalone reStructuredText documents and PEPs can be converted to HTML; other output format writers are being worked on and will become available over time. Work is progressing on a Python source "Reader" which will implement auto-documentation from docstrings. Authors of existing auto-documentation tools are encouraged to integrate the reStructuredText parser into their projects, or better yet, to join forces to produce a world-class toolset for the Python standard library.

Tools will become available in the near future, which will allow programmers to generate HTML for online help, XML for multiple purposes, and eventually PDF, DocBook, and LaTeX for printed documentation, essentially "for free" from the existing docstrings. The adoption of a standard will, at the very least, benefit docstring processing tools by preventing further "reinventing the wheel".

Eventually PyDoc, the one existing standard auto-documentation tool, could have reStructuredText support added. In the interim it will have no problem with reStructuredText markup, since it treats all docstrings as preformatted plaintext.

Goals

These are the generally accepted goals for a docstring format, as discussed in the Doc-SIG:

  1. It must be readable in source form by the casual observer.
  2. It must be easy to type with any standard text editor.
  3. It must not need to contain information which can be deduced from parsing the module.
  4. It must contain sufficient information (structure) so it can be converted to any reasonable markup format.
  5. It must be possible to write a module's entire documentation in docstrings, without feeling hampered by the markup language.

reStructuredText meets and exceeds all of these goals, and sets its own goals as well, even more stringent. See Docstring-Significant Features below.

The goals of this PEP are as follows:

  1. To establish reStructuredText as a standard structured plaintext format for docstrings (inline documentation of Python modules and packages), PEPs, README-type files and other standalone documents. "Accepted" status will be sought through Python community consensus and eventual BDFL pronouncement.

    Please note that reStructuredText is being proposed as a standard, not the only standard. Its use will be entirely optional. Those who don't want to use it need not.

  2. To solicit and address any related concerns raised by the Python community.

  3. To encourage community support. As long as multiple competing markups are out there, the development community remains fractured. Once a standard exists, people will start to use it, and momentum will inevitably gather.

  4. To consolidate efforts from related auto-documentation projects. It is hoped that interested developers will join forces and work on a joint/merged/common implementation.

Once reStructuredText is a Python standard, effort can be focused on tools instead of arguing for a standard. Python needs a standard set of documentation tools.

With regard to PEPs, one or both of the following strategies may be applied:

  1. Keep the existing PEP section structure constructs (one-line section headers, indented body text). Subsections can either be forbidden, or supported with reStructuredText-style underlined headers in the indented body text.
  2. Replace the PEP section structure constructs with the reStructuredText syntax. Section headers will require underlines, subsections will be supported out of the box, and body text need not be indented (except for block quotes).

Strategy (b) is recommended, and its implementation is complete.

Support for RFC 2822 headers has been added to the reStructuredText parser for PEPs (unambiguous given a specific context: the first contiguous block of the document). It may be desired to concretely specify what over/underline styles are allowed for PEP section headers, for uniformity.

Rationale

The lack of a standard syntax for docstrings has hampered the development of standard tools for extracting and converting docstrings into documentation in standard formats (e.g., HTML, DocBook, TeX). There have been a number of proposed markup formats and variations, and many tools tied to these proposals, but without a standard docstring format they have failed to gain a strong following and/or floundered half-finished.

Throughout the existence of the Doc-SIG, consensus on a single standard docstring format has never been reached. A lightweight, implicit markup has been sought, for the following reasons (among others):

  1. Docstrings written within Python code are available from within the interactive interpreter, and can be "print"ed. Thus the use of plaintext for easy readability.
  2. Programmers want to add structure to their docstrings, without sacrificing raw docstring readability. Unadorned plaintext cannot be transformed ("up-translated") into useful structured formats.
  3. Explicit markup (like XML or TeX) is widely considered unreadable by the uninitiated.
  4. Implicit markup is aesthetically compatible with the clean and minimalist Python syntax.

Many alternative markups for docstrings have been proposed on the Doc-SIG over the years; a representative sample is listed below. Each is briefly analyzed in terms of the goals stated above. Please note that this is not intended to be an exclusive list of all existing markup systems; there are many other markups (Texinfo, Doxygen, TIM, YODL, AFT, ...) which are not mentioned.

  • XML [7], SGML [8], DocBook [9], HTML [10], XHTML [11]

    XML and SGML are explicit, well-formed meta-languages suitable for all kinds of documentation. XML is a variant of SGML. They are best used behind the scenes, because to untrained eyes they are verbose, difficult to type, and too cluttered to read comfortably as source. DocBook, HTML, and XHTML are all applications of SGML and/or XML, and all share the same basic syntax and the same shortcomings.

  • TeX [12]

    TeX is similar to XML/SGML in that it's explicit, but not very easy to write, and not easy for the uninitiated to read.

  • Perl POD [13]

    Most Perl modules are documented in a format called POD (Plain Old Documentation). This is an easy-to-type, very low level format with strong integration with the Perl parser. Many tools exist to turn POD documentation into other formats: info, HTML and man pages, among others. However, the POD syntax takes after Perl itself in terms of readability.

  • JavaDoc [14]

    Special comments before Java classes and functions serve to document the code. A program to extract these, and turn them into HTML documentation is called javadoc, and is part of the standard Java distribution. However, JavaDoc has a very intimate relationship with HTML, using HTML tags for most markup. Thus it shares the readability problems of HTML.

  • Setext [15], StructuredText [16]

    Early on, variants of Setext (Structure Enhanced Text), including Zope Corp's StructuredText, were proposed for Python docstring formatting. Hereafter these variants will collectively be called "STexts". STexts have the advantage of being easy to read without special knowledge, and relatively easy to write.

    Although used by some (including in most existing Python auto-documentation tools), until now STexts have failed to become standard because:

    • STexts have been incomplete. Lacking "essential" constructs that people want to use in their docstrings, STexts are rendered less than ideal. Note that these "essential" constructs are not universal; everyone has their own requirements.
    • STexts have been sometimes surprising. Bits of text are unexpectedly interpreted as being marked up, leading to user frustration.
    • SText implementations have been buggy.
    • Most STexts have have had no formal specification except for the implementation itself. A buggy implementation meant a buggy spec, and vice-versa.
    • There has been no mechanism to get around the SText markup rules when a markup character is used in a non-markup context. In other words, no way to escape markup.

Proponents of implicit STexts have vigorously opposed proposals for explicit markup (XML, HTML, TeX, POD, etc.), and the debates have continued off and on since 1996 or earlier.

reStructuredText is a complete revision and reinterpretation of the SText idea, addressing all of the problems listed above.

Specification

The specification and user documentaton for reStructuredText is quite extensive. Rather than repeating or summarizing it all here, links to the originals are provided.

Please first take a look at A ReStructuredText Primer [17], a short and gentle introduction. The Quick reStructuredText [18] user reference quickly summarizes all of the markup constructs. For complete and extensive details, please refer to the following documents:

  • An Introduction to reStructuredText [19]
  • reStructuredText Markup Specification [20]
  • reStructuredText Directives [21]

In addition, Problems With StructuredText [22] explains many markup decisions made with regards to StructuredText, and A Record of reStructuredText Syntax Alternatives [23] records markup decisions made independently.

Docstring-Significant Features

  • A markup escaping mechanism.

    Backslashes (\) are used to escape markup characters when needed for non-markup purposes. However, the inline markup recognition rules have been constructed in order to minimize the need for backslash-escapes. For example, although asterisks are used for emphasis, in non-markup contexts such as "*" or "(*)" or "x * y", the asterisks are not interpreted as markup and are left unchanged. For many non-markup uses of backslashes (e.g., describing regular expressions), inline literals or literal blocks are applicable; see the next item.

  • Markup to include Python source code and Python interactive sessions: inline literals, literal blocks, and doctest blocks.

    Inline literals use double-backquotes to indicate program I/O or code snippets. No markup interpretation (including backslash-escape [\] interpretation) is done within inline literals.

    Literal blocks (block-level literal text, such as code excerpts or ASCII graphics) are indented, and indicated with a double-colon ("::") at the end of the preceding paragraph (right here -->):

    if literal_block:
    text = 'is left as-is'
    spaces_and_linebreaks = 'are preserved'
    markup_processing = None

    Doctest blocks begin with ">>> " and end with a blank line. Neither indentation nor literal block double-colons are required. For example:

    Here's a doctest block:

>>> print 'Python-specific usage examples; begun with ">>>"'
Python-specific usage examples; begun with ">>>"
>>> print '(cut and pasted from interactive sessions)'
(cut and pasted from interactive sessions)

  • Markup that isolates a Python identifier: interpreted text.

    Text enclosed in single backquotes is recognized as "interpreted text", whose interpretation is application-dependent. In the context of a Python docstring, the default interpretation of interpreted text is as Python identifiers. The text will be marked up with a hyperlink connected to the documentation for the identifier given. Lookup rules are the same as in Python itself: LGB namespace lookups (local, global, builtin). The "role" of the interpreted text (identifying a class, module, function, etc.) is determined implicitly from the namespace lookup. For example:

    class Keeper(Storer):

    """
    Keep data fresher longer.

    Extend `Storer`.  Class attribute `instances` keeps track
    of the number of `Keeper` objects instantiated.
    """

    instances = 0
    """How many `Keeper` objects are there?"""

    def __init__(self):
        """
        Extend `Storer.__init__()` to keep track of
        instances.  Keep count in `self.instances` and data
        in `self.data`.
        """
        Storer.__init__(self)
        self.instances += 1

        self.data = []
        """Store data in a list, most recent last."""

    def storedata(self, data):
        """
        Extend `Storer.storedata()`; append new `data` to a
        list (in `self.data`).
        """
        self.data = data

    Each piece of interpreted text is looked up according to the local namespace of the block containing its docstring.

  • Markup that isolates a Python identifier and specifies its type: interpreted text with roles.

    Although the Python source context reader is designed not to require explicit roles, they may be used. To classify identifiers explicitly, the role is given along with the identifier in either prefix or suffix form:

    Use :method:`Keeper.storedata` to store the object's data in
`Keeper.data`:instance_attribute:.

    The syntax chosen for roles is verbose, but necessarily so (if anyone has a better alternative, please post it to the Doc-SIG [6]). The intention of the markup is that there should be little need to use explicit roles; their use is to be kept to an absolute minimum.

  • Markup for "tagged lists" or "label lists": field lists.

    Field lists represent a mapping from field name to field body. These are mostly used for extension syntax, such as "bibliographic field lists" (representing document metadata such as author, date, and version) and extension attributes for directives (see below). They may be used to implement methodologies (docstring semantics), such as identifying parameters, exceptions raised, etc.; such usage is beyond the scope of this PEP.

    A modified RFC 2822 syntax is used, with a colon before as well as after the field name. Field bodies are more versatile as well; they may contain multiple field bodies (even nested field lists). For example:

    :Date: 2002-03-22
:Version: 1
:Authors:
    - Me
    - Myself
    - I

    Standard RFC 2822 header syntax cannot be used for this construct because it is ambiguous. A word followed by a colon at the beginning of a line is common in written text.

  • Markup extensibility: directives and substitutions.

    Directives are used as an extension mechanism for reStructuredText, a way of adding support for new block-level constructs without adding new syntax. Directives for images, admonitions (note, caution, etc.), and tables of contents generation (among others) have been implemented. For example, here's how to place an image:

    .. image:: mylogo.png

    Substitution definitions allow the power and flexibility of block-level directives to be shared by inline text. For example:

    The |biohazard| symbol must be used on containers used to
dispose of medical waste.

.. |biohazard| image:: biohazard.png

  • Section structure markup.

    Section headers in reStructuredText use adornment via underlines (and possibly overlines) rather than indentation. For example:

    This is a Section Title
=======================

This is a Subsection Title
--------------------------

This paragraph is in the subsection.

This is Another Section Title
=============================

This paragraph is in the second section.

Questions & Answers

  1. Is reStructuredText rich enough?

    Yes, it is for most people. If it lacks some construct that is required for a specific application, it can be added via the directive mechanism. If a useful and common construct has been overlooked and a suitably readable syntax can be found, it can be added to the specification and parser.

  2. Is reStructuredText too rich?

    For specific applications or individuals, perhaps. In general, no.

    Since the very beginning, whenever a docstring markup syntax has been proposed on the Doc-SIG [6], someone has complained about the lack of support for some construct or other. The reply was often something like, "These are docstrings we're talking about, and docstrings shouldn't have complex markup." The problem is that a construct that seems superfluous to one person may be absolutely essential to another.

    reStructuredText takes the opposite approach: it provides a rich set of implicit markup constructs (plus a generic extension mechanism for explicit markup), allowing for all kinds of documents. If the set of constructs is too rich for a particular application, the unused constructs can either be removed from the parser (via application-specific overrides) or simply omitted by convention.

  3. Why not use indentation for section structure, like StructuredText does? Isn't it more "Pythonic"?

    Guido van Rossum wrote the following in a 2001-06-13 Doc-SIG post:

    I still think that using indentation to indicate sectioning is wrong. If you look at how real books and other print publications are laid out, you'll notice that indentation is used frequently, but mostly at the intra-section level. Indentation can be used to offset lists, tables, quotations, examples, and the like. (The argument that docstrings are different because they are input for a text formatter is wrong: the whole point is that they are also readable without processing.)

    I reject the argument that using indentation is Pythonic: text is not code, and different traditions and conventions hold. People have been presenting text for readability for over 30 centuries. Let's not innovate needlessly.

    See Section Structure via Indentation [25] in Problems With StructuredText [22] for further elaboration.

  4. Why use reStructuredText for PEPs? What's wrong with the existing standard?

    The existing standard for PEPs is very limited in terms of general expressibility, and referencing is especially lacking for such a reference-rich document type. PEPs are currently converted into HTML, but the results (mostly monospaced text) are less than attractive, and most of the value-added potential of HTML (especially inline hyperlinks) is untapped.

    Making reStructuredText a standard markup for PEPs will enable much richer expression, including support for section structure, inline markup, graphics, and tables. In several PEPs there are ASCII graphics diagrams, which are all that plaintext documents can support. Since PEPs are made available in HTML form, the ability to include proper diagrams would be immediately useful.

    Current PEP practices allow for reference markers in the form "[1]" in the text, and the footnotes/references themselves are listed in a section toward the end of the document. There is currently no hyperlinking between the reference marker and the footnote/reference itself (it would be possible to add this to pep2html.py, but the "markup" as it stands is ambiguous and mistakes would be inevitable). A PEP with many references (such as this one ;-) requires a lot of flipping back and forth. When revising a PEP, often new references are added or unused references deleted. It is painful to renumber the references, since it has to be done in two places and can have a cascading effect (insert a single new reference 1, and every other reference has to be renumbered; always adding new references to the end is suboptimal). It is easy for references to go out of sync.

    PEPs use references for two purposes: simple URL references and footnotes. reStructuredText differentiates between the two. A PEP might contain references like this:

    Abstract

    This PEP proposes adding frungible doodads [1] to the core.
    It extends PEP 9876 [2] via the BCA [3] mechanism.

...

References and Footnotes

    [1] http://www.example.org/

    [2] PEP 9876, Let's Hope We Never Get Here
        http://www.python.org/peps/pep-9876.html

    [3] "Bogus Complexity Addition"

    Reference 1 is a simple URL reference. Reference 2 is a footnote containing text and a URL. Reference 3 is a footnote containing text only. Rewritten using reStructuredText, this PEP could look like this:

    Abstract
========

This PEP proposes adding `frungible doodads`_ to the core.  It
extends PEP 9876 [#pep9876]_ via the BCA [#]_ mechanism.

...

References & Footnotes
======================

.. _frungible doodads: http://www.example.org/

.. [#pep9876] PEP 9876, Let's Hope We Never Get Here

.. [#] "Bogus Complexity Addition"

    URLs and footnotes can be defined close to their references if desired, making them easier to read in the source text, and making the PEPs easier to revise. The "References and Footnotes" section can be auto-generated with a document tree transform. Footnotes from throughout the PEP would be gathered and displayed under a standard header. If URL references should likewise be written out explicitly (in citation form), another tree transform could be used.

    URL references can be named ("frungible doodads"), and can be referenced from multiple places in the document without additional definitions. When converted to HTML, references will be replaced with inline hyperlinks (HTML <a> tags). The two footnotes are automatically numbered, so they will always stay in sync. The first footnote also contains an internal reference name, "pep9876", so it's easier to see the connection between reference and footnote in the source text. Named footnotes can be referenced multiple times, maintaining consistent numbering.

    The "#pep9876" footnote could also be written in the form of a citation:

    It extends PEP 9876 [PEP9876]_ ...

.. [PEP9876] PEP 9876, Let's Hope We Never Get Here

    Footnotes are numbered, whereas citations use text for their references.

  5. Wouldn't it be better to keep the docstring and PEP proposals separate?

    The PEP markup proposal may be removed if it is deemed that there is no need for PEP markup, or it could be made into a separate PEP. If accepted, PEP 1, PEP Purpose and Guidelines [1], and PEP 9, Sample PEP Template [2] will be updated.

    It seems natural to adopt a single consistent markup standard for all uses of structured plaintext in Python, and to propose it all in one place.

  6. The existing pep2html.py script converts the existing PEP format to HTML. How will the new-format PEPs be converted to HTML?

    A new version of pep2html.py with integrated reStructuredText parsing has been completed. The Docutils project supports PEPs with a "PEP Reader" component, including all functionality currently in pep2html.py (auto-recognition of PEP & RFC references, email masking, etc.).

  7. Who's going to convert the existing PEPs to reStructuredText?

    PEP authors or volunteers may convert existing PEPs if they like, but there is no requirement to do so. The reStructuredText-based PEPs will coexist with the old PEP standard. The pep2html.py mentioned in answer 6 processes both old and new standards.

  8. Why use reStructuredText for README and other ancillary files?

    The reasoning given for PEPs in answer 4 above also applies to README and other ancillary files. By adopting a standard markup, these files can be converted to attractive cross-referenced HTML and put up on python.org. Developers of other projects can also take advantage of this facility for their own documentation.

  9. Won't the superficial similarity to existing markup conventions cause problems, and result in people writing invalid markup (and not noticing, because the plaintext looks natural)? How forgiving is reStructuredText of "not quite right" markup?

    There will be some mis-steps, as there would be when moving from one programming language to another. As with any language, proficiency grows with experience. Luckily, reStructuredText is a very little language indeed.

    As with any syntax, there is the possibility of syntax errors. It is expected that a user will run the processing system over their input and check the output for correctness.

    In a strict sense, the reStructuredText parser is very unforgiving (as it should be; "In the face of ambiguity, refuse the temptation to guess" [3] applies to parsing markup as well as computer languages). Here's design goal 3 from An Introduction to reStructuredText [19]:

    Unambiguous. The rules for markup must not be open for interpretation. For any given input, there should be one and only one possible output (including error output).

    While unforgiving, at the same time the parser does try to be helpful by producing useful diagnostic output ("system messages"). The parser reports problems, indicating their level of severity (from least to most: debug, info, warning, error, severe). The user or the client software can decide on reporting thresholds; they can ignore low-level problems or cause high-level problems to bring processing to an immediate halt. Problems are reported during the parse as well as included in the output, often with two-way links between the source of the problem and the system message explaining it.

  10. Will the docstrings in the Python standard library modules be converted to reStructuredText?

    No. Python's library reference documentation is maintained separately from the source. Docstrings in the Python standard library should not try to duplicate the library reference documentation. The current policy for docstrings in the Python standard library is that they should be no more than concise hints, simple and markup-free (although many do contain ad-hoc implicit markup).

  11. I want to write all my strings in Unicode. Will anything break?

    The parser fully supports Unicode. Docutils supports arbitrary input and output encodings.

  12. Why does the community need a new structured text design?

    The existing structured text designs are deficient, for the reasons given in "Rationale" above. reStructuredText aims to be a complete markup syntax, within the limitations of the "readable plaintext" medium.

  13. What is wrong with existing documentation methodologies?

    What existing methodologies? For Python docstrings, there is no official standard markup format, let alone a documentation methodology akin to JavaDoc. The question of methodology is at a much higher level than syntax (which this PEP addresses). It is potentially much more controversial and difficult to resolve, and is intentionally left out of this discussion.

References & Footnotes

[1] PEP 1, PEP Guidelines, Warsaw, Hylton (http://www.python.org/peps/pep-0001.html)
[2] PEP 9, Sample PEP Template, Warsaw (http://www.python.org/peps/pep-0009.html)
[3] From The Zen of Python (by Tim Peters) [26] (or just "import this" in Python)
[4] PEP 216, Docstring Format, Zadka (http://www.python.org/peps/pep-0216.html)
[5] http://docutils.sourceforge.net/rst.html
[6] (1, 2, 3, 4) http://www.python.org/sigs/doc-sig/
[7] http://www.w3.org/XML/
[8] http://www.oasis-open.org/cover/general.html
[9] http://docbook.org/tdg/en/html/docbook.html
[10] http://www.w3.org/MarkUp/
[11] http://www.w3.org/MarkUp/#xhtml1
[12] http://www.tug.org/interest.html
[13] http://www.perldoc.com/perl5.6/pod/perlpod.html
[14] http://java.sun.com/j2se/javadoc/
[15] http://docutils.sourceforge.net/mirror/setext.html
[16] http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage
[17] http://docutils.sourceforge.net/docs/user/rst/quickstart.html
[18] http://docutils.sourceforge.net/docs/user/rst/quickref.html
[19] (1, 2) http://docutils.sourceforge.net/docs/ref/rst/introduction.html
[20] http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
[21] http://docutils.sourceforge.net/docs/ref/rst/directives.html
[22] (1, 2) http://docutils.sourceforge.net/docs/dev/rst/problems.html
[23] http://docutils.sourceforge.net/docs/dev/rst/alternatives.html
[24] http://docutils.sourceforge.net/
[25] http://docutils.sourceforge.net/docs/dev/rst/problems.html#section-structure-via-indentation
[26] http://www.python.org/doc/Humor.html#zen

Copyright

This document has been placed in the public domain.

Acknowledgements

Some text is borrowed from PEP 216, Docstring Format [4], by Moshe Zadka.

Special thanks to all members past & present of the Python Doc-SIG [6].

View document source. Generated on: 2004-07-01 23:07 UTC. Generated by Docutils from reStructuredText source.


Inside A Docutils Command-Line Front-End Tool

Author: | David Goodger
Contact: | goodger@python.org
Date: | 2004-06-27
Revision: | 1.1
Copyright: | This document has been placed in the public domain.

The Docutils Publisher class was set up to make building command-line tools easy. All that's required is to choose components and supply settings for variations. Let's take a look at a typical command-line front-end tool, tools/rst2html.py, from top to bottom.

On Unixish systems, it's best to make the file executable (chmod +x file), and supply an interpreter on the first line, the "shebang" or "hash-bang" line:

#!/usr/bin/env python

Windows systems can be set up to associate the Python interpreter with the .py extension.

Next are some comments providing metadata:

# Author: David Goodger
# Contact: goodger@python.org
# Revision: $Revision: ...
# Date: $Date: ...
# Copyright: This module has been placed in the public domain.

The module docstring describes the purpose of the tool:

"""
A minimal front end to the Docutils Publisher, producing HTML.
"""

This next block attempts to invoke locale support for internationalization services, specifically text encoding. It's not supported on all platforms though, so it's forgiving:

try:
    import locale
    locale.setlocale(locale.LC_ALL, '')
except:
    pass

The real work will be done by the code that's imported here:

from docutils.core import publish_cmdline, default_description

We construct a description of the tool, for command-line help:

description = ('Generates (X)HTML documents from standalone '
               'reStructuredText sources.  ' + default_description)

Now we call the Publisher convenience function, which takes over. Most of it's defaults are used ("standalone" Reader, "reStructuredText" Parser, etc.). The HTML Writer is chosen by name, and a description for command-line help is passed in:

publish_cmdline(writer_name='html', description=description)

That's it! The Docutils Publisher takes care of the rest.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


The Docutils Publisher

Author: | David Goodger
Contact: | goodger@python.org
Date: | 2004-06-27
Revision: | 1.2
Copyright: | This document has been placed in the public domain.

Contents

  • Publisher Convenience Functions
    • Configuration
    • Encodings

Publisher Convenience Functions

Each of these functions set up a docutils.core.Publisher object, then call its publish method. docutils.core.Publisher.publish handles everything else. There are five convenience functions in the docutils.core module:

  • publish_cmdline: for command-line front-end tools, like rst2html.py. There are several examples in the tools/ directory. A detailed analysis of one such tool is in Inside A Docutils Command-Line Front-End Tool

  • publish_file: for programmatic use with file-like I/O. In addition to writing the encoded output to a file, also returns the encoded output as a string.

  • publish_string: for programmatic use with string I/O. Returns the encoded output as a string.

  • publish_parts: for programmatic use with string input; returns a dictionary of document parts. Dictionary keys are the names of parts, and values are Unicode strings; encoding is up to the client. Useful when only portions of the processed document are desired. Currently only implemented for the HTML Writer.

    There are examples in the docutils/examples.py module.

  • publish_programmatically: for custom programmatic use. This function implements common code and is used by publish_file, publish_string, and publish_parts. It returns a 2-tuple: the encoded string output and the Publisher object.

Configuration

To pass application-specific setting defaults to the Publisher convenience functions, use the settings_overrides parameter. Pass a dictionary of setting names & values, like this:

overrides = {'input_encoding': 'ascii',
             'output_encoding': 'latin-1'}
output = publish_string(..., settings_overrides=overrides)

Settings from command-line options override configuration file settings, and they override application defaults. For details, see Docutils Runtime Settings. See Docutils Configuration Files for details about individual settings.

Encodings

The default output encoding of Docutils is UTF-8. If you have any non-ASCII in your text, you may have to do a bit more setup. Docutils may introduce some non-ASCII text if you use symbol-footnotes or section numbering.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils Runtime Settings

Author: | David Goodger
Contact: | goodger@python.org
Date: | 2004-06-27
Revision: | 1.6
Copyright: | This document has been placed in the public domain.

Contents

  • Introduction
  • Runtime Settings Processing for Command-Line Tools
  • Runtime Settings Processing From Applications

Introduction

Docutils runtime settings are assembled from several sources: component settings specifications, application settings specifications, configuration files, and command-line options. Docutils overlays default and explicitly specified values from these sources such that settings behave the way we want and expect them to behave.

To understand how Docutils deals with runtime settings, the attributes and parameters involved must first be understood. Begin with the the docstrings of the attributes of the docutils.SettingsSpec base class (in the docutils/__init__.py module):

  • settings_spec
  • settings_defaults
  • settings_default_overrides
  • relative_path_settings
  • config_section
  • config_section_dependencies

Next, several convenience function parameters are also significant (described in the docutils.core.publish_programmatically function docstring):

  • settings, if present, is assumed to be complete and no further runtime settings processing is done.
  • settings_spec is treated as a fourth component (after the Parser, Reader, and Writer).
  • settings_overrides is a dictionary which will override the defaults of the components.
  • config_section sets or overrides an application-specific configuration file section.

Runtime Settings Processing for Command-Line Tools

Following along with the actual code is recommended. The docutils/__init__.py, docutils/core.py, and docutils.frontend modules are described.

  1. A command-line front-end tool imports and calls docutils.core.publish_cmdline. The relevant convenience function parameters are described above.

  2. docutils.core.publish_cmdline initializes a docutils.core.Publisher object, then calls its publish method.

  3. The docutils.core.Publisher object's publish method checks its settings attribute to see if it's defined. If it is, no further runtime settings processing is done.

    If settings is not defined, self.process_command_line is called with the following relevant arguments:

    • settings_spec
    • config_section
    • settings_overrides (in the form of excess keyword arguments, collected in the defaults parameter)

  4. self.process_command_line calls self.setup_option_parser, passing settings_spec, config_section, and defaults.

  5. self.setup_option_parser checks its config_section parameter; if defined, it adds that config file section to settings_spec (or to a new, empty docutils.SettingsSpec object), replacing anything defined earlier. (See Docutils Configuration Files for details.) Then it instantiates a new docutils.frontend.OptionParser object, passing the following relevant arguments:

    • components: A tuple of docutils.SettingsSpec objects, (self.parser, self.reader, self.writer, settings_spec)
    • defaults (originally from settings_overrides)

  6. The docutils.frontend.OptionParser object's __init__ method calls self.populate_from_components with self.components, which consists of self prepended to the components tuple it received. self (docutils.frontend.OptionParser) defines general Docutils settings.

  7. In self.populate_from_components, for each component passed, component.settings_spec is processed and component.settings_defaults is applied. Then, for each component, component.settings_default_overrides is applied. This two-loop process ensures that component.settings_default_overrides can override the default settings of any other component.

  8. Back in docutils.frontend.OptionParser.__init__, the defaults parameter (derived from the settings_overrides parameter of docutils.core.Publisher.publish) is overlaid over self.defaults. So settings_overrides has priority over all SettingsSpec data.

  9. Next, docutils.frontend.OptionParser.__init__ checks if configuration files are enabled (its read_config_files parameter is true, and self.defaults['_disable_config'] is false). If they are enabled (and normally, they are), self.get_standard_config_settings is called. This reads the docutils configuration files, and returns a dictionary of settings. This is then overlaid on self.defaults. So configuration file settings have priority over all software-defined defaults.

  10. Back in the docutils.core.Publisher object, self.setup_option_parser returns the option_parser object to its caller, self.process_command_line.

  11. self.process_command_line calls option_parser.parse_args, which parses all command line options and returns a docutils.frontend.Values object. This is assigned to the docutils.core.Publisher object's self.settings. So command-line options have priority over configuration file settings.

    When option_parser.parse_args is called, the source and destination command-line arguments are also parsed, and assigned to the _source and _destination attributes of what becomes the docutils.core.Publisher object's self.settings.

  12. From docutils.core.Publisher.publish, self.set_io is called with no arguments. If either self.source or self.destination are not set, the corresponding self.set_source and self.set_destination are called, effectively with no arguments.

  13. self.set_source checks for a source_path parameter, and if there is none (which is the case for command-line use), it is taken from self.settings._source. self.source is set by instantiating a self.source_class object. For command-line front-end tools, the default self.source_class is used, docutils.io.FileInput.

  14. self.set_destination does the same job for the destination that self.set_source does for the source (the default self.destination_class is docutils.io.FileOutput).

Runtime Settings Processing From Applications

Applications process runtime settings in a significantly different way than command-line tools do. Instead of calling publish_cmdline, the application calls one of publish_file, publish_string, or publish_parts. These in turn call publish_programmatically, which implements a generic programmatic interface. Although an application may also call publish_programmatically directly, it is not recommended (if it does seem to be necessary, please write to the Docutils-Develop mailing list).

publish_programmatically accepts the same convenience function parameters as publish_cmdline. Where things differ is that programmatic use does not

TO BE COMPLETED.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils Internationalization

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Date: | 2004-04-17
Revision: | 1.11
Copyright: | This document has been placed in the public domain.

Contents

  • Language Module Names
  • Python Code
  • Docutils Language Module
  • reStructuredText Language Module
  • Testing the Language Modules

This document describes the internationalization facilities of the Docutils project. Introduction to i18n by Tomohiro KUBOTA is a good general reference. "Internationalization" is often abbreviated as "i18n": "i" + 18 letters + "n".

Note

The i18n facilities of Docutils should be considered a "first draft". They work so far, but improvements are welcome. Specifically, standard i18n facilities like "gettext" have yet to be explored.

Docutils is designed to work flexibly with text in multiple languages (one language at a time). Language-specific features are (or should be 1) fully parameterized. To enable a new language, two modules have to be added to the project: one for Docutils itself (the Docutils Language Module) and one for the reStructuredText parser (the reStructuredText Language Module).

[1] If anything in Docutils is insufficiently parameterized, it should be considered a bug. Please report bugs to the Docutils project bug tracker on SourceForge at http://sourceforge.net/tracker/?group_id=38414&atid=422030.

Language Module Names

Language modules are named using a case-insensitive language identifier as defined in RFC 1766, converting hyphens to underscores 2. A typical language identifier consists of a 2-letter language code from ISO 639 (3-letter codes can be used if no 2-letter code exists; RFC 1766 is currently being revised to allow 3-letter codes). The language identifier can have an optional subtag, typically for variations based on country (from ISO 3166 2-letter country codes). If no language identifier is specified, the default is "en" for English. Examples of module names include en.py, fr.py, ja.py, and pt_br.py.

[2] Subtags are separated from primary tags by underscores instead of hyphens, to conform to Python naming rules.

Python Code

All Python code in Docutils will be ASCII-only. In language modules, Unicode-escapes will have to be used for non-ASCII characters. Although it may be possible for some developers to store non-ASCII characters directly in strings, it will cause problems for other developers whose locales are set up differently.

PEP 263 introduces source code encodings to Python modules, implemented beginning in Python 2.3. Until PEP 263 is fully implemented as a well-established convention, proven robust in daily use, and the tools (editors, CVS, email, etc.) recognize this convention, Docutils shall remain conservative.

As mentioned in the note above, developers are invited to explore "gettext" and other i18n technologies.

Docutils Language Module

Modules in docutils/languages contain language mappings for markup-independent language-specific features of Docutils. To make a new language module, just copy the en.py file, rename it with the code for your language (see Language Module Names above), and translate the terms as described below.

Each Docutils language module contains three module attributes:

labels

This is a mapping of node class names to language-dependent boilerplate label text. The label text is used by Writer components when they encounter document tree elements whose class names are the mapping keys.

The entry values (not the keys) should be translated to the target language.

bibliographic_fields

This is a mapping of language-dependent field names (converted to lower case) to canonical field names (keys of DocInfo.biblio_notes in docutils.transforms.frontmatter). It is used when transforming bibliographic fields.

The keys should be translated to the target language.

author_separators

This is a list of strings used to parse the 'Authors' bibliographic field. They separate individual authors' names, and are tried in order (i.e., earlier items take priority, and the first item that matches wins). The English-language module defines them as [';', ',']; semi-colons can be used to separate names like "Arthur Pewtie, Esq.".

Most languages won't have to "translate" this list.

reStructuredText Language Module

Modules in docutils/parsers/rst/languages contain language mappings for language-specific features of the reStructuredText parser. To make a new language module, just copy the en.py file, rename it with the code for your language (see Language Module Names above), and translate the terms as described below.

Each reStructuredText language module contains just one module attribute:

directives

This is a mapping from language-dependent directive names to canonical directive names. The canonical directive names are registered in docutils/parsers/rst/directives/__init__.py, in _directive_registry.

The keys should be translated to the target language. Synonyms (multiple keys with the same values) are allowed; this is useful for abbreviations.

roles

This is a mapping language-dependent role names to canonical role names for interpreted text. The canonical directive names are registered in docutils/parsers/rst/states.py, in Inliner._interpreted_roles (this may change).

The keys should be translated to the target language. Synonyms (multiple keys with the same values) are allowed; this is useful for abbreviations.

Testing the Language Modules

Whenever a new language module is added or an existing one modified, the unit tests should be run. The test modules can be found in the docutils/test directory from CVS or from the latest CVS snapshot.

The test_language.py module can be run as a script. With no arguments, it will test all language modules. With one or more language codes, it will test just those languages. For example:

$ python test_language.py en
..
----------------------------------------
Ran 2 tests in 0.095s

OK

Use the "alltests.py" script to run all test modules, exhaustively testing the parser and other parts of the Docutils system.

View document source. Generated on: 2004-07-01 22:45 UTC. Generated by Docutils from reStructuredText source.


Creating reStructuredText Directives

Author: | Dethe Elza
Author: | David Goodger
Contact: | delza@enfoldingsystems.com
Date: | 2004-06-05
Revision: | 1.11
Copyright: | This document has been placed in the public domain.

Directives are the primary extension mechanism of reStructuredText. This document aims to make the creation of new directives as easy and understandable as possible. There are only a couple of reStructuredText-specific features the developer needs to know to create a basic directive.

The syntax of directives is detailed in the reStructuredText Markup Specification, and standard directives are described in reStructuredText Directives.

Directives are a reStructuredText markup/parser concept. There is no "directive" element, no single element that corresponds exactly to the concept of directives. Instead, choose the most appropriate elements from the existing Docutils elements. Directives build structures using the existing building blocks. See The Docutils Document Tree and the docutils.nodes module for more about the building blocks of Docutils documents.

Table of Contents

  • Define the Directive Function
  • Specify Directive Arguments, Options, and Content
  • Register the Directive
  • Examples
    • Admonitions
    • "image"
    • "contents"

Define the Directive Function

The directive function does any processing that the directive requires. This may require the use of other parts of the reStructuredText parser. This is where the directive actually does something.

The directive implementation itself is a callback function whose signature is as follows:

def directive_fn(name, arguments, options, content, lineno,
                 content_offset, block_text, state, state_machine):
    code...

# Set function attributes:
directive_fn.arguments = ...
directive_fn.options = ...
direcitve_fn.content = ...

Function attributes are described below (see Specify Directive Arguments, Options, and Content). The directive function parameters are as follows:

  • name is the directive type or name.
  • arguments is a list of positional arguments, as specified in the arguments function attribute.
  • options is a dictionary mapping option names to values. The options handled by a directive function are specified in the options function attribute.
  • content is a list of strings, the directive content. Use the content function attribute to allow directive content.
  • lineno is the line number of the first line of the directive.
  • content_offset is the line offset of the first line of the content from the beginning of the current input. Used when initiating a nested parse.
  • block_text is a string containing the entire directive. Include it as the content of a literal block in a system message if there is a problem.
  • state is the state which called the directive function.
  • state_machine is the state machine which controls the state which called the directive function.

Directive functions return a list of nodes which will be inserted into the document tree at the point where the directive was encountered. This can be an empty list if there is nothing to insert. For ordinary directives, the list must contain body elements or structural elements. Some directives are intended specifically for substitution definitions, and must return a list of Text nodes and/or inline elements (suitable for inline insertion, in place of the substitution reference). Such directives must verify substitution definition context, typically using code like this:

if not isinstance(state, states.SubstitutionDef):
    error = state_machine.reporter.error(
        'Invalid context: the "%s" directive can only be used '
        'within a substitution definition.' % (name),
        nodes.literal_block(block_text, block_text), line=lineno)
    return [error]

Specify Directive Arguments, Options, and Content

Function attributes are interpreted by the directive parser (from the docutils.parsers.rst.states.Body.run_directive() method). If unspecified, directive function attributes are assumed to have the value None. Three directive function attributes are recognized:

  • arguments: A 3-tuple specifying the expected positional arguments, or None if the directive has no arguments. The 3 items in the tuple are:

    1. The number of required arguments.
    2. The number of optional arguments.
    3. A boolean, indicating if the final argument may contain whitespace.

    Arguments are normally single whitespace-separated words. The final argument may contain whitespace when indicated by the value 1 (True) for the third item in the argument spec tuple. In this case, the final argument in the arguments parameter to the directive function will contain spaces and/or newlines, preserved from the input text.

    If the form of the arguments is more complex, specify only one argument (either required or optional) and indicate that final whitespace is OK (1/True); the client code must do any context-sensitive parsing.

  • options: The option specification. None or an empty dict implies no options to parse.

    An option specification must be defined detailing the options available to the directive. An option spec is a mapping of option name to conversion function; conversion functions are applied to each option value to check validity and convert them to the expected type. Python's built-in conversion functions are often usable for this, such as int, float, and bool (included in Python from version 2.2.1). Other useful conversion functions are included in the docutils.parsers.rst.directives package (in the __init__.py module):

    • flag: For options with no option arguments. Checks for an argument (raises ValueError if found), returns None for valid flag options.
    • unchanged_required: Returns the text argument, unchanged. Raises ValueError if no argument is found.
    • unchanged: Returns the text argument, unchanged. Returns an empty string ("") if no argument is found.
    • path: Returns the path argument unwrapped (with newlines removed). Raises ValueError if no argument is found or if the path contains internal whitespace.
    • nonnegative_int: Checks for a nonnegative integer argument, and raises ValueError if not.
    • class_option: Converts the argument into an ID-compatible string and returns it. Raises ValueError if no argument is found.

    A further utility function, choice, is supplied to enable options whose argument must be a member of a finite set of possible values. A custom conversion function must be written to use it. For example:

    from docutils.parsers.rst import directives

def yesno(argument):
    return directives.choice(argument, ('yes', 'no'))

    For example, here is an option spec for a directive which allows two options, "name" and "value", each with an option argument:

    directive_fn.options = {'name': unchanged, 'value': int}

  • content: A boolean; true if content is allowed. Directive functions must handle the case where content is required but not present in the input text (an empty content list will be supplied).

The final step of the run_directive() method is to call the directive function itself.

Register the Directive

If the directive is a general-use addition to the Docutils core, it must be registered with the parser and language mappings added:

  1. Register the new directive using its canonical name in docutils/parsers/rst/directives/__init__.py, in the _directive_registry dictionary. This allows the reStructuredText parser to find and use the directive.
  2. Add an entry to the directives dictionary in docutils/parsers/rst/languages/en.py for the directive, mapping the English name to the canonical name (both lowercase). Usually the English name and the canonical name are the same.
  3. Update all the other language modules as well. For languages in which you are proficient, please add translations. For other languages, add the English directive name plus "(translation required)".

If the directive is application-specific, use the register_directive function:

from docutils.parsers.rst import directives
directives.register_directive(directive_name, directive_function)

Examples

For the most direct and accurate information, "Use the Source, Luke!". All standard directives are documented in reStructuredText Directives, and the source code implementing them is located in the docutils/parsers/rst/directives package. The __init__.py module contains a mapping of directive name to module & function name. Several representative directives are described below.

Admonitions

Admonition directives, such as "note" and "caution", are quite simple. They have no directive arguments or options. Admonition directive content is interpreted as ordinary reStructuredText. The directive function simply hands off control to a generic directive function:

def note(*args):
    return admonition(nodes.note, *args)

attention.content = 1

Note that the only thing distinguishing the various admonition directives is the element (node class) generated. In the code above, the node class is passed as the first argument to the generic directive function (early version), where the actual processing takes place:

def admonition(node_class, name, arguments, options, content, lineno,
               content_offset, block_text, state, state_machine):
    text = '\n'.join(content)
    admonition_node = node_class(text)
    if text:
        state.nested_parse(content, content_offset, admonition_node)
        return [admonition_node]
    else:
        warning = state_machine.reporter.warning(
            'The "%s" admonition is empty; content required.'
            % (name), '',
            nodes.literal_block(block_text, block_text), line=lineno)
        return [warning]

Three things are noteworthy in the function above:

  1. The admonition_node = node_class(text) line creates the wrapper element, using the class passed in from the initial (stub) directive function.
  2. The call to state.nested_parse() is what does the actual processing. It parses the directive content and adds any generated elements as child elements of admonition_node.
  3. If there was no directive content, a warning is generated and returned. The call to state_machine.reporter.warning() includes a literal block containing the entire directive text (block_text) and the line (lineno) of the top of the directive.

"image"

The "image" directive is used to insert a picture into a document. This directive has one argument, the path to the image file, and supports several options. There is no directive content. Here's an early version of the image directive function:

def image(name, arguments, options, content, lineno,
          content_offset, block_text, state, state_machine):
    reference = ''.join(arguments[0].split('\n'))
    if reference.find(' ') != -1:
        error = state_machine.reporter.error(
              'Image URI contains whitespace.', '',
              nodes.literal_block(block_text, block_text),
              line=lineno)
        return [error]
    options['uri'] = reference
    image_node = nodes.image(block_text, **options)
    return [image_node]

image.arguments = (1, 0, 1)
image.options = {'alt': directives.unchanged,
                 'height': directives.nonnegative_int,
                 'width': directives.nonnegative_int,
                 'scale': directives.nonnegative_int,
                 'align': align}

Several things are noteworthy in the code above:

  1. The "image" directive requires a single argument, which is allowed to contain whitespace (see the argument spec above, image.arguments = (1, 0, 1)). This is to allow for long URLs which may span multiple lines. The first line of the image function joins the URL, discarding any embedded newlines. Then the result is checked for embedded spaces, which are not allowed.

  2. The reference is added to the options dictionary under the "uri" key; this becomes an attribute of the nodes.image element object. Any other attributes have already been set explicitly in the source text.

  3. The "align" option depends on the following definitions (which actually occur earlier in the source code):

    align_values = ('top', 'middle', 'bottom', 'left', 'center',
                'right')

def align(argument):
    return directives.choice(argument, align_values)

"contents"

The "contents" directive is used to insert an auto-generated table of contents (TOC) into a document. It takes one optional argument, a title for the TOC. If no title is specified, a default title is used instead. The directive also handles several options. Here's an early version of the code:

def contents(name, arguments, options, content, lineno,
             content_offset, block_text, state, state_machine):
    """Table of contents."""
    if arguments:
        title_text = arguments[0]
        text_nodes, messages = state.inline_text(title_text, lineno)
        title = nodes.title(title_text, '', *text_nodes)
    else:
        messages = []
        title = None
    pending = nodes.pending(parts.Contents, {'title': title},
                            block_text)
    pending.details.update(options)
    state_machine.document.note_pending(pending)
    return [pending] + messages

contents.arguments = (0, 1, 1)
contents.options = {'depth': directives.nonnegative_int,
                    'local': directives.flag,
                    'backlinks': backlinks}

Aspects of note include:

  1. The contents.arguments = (0, 1, 1) function attribute specifies a single, optional argument. If no argument is present, the arguments parameter to the directive function will be an empty list.
  2. If an argument is present, its text is passed to state.inline_text() for parsing. Titles may contain inline markup, such as emphasis or inline literals.
  3. The table of contents is not generated right away. Typically, a TOC is placed near the beginning of a document, and is a summary or outline of the section structure of the document. The entire document must already be processed before a summary can be made. This directive leaves a nodes.pending placeholder element in the document tree, marking the position of the TOC and including a details internal attribute containing all the directive options, effectively communicating the options forward. The actual table of contents processing is performed by a transform, docutils.transforms.parts.Contents, after the rest of the document has been parsed.
View document source. Generated on: 2004-07-01 22:45 UTC. Generated by Docutils from reStructuredText source.


Creating reStructuredText Interpreted Text Roles

Author: | David Goodger
Contact: | goodger@python.org
Date: | 2004-06-03
Revision: | 1.5
Copyright: | This document has been placed in the public domain.

Interpreted text roles are an extension mechanism for inline markup in reStructuredText. This document aims to make the creation of new roles as easy and understandable as possible.

Standard roles are described in reStructuredText Interpreted Text Roles. See the Interpreted Text section in the reStructuredText Markup Specification for syntax details.

Contents

  • Define the Role Function
  • Specify Role Function Options and Content
  • Register the Role
  • Examples
    • Generic Roles
    • RFC Reference Role

Define the Role Function

The role function creates and returns inline elements (nodes) and does any additional processing required. Its signature is as follows:

def role_fn(name, rawtext, text, lineno, inliner,
            options={}, content=[]):
    code...

# Set function attributes for customization:
role_fn.options = ...
role_fn.content = ...

Function attributes are described below (see Specify Role Function Options and Content). The role function parameters are as follows:

  • name: The local name of the interpreted role, the role name actually used in the document.
  • rawtext: A string containing the enitre interpreted text input, including the role and markup. Return it as a problematic node linked to a system message if a problem is encountered.
  • text: The interpreted text content.
  • lineno: The line number where the interpreted text begins.
  • inliner: The docutils.parsers.rst.states.Inliner object that called role_fn. It contains the several attributes useful for error reporting and document tree access.
  • options: A dictionary of directive options for customization (from the "role" directive), to be interpreted by the role function. Used for additional attributes for the generated elements and other functionality.
  • content: A list of strings, the directive content for customization (from the "role" directive). To be interpreted by the role function.

Role functions return a tuple of two values:

  • A list of nodes which will be inserted into the document tree at the point where the interpreted role was encountered (can be an empty list).
  • A list of system messages, which will be inserted into the document tree immediately after the end of the current block (can also be empty).

Specify Role Function Options and Content

Function attributes are for customization, and are interpreted by the "role" directive. If unspecified, role function attributes are assumed to have the value None. Two function attributes are recognized:

  • options: The option specification. All role functions implicitly support the "class" option, unless disabled with an explicit {'class': None}.

    An option specification must be defined detailing the options available to the "role" directive. An option spec is a mapping of option name to conversion function; conversion functions are applied to each option value to check validity and convert them to the expected type. Python's built-in conversion functions are often usable for this, such as int, float, and bool (included in Python from version 2.2.1). Other useful conversion functions are included in the docutils.parsers.rst.directives package. For further details, see Creating reStructuredText Directives.

  • content: A boolean; true if "role" directive content is allowed. Role functions must handle the case where content is required but not supplied (an empty content list will be supplied).

    As of this writing, no roles accept directive content.

Note that unlike directives, the "arguments" function attribute is not supported for role customization. Directive arguments are handled by the "role" directive itself.

Register the Role

If the role is a general-use addition to the Docutils core, it must be registered with the parser and language mappings added:

  1. Register the new role using the canonical name:

    from docutils.parsers.rst import roles
roles.register_canonical_role(name, role_function)

    This code is normally placed immediately after the definition of the role funtion.

  2. Add an entry to the roles dictionary in docutils/parsers/rst/languages/en.py for the role, mapping the English name to the canonical name (both lowercase). Usually the English name and the canonical name are the same. Abbreviations and other aliases may also be added here.

  3. Update all the other language modules as well. For languages in which you are proficient, please add translations. For other languages, add the English role name plus "(translation required)".

If the role is application-specific, use the register_local_role function:

from docutils.parsers.rst import roles
roles.register_local_role(name, role_function)

Examples

For the most direct and accurate information, "Use the Source, Luke!". All standard roles are documented in reStructuredText Interpreted Text Roles, and the source code implementing them is located in the docutils/parsers/rst/roles.py module. Several representative roles are described below.

Generic Roles

Many roles simply wrap a given element around the text. There's a special helper function, register_generic_role, which generates a role function from the canonical role name and node class:

register_generic_role('emphasis', nodes.emphasis)

For the implementation of register_generic_role, see the docutils.parsers.rst.roles module.

RFC Reference Role

This role allows easy references to RFCs (Request For Comments documents) by automatically providing the base URL, http://www.faqs.org/rfcs/, and appending the RFC document itself (rfcXXXX.html, where XXXX is the RFC number). For example:

See :RFC:`2822` for information about email headers.

This is equivalent to:

See `RFC 2822`__ for information about email headers.

__ http://www.faqs.org/rfcs/rfc2822.html

Here is the implementation of the role:

def rfc_reference_role(role, rawtext, text, lineno, inliner,
                       options={}, content=[]):
    try:
        rfcnum = int(text)
        if rfcnum <= 0:
            raise ValueError
    except ValueError:
        msg = inliner.reporter.error(
            'RFC number must be a number greater than or equal to 1; '
            '"%s" is invalid.' % text, line=lineno)
        prb = inliner.problematic(rawtext, rawtext, msg)
        return [prb], [msg]
    # Base URL mainly used by inliner.rfc_reference, so this is correct:
    ref = inliner.rfc_url % rfcnum
    node = nodes.reference(rawtext, 'RFC ' + text, refuri=ref, **options)
    return [node], []

register_canonical_role('rfc-reference', rfc_reference_role)

Noteworthy in the code above are:

  1. The interpreted text itself should contain the RFC number. The try clause verifies by converting it to an integer. If the conversion fails, the except clause is executed: a system message is generated, the entire interpreted text construct (in rawtext) is wrapped in a problematic node (linked to the system message), and the two are returned.
  2. The RFC reference itself is constructed from a stock URI, set as the "refuri" attribute of a "reference" element.
  3. The options function parameter, a dictionary, may contain a "class" customization attribute; it is passed through to the "reference" element node constructor.
View document source. Generated on: 2004-07-01 22:45 UTC. Generated by Docutils from reStructuredText source.


Plan for Enthought API Documentation Tool

Author: | David Goodger
Contact: | goodger@python.org
Date: | 2004-06-10
Revision: | 1.4
Copyright: | 2004 by Enthought, Inc.
License: | Enthought License (BSD-style)

This document should be read in conjunction with the Enthought API Documentation Tool RFP prepared by Janet Swisher.

Contents

  • 1 Introduction
  • 2 Development Plan
  • 3 Repository
  • 4 Copyright & License
  • 5 Proposed Changes to reStructuredText
    • 5.1 Doc Comment Syntax
      • 5.1.1 Recommendation
    • 5.2 Docstring Density & Whitespace Minimization
      • 5.2.1 Recommendation

1 Introduction

In March 2004 at I met Eric Jones, president and CTO of Enthought, Inc., at PyCon 2004 in Washington DC. He told me that Enthought was using reStructuredText for source code documentation, but they had some issues. He asked if I'd be interested in doing some work on a customized API documentation tool. Shortly after PyCon, Janet Swisher, Enthought's senior technical writer, contacted me to work out details. Some email, a trip to Austin in May, and plenty of Texas hospitality later, we had a project. This document will record the details, milestones, and evolution of the project.

In a nutshell, Enthought is sponsoring the implementation of an open source API documentation tool that meets their needs. Fortuitously, their needs coincide well with the "Python Source Reader" description in PEP 258. In other words, Enthought is funding some significant improvements to Docutils, improvements that were planned but never implemented due to time and other constraints. The implementation will take place gradually over several months, on a part-time basis.

This is an ideal example of cooperation between a corporation and an open-source project. The corporation, the project, I personally, and the community all benefit. Enthought, whose commitment to open source is also evidenced by their sponsorship of SciPy, benefits by obtaining a useful piece of software, much more quickly than would have been possible without their support. Docutils benefits directly from the implementation of one of its core subsystems. I benefit from the funding, which allows me to justify the long hours to my wife and family. All the corporations, projects, and individuals that make up the community will benefit from the end result, which will be great.

All that's left now is to actually do the work!

2 Development Plan

  1. Analyze prior art, most notably Epydoc and HappyDoc, to see how they do what they do. I have no desire to reinvent wheels unnecessarily. I want to take the best ideas from each tool, combined with the outline in PEP 258 (which will evolve), and build at least the foundation of the definitive Python auto-documentation tool.
  2. Decide on a base platform. The best way to achieve Enthought's goals in a reasonable time frame may be to extend Epydoc or HappyDoc. Or it may be necessary to start fresh.
  3. Extend the reStructuredText parser. See Proposed Changes to reStructuredText below.
  4. Depending on the base platform chosen, build or extend the docstring & doc comment extraction tool. This may be the biggest part of the project, but I won't be able to break it down into details until more is known.

3 Repository

If possible, all software and documentation files will be stored in the CVS repository of Docutils and/or the base project, which are all publicly-available via anonymous pserver access.

The Docutils project is very open about granting CVS write access; so far, everyone who asked has been given access. Any Enthought staff member who would like CVS write access will get it.

If either Epydoc or HappyDoc is chosen as the base platform, I will ask the project's administrator for CVS access for myself and any Enthought staff member who wants it. If sufficient access is not granted -- although I doubt that there would be any problem -- we may have to begin a fork, which could be hosted on SourceForge, on Enthought's Subversion server, or anywhere else deemed appropriate.

4 Copyright & License

Most existing Docutils files have been placed in the public domain, as follows:

:Copyright: This document has been placed in the public domain.

This is in conjunction with the "Public Domain Dedication" section of COPYING.txt.

The code and documentation originating from Enthought funding will have Enthought's copyright and license declaration. While I will try to keep Enthought-specific code and documentation separate from the existing files, there will inevitably be cases where it makes the most sense to extend existing files.

I propose the following:

  1. New files related to this Enthought-funded work will be identified with the following field-list headers:

    :Copyright: 2004 by Enthought, Inc.
:License: Enthought License (BSD Style)

    The license field text will be linked to the license file itself.

  2. For significant or major changes to an existing file (more than 10% change), the headers shall change as follows (for example):

    :Copyright: 2001-2004 by David Goodger
:Copyright: 2004 by Enthought, Inc.
:License: BSD-style

    If the Enthought-funded portion becomes greater than the previously existing portion, Enthought's copyright line will be shown first.

  3. In cases of insignificant or minor changes to an existing file (less than 10% change), the public domain status shall remain unchanged.

A section describing all of this will be added to the Docutils COPYING instructions file.

If another project is chosen as the base project, similar changes would be made to their files, subject to negotiation.

5 Proposed Changes to reStructuredText

5.1 Doc Comment Syntax

The "traits" construct is implemented as dictionaries, where standalone strings would be Python syntax errors. Therefore traits require documentation in comments. We also need a way to differentiate between ordinary "internal" comments and documentation comments (doc comments).

Javadoc uses the following syntax for doc comments:

/**
 * The first line of a multi-line doc comment begins with a slash
 * and *two* asterisks.  The doc comment ends normally.
 */

Python doesn't have multi-line comments; only single-line. A similar convention in Python might look like this:

##
# The first line of a doc comment begins with *two* hash marks.
# The doc comment ends with the first non-comment line.
'data' : AnyValue,

## The double-hash-marks could occur on the first line of text,
#  saving a line in the source.
'data' : AnyValue,

How to indicate the end of the doc comment?

##
# The first line of a doc comment begins with *two* hash marks.
# The doc comment ends with the first non-comment line, or another
# double-hash-mark.
##
# This is an ordinary, internal, non-doc comment.
'data' : AnyValue,

## First line of a doc comment, terse syntax.
#  Second (and last) line.  Ends here: ##
# This is an ordinary, internal, non-doc comment.
'data' : AnyValue,

Or do we even need to worry about this case? A simple blank line could be used:

## First line of a doc comment, terse syntax.
#  Second (and last) line.  Ends with a blank line.

# This is an ordinary, internal, non-doc comment.
'data' : AnyValue,

Other possibilities:

#" Instead of double-hash-marks, we could use a hash mark and a
#  quotation mark to begin the doc comment.
'data' : AnyValue,

## We could require double-hash-marks on every line.  This has the
## added benefit of delimiting the *end* of the doc comment, as
## well as working well with line wrapping in Emacs
## ("fill-paragraph" command).
# Ordinary non-doc comment.
'data' : AnyValue,

#" A hash mark and a quotation mark on each line looks funny, and
#" it doesn't work well with line wrapping in Emacs.
'data' : AnyValue,

These styles (repeated on each line) work well with line wrapping in Emacs:

##  #>  #|  #-  #%  #!  #*

These styles do not work well with line wrapping in Emacs:

#"  #'  #:  #)  #.  #/  #@  #$  #^  #=  #+  #_  #~

The style of doc comment indicator used could be a runtime, global and/or per-module setting. That may add more complexity than it's worth though.

5.1.1 Recommendation

I recommend adopting "#*" on every line:

# This is an ordinary non-doc comment.

#* This is a documentation comment, with an asterisk after the
#* hash marks on every line.
'data' : AnyValue,

I initially recommended adopting double-hash-marks:

# This is an ordinary non-doc comment.

## This is a documentation comment, with double-hash-marks on
## every line.
'data' : AnyValue,

But Janet Swisher rightly pointed out that this could collide with ordinary comments that are then block-commented. This applies to double-hash-marks on the first line only as well. So they're out.

5.2 Docstring Density & Whitespace Minimization

One problem with extensively documented classes & functions, is that there is a lot of screen space wasted on whitespace. Here's some current Enthought code (from lib/cp/fluids/gassmann.py):

def max_gas(temperature, pressure, api, specific_gravity=.56):
    """
    Computes the maximum dissolved gas in oil using Batzle and
    Wang (1992).

    Parameters
    ----------
    temperature : sequence
        Temperature in degrees Celsius
    pressure: sequence
        Pressure in MPa
    api : sequence
        Stock tank oil API
    specific_gravity : sequence
        Specific gravity of gas at STP, default is .56

    Returns
    -------
    max_gor : sequence
        Maximum dissolved gas in liters/liter

    Description
    -----------
    This estimate is based on equations given by Mavko, Mukerji,
    and Dvorkin, (1998, pp. 218-219, or 2003, p. 236) obtained
    originally from Batzle and Wang (1992).
    """
    code...

The docstring is 24 lines long.

Rather than using subsections, field lists (which exist now) can save 5 lines:

def max_gas(temperature, pressure, api, specific_gravity=.56):
    """
    Computes the maximum dissolved gas in oil using Batzle and
    Wang (1992).

    :Parameters:
        temperature : sequence
            Temperature in degrees Celsius
        pressure: sequence
            Pressure in MPa
        api : sequence
            Stock tank oil API
        specific_gravity : sequence
            Specific gravity of gas at STP, default is .56
    :Returns:
        max_gor : sequence
            Maximum dissolved gas in liters/liter
    :Description:
        This estimate is based on equations given by Mavko,
        Mukerji, and Dvorkin, (1998, pp. 218-219, or 2003, p. 236)
        obtained originally from Batzle and Wang (1992).
    """
    code...

The output for field lists is typically a table structure. For example:

Parameters:
temperature : sequence
Temperature in degrees Celsius
pressure: sequence
Pressure in MPa
api : sequence
Stock tank oil API
specific_gravity : sequence
Specific gravity of gas at STP, default is .56
Returns:
max_gor : sequence
Maximum dissolved gas in liters/liter
Description: This estimate is based on equations given by Mavko, Mukerji, and Dvorkin, (1998, pp. 218-219, or 2003, p. 236) obtained originally from Batzle and Wang (1992).

But the definition lists describing the parameters and return values are still wasteful of space. There are a lot of half-filled lines.

Definition lists are currently defined as:

term : classifier
    definition

Where the classifier part is optional.

  • We could allow multiple classifiers:

    term : classifier one : two : three ...
    definition

  • We could allow the definition on the same line as the term, but only in limited and well-known contexts:

    term -- definition

    This is the syntax used by StructuredText (one of reStructuredText's predecessors). It was not adopted for reStructuredText because it is ambiguous -- people often use "--" in their text, as I just did. But given a constrained context, the ambiguity would be acceptable. That context would be: in docstrings, within a field list, perhaps only with certain well-defined field names (parameters, returns).

5.2.1 Recommendation

Combining these ideas, the function definition becomes:

def max_gas(temperature, pressure, api, specific_gravity=.56):
    """
    Computes the maximum dissolved gas in oil using Batzle and
    Wang (1992).

    :Parameters:
        temperature : sequence -- Temperature in degrees Celsius
        pressure: sequence -- Pressure in MPa
        api : sequence -- Stock tank oil API
        specific_gravity : sequence -- Specific gravity of gas at
            STP, default is .56
    :Returns:
        max_gor : sequence -- Maximum dissolved gas in liters/liter
    :Description:
        This estimate is based on equations given by Mavko,
        Mukerji, and Dvorkin, (1998, pp. 218-219, or 2003, p. 236)
        obtained originally from Batzle and Wang (1992).
    """
    code...

The docstring is reduced to 15 lines, from the original 24. For longer docstrings with many parameters and return values, the difference would be more significant.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Enthought API Documentation Tool

Request for Proposals

Author: | Janet Swisher, Senior Technical Writer
Organization: | Enthought, Inc.
Copyright: | 2004 by Enthought, Inc.
License: | Enthought License (BSD Style)

The following is excerpted from the full RFP, and is published here with permission from Enthought, Inc. See the Plan for Enthought API Documentation Tool.

Contents

  • 1 Requirements
    • 1.1 Documentation Extraction
    • 1.2 Source Format
    • 1.3 Output Format
    • 1.4 Output Structure and Navigation
  • 2 License

1 Requirements

The documentation tool will address the following high-level goals:

1.1 Documentation Extraction

  1. Documentation will be generated directly from Python source code, drawing from the code structure, docstrings, and possibly other comments.
  2. The tool will extract logical constructs as appropriate, minimizing the need for comments that are redundant with the code structure. The output should reflect both documented and undocumented elements.

1.2 Source Format

  1. The docstrings will be formatted in as terse syntax as possible. Required tags, syntax, and white space should be minimized.

  2. The tool must support the use of Traits. Special comment syntax for Traits may be necessary. Information about the Traits package is available at http://old.scipy.org/site_content/traits. In the following example, each trait definition is prefaced by a plain comment:

    __traits__ = {

# The current selection within the frame.
'selection' : Trait([], TraitInstance(list)),

# The frame has been activated or deactivated.
'activated' : TraitEvent(),

'closing' : TraitEvent(),

# The frame is closed.
'closed' : TraitEvent(),
}

  3. Support for ReStructuredText (ReST) format is desirable, because much of the existing docstrings uses ReST. However, the complete ReST specification need not be supported, if a subset can achieve the project goals. If the tool does not support ReST, the contractor should also provide a tool or path to convert existing docstrings.

1.3 Output Format

  1. Documentation will be output as a navigable suite of HTML files.
  2. The style of the HTML files will be customizable by a cascading style sheet and/or a customizable template.
  3. Page elements such as headers and footer should be customizable, to support differing requirements from one documentation project to the next.

1.4 Output Structure and Navigation

  1. The navigation scheme for the HTML files should not rely on frames, and should harmonize with conversion to Microsoft HTML Help (.chm) format.

  2. The output should be structured to make navigable the architecture of the Python code. Packages, modules, classes, traits, and functions should be presented in clear, logical hierarchies. Diagrams or trees for inheritance, collaboration, sub-packaging, etc. are desirable but not required.

  3. The output must include indexes that provide a comprehensive view of all packages, modules, and classes. These indexes will provide readers with a clear and exhaustive view of the code base. These indexes should be presented in a way that is easily accessible and allows easy navigation.

  4. Cross-references to other documented elements will be used throughout the documentation, to enable the reader to move quickly relevant information. For example, where type information for an element is available, the type definition should be cross-referenced.

  5. The HTML suite should provide consistent navigation back to the home page, which will include the following information:

    • Bibliographic information
      • Author
      • Copyright
      • Release date
      • Version number
    • Abstract
    • References
      • Links to related internal docs (i.e., other docs for the same product)
      • Links to related external docs (e.g., supporting development docs, Python support docs, docs for included packages)

    It should be possible to specify similar information at the top level of each package, so that packages can be included as appropriate for a given application.

2 License

Enthought intends to release the software under an open-source ("BSD-style") license.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils Project Policies

Author: | David Goodger; open to all Docutils developers
Contact: | goodger@python.org
Date: | 2004-06-25
Revision: | 1.6
Copyright: | This document has been placed in the public domain.

Contents

  • Python Coding Conventions
  • Copyrights and Licensing
  • CVS Check-ins
    • Additions to Docutils
    • CVS Startup File
    • Setting Up For Docutils Development
  • Mailing Lists
  • The Sandbox
  • Parallel Projects

A few quotes sum up the policies of the Docutils project. The IETF's classic credo (by MIT professor Dave Clark) is an ideal we can aspire to:

We reject: kings, presidents, and voting. We believe in: rough consensus and running code.

As architect, chief cook and bottle-washer, I currently function as BDFN (Benevolent Dictator For Now), but I would happily abdicate the throne given a suitable candidate. Any takers?

Eric S. Raymond, anthropologist of the hacker subculture, writes in his essay The Magic Cauldron:

The number of contributors [to] projects is strongly and inversely correlated with the number of hoops each project makes a user go through to contribute.

Therefore, we will endeavour to keep the barrier to entry as low as possible. The policies below should not be thought of as barriers, but merely as a codification of experience to date. These are "best practices", not absolutes; exceptions are expected, tolerated, and used as a source of improvement.

As for control issues, Emmett Plant (CEO of the Xiph.org Foundation, originators of Ogg Vorbis) put it well when he said:

Open source dictates that you lose a certain amount of control over your codebase, and that's okay with us.

Python Coding Conventions

These are the conventions I use in my own code. Contributed code will not be refused merely because it does not strictly adhere to these conditions; as long as it's internally consistent, clean, and correct, it probably will be accepted. But don't be surprised if the "offending" code gets fiddled over time to conform to these conventions.

The Docutils project shall follow the generic coding conventions as specified in the Style Guide for Python Code and Docstring Conventions PEPs, with the following clarifications (from most to least important):

  • 4 spaces per indentation level. No tabs.
  • Use only ASCII, no 8-bit strings. See Docutils Internationalization.
  • No one-liner compound statements (i.e., no if x: return: use two lines & indentation), except for degenerate class or method definitions (i.e., class X: pass is O.K.).
  • Lines should be no more than 78 characters long.
  • Use "StudlyCaps" for class names (except for element classes in docutils.nodes).
  • Use "lowercase" or "lowercase_with_underscores" for function, method, and variable names. For short names, maximum two words, joined lowercase may be used (e.g. "tagname"). For long names with three or more words, or where it's hard to parse the split between two words, use lowercase_with_underscores (e.g., "note_explicit_target", "explicit_target"). If in doubt, use underscores.
  • Use 'single quotes' for string literals, and """triple double quotes""" for docstrings.

Copyrights and Licensing

The majority of the Docutils project code and documentation has been placed in the public domain. Unless clearly and explicitly indicated otherwise, any patches (modifications to existing files) submitted to the project for inclusion (via CVS, SourceForge trackers, mailing lists, or private email) are assumed to be in the public domain as well.

Any new files contributed to the project should clearly state their intentions regarding copyright, in one of the following ways:

  • Public domain (preferred): include the statement "This module/document has been placed in the public domain."
  • Copyright & open source license: include a copyright notice, along with either an embedded license statement, a reference to an accompanying license file, or a license URL.

One of the goals of the Docutils project, once complete, is to be incorporated into the Python standard library. At that time copyright of the Docutils code will be assumed by or transferred to the Python Software Foundation (PSF), and will be released under Python's license. If the copyright/license option is chosen for new files, the license should be compatible with Python's current license, and the author(s) of the files should be willing to assign copyright to the PSF.

CVS Check-ins

Instructions for CVS access can be found at http://sourceforge.net/cvs/?group_id=38414. Anyone can access the CVS repository anonymously. Only project developers can make changes. See Setting Up For Docutils Development below for more info.

Unless you really really know what you're doing, please limit your CVS commands to cvs checkout, cvs commit/checkin, and cvs add. Do NOT use cvs import unless you're absolutely sure you know what you're doing. Even then, grab a copy of the nightly CVS tarball, set it up on your own machine, and experiment there first.

The main source tree ("docutils" CVS module) should always be kept in a stable state (usable and as problem-free as possible). The Docutils project shall follow the Python Check-in Policies (as applicable), with particular emphasis as follows:

  • Before checking in any changes, run the entire Docutils test suite to be sure that you haven't broken anything. From a shell:

    cd docutils/test
alltests.py

  • When adding new functionality (or fixing bugs), be sure to add test cases to the test suite. Practise test-first programming; it's fun, it's addictive, and it works!

  • The sandbox CVS directory is the place to put new, incomplete or experimental code. See Additions to Docutils and The Sandbox below.

  • For bugs or omissions that have an obvious fix and can't possibly mess up anything else, go right ahead and check it in directly.

  • For larger changes, use your best judgement. If you're unsure of the impact, or feel that you require advice or approval, patches or the sandbox are the way to go.

Docutils will pursue an open and trusting policy for as long as possible, and deal with any aberrations if (and hopefully not when) they happen. I'd rather see a torrent of loose contributions than just a trickle of perfect-as-they-stand changes. The occasional mistake is easy to fix. That's what CVS is for.

Additions to Docutils

Additions to the project, such as new components, should be developed in the sandbox CVS directory until they're in good shape, usable, documented, and reasonably complete. Adding to the main source tree or to a parallel project implies a commitment to the Docutils user community.

  • Why the sandbox?

    Developers should be able to try out new components while they're being developed for addition to main source tree. See The Sandbox below.

  • Good shape means that the component code is clean, readable, and free of junk code (unused legacy code; by analogy with "junk DNA").

  • Usable means that the code does what it claims to do. An "XYZ Writer" should produce reasonable XYZ.

  • Documented: The more the better. The modules/files must be at least minimally documented internally. Docutils Front-End Tools should have a new section for any front-end tool that is added. Docutils Configuration Files should be modified with any settings/options defined.

  • Reasonably complete means that the code must handle all input. Here "handle" means that no input can cause the code to fail (cause an exception, or silently and incorrectly produce nothing). "Reasonably complete" does not mean "finished" (no work left to be done). For example, a writer must handle every standard element from the Docutils document model; for unimplemented elements, it must at the very least warn that "Output for element X is not yet implemented in writer Y".

If you really want to check code into the main source tree, you can, but you'll have to be prepared to work on it intensively and complete it quickly. People will start to use it and they will expect it to work! If there are any issues with your code, or if you only have time for gradual development, you should put it in the sandbox first. It's easy to move code over to the main source tree once it's closer to completion.

CVS Startup File

In your ~/.cvsrc file, make sure you have a line like this:

update -dP

This ensures that new directories are created and empty directories are pruned (removed) from your working copy. In addition, these lines are useful:

cvs -q -z3
diff -u

cvs -q suppresses many non-essential messages. -z3 compresses data between client and server. diff -u creates "unified diffs"; specify diff -c if you prefer context diffs.

Setting Up For Docutils Development

When making changes to the code, good developers always test their changes. That means running the code to check that it produces the expected results, and running the test suite too. The modified Docutils code has to be accessible to Python for the tests to have any meaning. There are two ways to keep the Docutils code accessible:

  • Update your PYTHONPATH environment variable so that Python picks up your local working copy of the code. This is the recommended method.

    For the bash shell and Docutils checked out from CVS in ~/projects/docutils/, add this to your ~/.profile:

    PYTHONPATH=$HOME/projects/docutils/docutils
PYTHONPATH=$PYTHONPATH:$HOME/projects/docutils/docutils/extras
export PYTHONPATH

    The first line points to the directory containing the docutils package. The second line adds the directory containing the third-party modules Docutils depends on. The third line exports this environment variable. You may also wish to add the tools directory to your PATH:

    PATH=$PATH:$HOME/projects/docutils/docutils/tools

  • Before you run anything, every time you make a change, reinstall Docutils:

    python setup.py install

    Caution!

    This method is not recommended for day-to-day development; it's too easy to forget. Confusion inevitably ensues.

    If you install Docutils this way, Python will always pick up the last-installed copy of the code. If you ever forget to reinstall the "docutils" package, Python won't see your latest changes.

Mailing Lists

Developers should subscribe to the mailing lists:

  • The Python Documentation Special Interest Group (Doc-SIG) mailing list for high-level discussions on syntax, strategy, and design (email to Doc-SIG@python.org).
  • Docutils-develop, for implementation discussions (email to docutils-develop@lists.sourceforge.net).
  • Docutils-checkins, to monitor CVS checkin messages (automatically generated; normally read-only).
  • Docutils-users, to monitor and help out with usage issues and bug reports.

The Sandbox

The sandbox CVS directory is a place to play around, to try out and share ideas. It's a part of the CVS repository but it isn't distributed as part of Docutils releases. Feel free to check in code to the CVS sandbox; that way people can try it out but you won't have to worry about it working 100% error-free, as is the goal of the main source tree. Each developer who wants to play in the sandbox should create either a project-specific subdirectory or personal subdirectory (suggested name: SourceForge ID, nickname, or given name + family initial). It's OK to make a mess! But please, play nice.

Please update the sandbox README file with links and a brief description of your work.

In order to minimize the work necessary for others to install and try out new, experimental components, the following sandbox directory structure is recommended:

sandbox/
    project_name/ # For a project where you invite contributions.
                  # Structure as in userid/component_name below.
    userid/       # For personal space.
        component_name/ # A verbose name is best.
            README.txt  # Please explain requirements,
                        # purpose/goals, and usage.
            docs/
                ...
            component.py    # The component is a single module.
                        # *OR* (but *not* both)
            component/  # The component is a package.
                __init__.py  # Contains the Reader/Writer class.
                other1.py    # Other modules and data files used
                data.txt     # by this component.
                ...
            test/       # Test suite.
                ...
            tools/      # For front ends etc.
                ...
            setup.py    # Use Distutils to install the component
                        # code and tools/ files into the right
                        # places in Docutils.

Some sandbox projects are destined to become Docutils components once completed. Others, such as add-ons to Docutils or applications of Docutils, graduate to become parallel projects.

Parallel Projects

Parallel projects contain useful code that is not central to the functioning of Docutils. Examples are specialized add-ons or plug-ins, and applications of Docutils. They use Docutils, but Docutils does not require their presence to function.

An official parallel project will have its own CVS directory beside (or parallel to) the main Docutils CVS directory. It can have its own web page in the docutils.sourceforge.net domain, its own file releases and downloadable CVS snapshots, and even a mailing list if that proves useful. However, an official parallel project has implications: it is expected to be maintained and continue to work with changes to the core Docutils.

A parallel project requires a project leader, who must commit to coordinate and maintain the implementation:

  • Answer questions from users and developers.
  • Review suggestions, bug reports, and patches.
  • Monitor changes and ensure the quality of the code and documentation.
  • Coordinate with Docutils to ensure interoperability.
  • Put together official project releases.

Of course, related projects may be created independently of Docutils. The advantage of a parallel project is that the SourceForge environment and the developer and user communities are already established. Core Docutils developers are available for consultation and may contribute to the parallel project. It's easier to keep the projects in sync when there are changes made to the core Docutils code.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Python Source Reader

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.7
Date: | 2004-06-05
Copyright: | This document has been placed in the public domain.

This document explores issues around extracting and processing docstrings from Python modules.

For definitive element hierarchy details, see the "Python Plaintext Document Interface DTD" XML document type definition, pysource.dtd (which modifies the generic docutils.dtd). Descriptions below list 'DTD elements' (XML 'generic identifiers' or tag names) corresponding to syntax constructs.

Contents

  • Model
  • Docstring Extractor
  • Interpreted Text

Model

The Python Source Reader ("PySource") model that's evolving in my mind goes something like this:

  1. Extract the docstring/namespace 1 tree from the module(s) and/or package(s).

    [1]

    See Docstring Extractor below.

  2. Run the parser on each docstring in turn, producing a forest of doctrees (per nodes.py).

  3. Join the docstring trees together into a single tree, running transforms:

    • merge hyperlinks
    • merge namespaces
    • create various sections like "Module Attributes", "Functions", "Classes", "Class Attributes", etc.; see pysource.dtd
    • convert the above special sections to ordinary doctree nodes

  4. Run transforms on the combined doctree. Examples: resolving cross-references/hyperlinks (including interpreted text on Python identifiers); footnote auto-numbering; first field list -> bibliographic elements.

    (Or should step 4's transforms come before step 3?)

  5. Pass the resulting unified tree to the writer/builder.

I've had trouble reconciling the roles of input parser and output writer with the idea of modes ("readers" or "directors"). Does the mode govern the tranformation of the input, the output, or both? Perhaps the mode should be split into two.

For example, say the source of our input is a Python module. Our "input mode" should be the "Python Source Reader". It discovers (from __docformat__) that the input parser is "reStructuredText". If we want HTML, we'll specify the "HTML" output formatter. But there's a piece missing. What kind or style of HTML output do we want? PyDoc-style, LibRefMan style, etc. (many people will want to specify and control their own style). Is the output style specific to a particular output format (XML, HTML, etc.)? Is the style specific to the input mode? Or can/should they be independent?

I envision interaction between the input parser, an "input mode" , and the output formatter. The same intermediate data format would be used between each of these, being transformed as it progresses.

Docstring Extractor

We need code that scans a parsed Python module, and returns an ordered tree containing the names, docstrings (including attribute and additional docstrings), and additional info (in parentheses below) of all of the following objects:

  • packages
  • modules
  • module attributes (+ values)
  • classes (+ inheritance)
  • class attributes (+ values)
  • instance attributes (+ values)
  • methods (+ formal parameters & defaults)
  • functions (+ formal parameters & defaults)

(Extract comments too? For example, comments at the start of a module would be a good place for bibliographic field lists.)

In order to evaluate interpreted text cross-references, namespaces for each of the above will also be required.

See python-dev/docstring-develop thread "AST mining", started on 2001-08-14.

Interpreted Text

DTD elements: package, module, class, method, function, module_attribute, class_attribute, instance_attribute, variable, parameter, type, exception_class, warning_class.

To classify identifiers explicitly, the role is given along with the identifier in either prefix or suffix form:

Use :method:`Keeper.storedata` to store the object's data in
`Keeper.data`:instance_attribute:.

The role may be one of 'package', 'module', 'class', 'method', 'function', 'module_attribute', 'class_attribute', 'instance_attribute', 'variable', 'parameter', 'type', 'exception_class', 'exception', 'warning_class', or 'warning'. Other roles may be defined.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils Release Procedure

Author: | David Goodger; open to all Docutils developers
Contact: | goodger@python.org
Date: | 2004-06-03
Revision: | 1.1
Copyright: | This document has been placed in the public domain.

  1. Edit the version number in the following files:

    • docutils:
      • setup.py
      • HISTORY.txt
      • docutils/__init__.py
    • web: index.txt

  2. Run the test suite: cd test ; alltests.py.

  3. Isolate from outside influence:

    1. Remove the old installation from site-packages (including roman.py, and optparse.py, textwrap.py for pre-2.3 installations).
    2. Clear/unset the PYTHONPATH environment variable.

  4. Create the release tarball:

    1. Create a new empty directory and cd into it.

    2. Get a clean snapshot of the CVS files:

      cvs -z3 -d:pserver:anonymous@cvs.sf.net:/cvsroot/docutils \
    export -rHEAD docutils

    3. Use Distutils to create the release tarball:

      cd docutils
python setup.py sdist

  5. Expand and install the release tarball in isolation (as per step 3 above):

    1. Expand the tarball in a new location, not over any existing files.

    2. Install from expanded directory:

      cd docutils-X.Y
python setup.py install

      The "install" command may require root permissions.

  6. Run the test suite from the expanded archive directory: cd test ; alltests.py.

  7. Run cd tools ; buildhtml.py .. to confirm that there are no unexpected issues with the docs.

  8. Upload the release tarball:

    $ ftp upload.sourceforge.net
Connected to osdn.dl.sourceforge.net.
...
Name (upload.sourceforge.net:david): anonymous
331 Anonymous login ok, send your complete e-mail address as password.
Password:
...
230 Anonymous access granted, restrictions apply.
ftp> bin
200 Type set to I.
ftp> cd /incoming
250 CWD command successful.
ftp> put filename

  9. Log in to the SourceForge web interface.

  10. Access the file release system on SourceForge (Admin interface). Fill in the fields:

    Package ID:

    docutils
    Release Name:

    <use release number only, e.g. 0.3>
    Release Date:

    <today's date>
    Status:

    Active
    File Name:

    <select the file just uploaded>
    File Type:

    Source .gz
    Processor Type:

    Platform-Independent
    Release Notes:

    <insert README.txt file here>
    Change Log:

    <insert summary from announcement>

    Also check the "Preserve my pre-formatted text" box.

  11. Wait up to 30 minutes for the file to become available on SourceForge.

  12. Download the release tarball and verify its integrity by walking through an installation, as outlined above (steps 5, 6, & 7).

  13. Add a SourceForge News item, with title "Docutils 0.x released" and containing the release tarball's download URL.

  14. Send announcement email to:

    • docutils-develop@lists.sourceforge.net
    • docutils-users@lists.sourceforge.net
    • doc-sig@python.org
    • python-list@python.org
    • python-announce@python.org

  15. Register

    1. with PyPI (Fill in details. python setup.py register? How to log in?)
    2. with Vaults of Parnassus
    3. with FreshMeat?
View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docstring Semantics

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.4
Date: | 2003-06-09
Copyright: | This document has been placed in the public domain.

These are notes for a possible future PEP providing the final piece of the Python docstring puzzle: docstring semantics or documentation methodology. PEP 257, Docstring Conventions, sketches out some guidelines, but does not get into methodology details.

I haven't explored documentation methodology more because, in my opinion, it is a completely separate issue from syntax, and it's even more controversial than syntax. Nobody wants to be told how to lay out their documentation, a la JavaDoc. I think the JavaDoc way is butt-ugly, but it is an established standard for the Java world. Any standard documentation methodology has to be formal enough to be useful but remain light enough to be usable. If the methodology is too strict, too heavy, or too ugly, many/most will not want to use it.

I think a standard methodology could benefit the Python community, but it would be a hard sell. A PEP would be the place to start. For most human-readable documentation needs, the free-form text approach is adequate. We'd only need a formal methodology if we want to extract the parameters into a data dictionary, index, or summary of some kind.

PythonDoc

(Not to be confused with Daniel Larsson's pythondoc project.)

A Python version of the JavaDoc semantics (not syntax). A set of conventions which are understood by the Docutils. What JavaDoc has done is to establish a syntax that enables a certain documentation methodology, or standard semantics. JavaDoc is not just syntax; it prescribes a methodology.

  • Use field lists or definition lists for "tagged blocks". By this I mean that field lists can be used similarly to JavaDoc's @tag syntax. That's actually one of the motivators behind field lists. For example, we could have:

    """
:Parameters:
    - `lines`: a list of one-line strings without newlines.
    - `until_blank`: Stop collecting at the first blank line if
      true (1).
    - `strip_indent`: Strip common leading indent if true (1,
      default).

:Return:
    - a list of indented lines with mininum indent removed;
    - the amount of the indent;
    - whether or not the block finished with a blank line or at
      the end of `lines`.
"""

    This is taken straight out of docutils/statemachine.py, in which I experimented with a simple documentation methodology. Another variation I've thought of exploits the Grouch-compatible "classifier" element of definition lists. For example:

    :Parameters:
    `lines` : [string]
        List of one-line strings without newlines.
    `until_blank` : boolean
        Stop collecting at the first blank line if true (1).
    `strip_indent` : boolean
        Strip common leading indent if true (1, default).

  • Field lists could even be used in a one-to-one correspondence with JavaDoc @tags, although I doubt if I'd recommend it. Several ports of JavaDoc's @tag methodology exist in Python, most recently Ed Loper's "epydoc".

Other Ideas

  • Can we extract comments from parsed modules? Could be handy for documenting function/method parameters:

    def method(self,
           source,        # path of input file
           dest           # path of output file
          ):

    This would save having to repeat parameter names in the docstring.

    Idea from Mark Hammond's 1998-06-23 Doc-SIG post, "Re: [Doc-SIG] Documentation tool":

    it would be quite hard to add a new param to this method without realising you should document it

  • Frederic Giacometti's iPhrase Python documentation conventions is an attachment to his Doc-SIG post of 2001-05-30.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils Testing

Author: | Felix Wiemann
Revision: | 1.2
Date: | 2004-06-27
Copyright: | This document has been placed in the public domain.

Contents

  • Unit Tests
    • Setting Up For Testing
  • Functional Tests
    • Directory Structure
    • The Testing Process
    • Creating New Tests
    • The Default Configuration File

This document describes how the tests are organized and how to add new tests or modify existing tests.

Unit Tests

Unit tests test single functions or modules (i.e. whitebox testing).

XXX Some details to be written.

Setting Up For Testing

Recent versions of Python's unittest.py module have an annoying new "feature", where newlines in test failure output are displayed as "n" (two characters) instead of as actual newlines. This means that test failure output is unreadable and unuseable. Earlier versions didn't have this feature, and the output could easily be copied and pasted into test data modules. Revision 1.7.2.1 of unittest.py works well. Download it, place it in the test/ directory, and enjoy much better output.

Functional Tests

The directory test/functional/ contains data for functional tests.

Performing functional testing means testing the Docutils system as a whole (i.e. blackbox testing).

Directory Structure

  • functional/ The main data directory.
    • input/ The input files.
      • some_test.txt, for example.
    • output/ The actual output.
      • some_test.html, for example.
    • expected/ The expected output.
      • some_test.html, for example.
    • tests/ The config files for processing the input files.
      • some_test.py, for example.
      • _default.py, the default configuration file.

The Testing Process

When running test_functional.py, all config files in functional/some_test.py are processed.

An example of what this means:

Provided functional/tests/some_test.py reads like this:

# Source and destination file names.
test_source = "some_test.txt"
test_destination = "some_test.html"

# Keyword parameters passed to publish_file.
reader_name = "standalone"
parser_name = "rst"
writer_name = "html"
settings_overrides['output-encoding'] = 'utf-8'

The two variables test_source and test_destination contain the input file name (relative to functional/input/) and the output file name (relative to functional/output/ and functional/expected/). Note that the file names can be chosen arbitrarily. However, the file names in functional/output/ must match the file names in functional/expected/.

All other variables are passed as keyword arguments to docutils.core.publish_file, so you can set reader, parser, writer and anything else you want to configure.

Note that settings_overrides is already initialized as an empty dictionary before the execution of the config file. This is done in order to allow subsequent assignments to settings_overrides in the default configuration file and in the actual configuration file.

Creating New Tests

In order to create a new test, put the input test file into functional/input/. Then create a config file in functional/tests/ which sets at least input and output file names, reader, parser and writer.

Now run test_functional.py. The test will fail, of course, because you do not have an expected output yet.

However, an output file will be generated in functional/output/. Check this output file for validity and correctness. Then copy the file to functional/expected/.

If you run test_functional.py later and the actual output doesn't match the expected output anymore, the test will fail.

If this is the case and you made an intentional change, check the actual output for validity and correctness and copy it to functional/expected/, overwriting the old expected output.

The Default Configuration File

The file functional/tests/_default.py contains default settings. It is executed just before the actual configuration files, which has the same effect as if the contents of _default.py were prepended to every configuration file.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils To Do List

Author: | David Goodger (with input from many); open to all Docutils developers
Contact: | goodger@python.org
Date: | 2004-06-28
Revision: | 1.245
Copyright: | This document has been placed in the public domain.

Contents

  • Priorities
  • General
  • Documentation
    • User Docs
    • Developer Docs
    • How-Tos
    • PEPs
  • Python Source Reader
  • reStructuredText Parser
    • Directives
    • Interpreted Text
  • Unimplemented Transforms
  • HTML Writer
  • Front-End Tools

Priority items are marked with "@" symbols. The more @s, the higher the priority. Items in question form (containing "?") are ideas which require more thought and debate; they are potential to-do's.

Many of these items are awaiting champions. If you see something you'd like to tackle, please do! If there's something you'd like to see done but are unable to implement it yourself, please consider contributing in other ways.

Priorities

  • Include substitution files for character entities, produced by the tools/unicode2rstsubs.py. As static data, these files could go inside the docutils package somewhere.
  • Rename front-end tools (perhaps to "rst2*" pattern) and have setup.py install them.
  • A Python Source Reader component (Python auto-documentation) will be added. See the document "Plan for Enthought API Documentation Tool" for details. If you'd like to help, let me know!

General

  • Refactor

    • Rename methods & variables according to the Python coding conventions.
    • The name-to-id conversion and hyperlink resolution code needs to be checked for correctness and refactored. I'm afraid it's a bit of a spaghetti mess now.

  • Add validation? See http://pytrex.sourceforge.net, RELAX NG, pyRXP.

  • Ask Python-dev for opinions (GvR for a pronouncement) on special variables (__author__, __version__, etc.): convenience vs. namespace pollution. Ask opinions on whether or not Docutils should recognize & use them.

  • In docutils.readers.get_reader_class (& parsers & writers too), should we be importing "standalone" or "docutils.readers.standalone"? (This would avoid importing top-level modules if the module name is not in docutils/readers. Potential nastiness.)

  • @@@ Perhaps store a name-to-id mapping file? This could be stored permanently, read by subsequent processing runs, and updated with new entries. ("Persistent ID mapping"?)

  • Need a Unicode to HTML entities codec for HTML writer? No, the "xmlcharrefreplace" output error handler is sufficient. Make it the default for HTML & XML writers?

  • Perhaps the Component.supports method should deal with individual features ("meta" etc.) instead of formats ("html" etc.)?

  • Add /usr/etc/docutils.conf to config file list? System-wide, whereas /etc/docutils.conf is machine-specific. /usr/local/etc/docutils.conf too? See the Filesystem Hierarchy Standard.

  • Add object numbering and object references (tables & figures). These would be the equivalent of DocBook's "formal" elements.

    We may need persistent sequences, such as chapter numbers. See OpenOffice.org XML "fields". Should the sequences be automatic or manual (user-specifyable)?

    We need to name the objects:

    • "name" option for the "figure" directive?

      .. figure:: image.png
   :name: image's name

      Same for the "table" directive:

      .. table:: optional title here
   :name: table's name

   =====  =====
     x    not x
   =====  =====
   True   False
   False  True
   =====  =====

      This would also allow other options to be set, like border styles. The same technique could be used for other objects.

      A preliminary "table" directive has been implemented, supporting table titles. Perhaps the name should derive from the title.

    • The object could also be done this way:

      .. _figure name:

.. figure:: image.png

      This may be a more general solution, equally applicable to tables. However, explicit naming using an option seems simpler to users.

    We'll also need syntax for object references. See OpenOffice.org XML "reference fields":

    • Parameterized substitutions? For example:

      See |figure (figure name)| on |page (figure name)|.

.. |figure (name)| figure-ref:: (name)
.. |page (name)| page-ref:: (name)

      The result would be:

      See figure 3.11 on page 157.

      But this would require substitution directives to be processed at reference-time, not at definition-time as they are now. Or, perhaps the directives could just leave pending elements behind, and the transforms do the work? How to pass the data through? Too complicated.

    • An interpreted text approach is simpler and better:

      See :figure:`figure name` on :page:`figure name`.

      The "figure" and "page" roles could generate appropriate boilerplate text. The position of the role (prefix or suffix) could also be utilized.

      See Interpreted Text below.

  • Think about large documents made up of multiple subdocument files. Issues: continuity (persistent sequences above), cross-references (name-to-id mapping file above and targets in other documents below).

    When writing a book, the author probably wants to split it up into files, perhaps one per chapter (but perhaps even more detailed). However, we'd like to be able to have references from one chapter to another, and have continuous numbering (pages and chapters, as applicable). Of course, none of this is implemented yet. There has been some thought put into some aspects; see the "include" directive and the Reference Merging transform below.

    When I was working with SGML in Japan, we had a system where there was a top-level coordinating file, book.sgml, which contained the top-level structure of a book: the <book> element, containing the book <title> and empty component elements (<preface>, <chapter>, <appendix>, etc.), each with filename attributes pointing to the actual source for the component. Something like this:

    <book id="bk01">
<title>Title of the Book</title>
<preface inrefid="pr01"></preface>
<chapter inrefid="ch01"></chapter>
<chapter inrefid="ch02"></chapter>
<chapter inrefid="ch03"></chapter>
<appendix inrefid="ap01"></appendix>
</book>

    (The "inrefid" attribute stood for "insertion reference ID".)

    The processing system would process each component separately, but it would recognize and use the book file to coordinate chapter and page numbering, and keep a persistent ID to (title, page number) mapping database for cross-references. Docutils could use a similar system for large-scale, multipart documents.

    Aahz's idea:

    First the ToC:

    .. ToC-list::
    Introduction.txt
    Objects.txt
    Data.txt
    Control.txt

    Then a sample use:

    .. include:: ToC.txt

As I said earlier in chapter :chapter:`Objects.txt`, the
reference count gets increased every time a binding is made.

    Which produces:

    As I said earlier in chapter 2, the
reference count gets increased every time a binding is made.

    The ToC in this form doesn't even need to be references to actual reST documents; I'm simply doing it that way for a minimum of future-proofing, in case I do want to add the ability to pick up references within external chapters.

    Perhaps, instead of ToC (which would overload the "contents" directive concept already in use), we could use "manifest". A "manifest" directive might associate local reference names with files:

    .. manifest::
   intro: Introduction.txt
   objects: Objects.txt
   data: Data.txt
   control: Control.txt

    Then the sample becomes:

    .. include:: manifest.txt

As I said earlier in chapter :chapter:`objects`, the
reference count gets increased every time a binding is made.

  • Add testing for Docutils' front end tools?

  • Changes to sandbox/davidg/infrastructure/docutils-update?

    • Modify the script to only update the snapshots if files have actually changed in CVS (saving some SourceForge server cycles).
    • Make passing the test suite a prerequisite to snapshot update, but only if the process is completely automatic.
    • Rewrite in Python?

  • Publisher: "Ordinary setup" shouldn't requre specific ordering; at the very least, there ought to be error checking higher up in the call chain. [Aahz]

    Publisher.get_settings requires that all components be set up before it's called. Perhaps the I/O objects shouldn't be set, but I/O classes. Then options are set up (.set_options), and Publisher.set_io (or equivalent code) is called with source & destination paths, creating the I/O objects.

    Perhaps I/O objects shouldn't be instantiated until required. For split output, the Writer may be called multiple times, once for each doctree, and each doctree should have a separate Output object (with a different path). Is the "Builder" pattern applicable here?

  • Perhaps I/O objects should become full-fledged components (i.e. subclasses of docutils.Component, as are Readers, Parsers, and Writers now), and thus have associated option/setting specs and transforms.

  • Multiple file I/O suggestion from Michael Hudson: use a file-like object or something you can iterate over to get file-like objects.

  • Language modules: in accented languages it may be useful to have both accented and unaccented entries in the bibliographic_fields mapping for versatility.

  • Add a "--strict-language" option & setting: no English fallback for language-dependent features.

  • Add an "--input-language" option & setting? Specify a different language module for input (bibliographic fields, directives) than for output. The "--language" option would set both input & output languages.

  • Auto-generate reference tables for language-dependent features? Could be generated from the source modules. A special command-line option could be added to Docutils front ends to do this. (Idea from Engelbert Gruber.)

  • Change the "class" attribute of elements (set with Element.set_class) to a list?

  • Enable feedback of some kind from internal decisions, such as reporting the successful input encoding. Modify runtime settings? System message? Simple stderr output?

  • Rationalize Writer settings (HTML/LaTeX/PEP) -- share settings.

  • The "docutils.conf" included with Docutils should become complete, with examples of every setting (many/most commented out). It's currently sparse, requiring doc lookups.

  • Merge docs/user/latex.txt info into tools.txt and config.txt.

  • Add internationalization to footer boilerplate text (resulting from "--generator", "--source-link", and "--date" etc.), allowing translations.

  • Add an "--include file" command-line option (config setting too?), equivalent to ".. include:: file" as the first line of the doc text? Especially useful for character entity sets, text transform specs, boilerplate, etc.

  • Parameterize the Reporter object or class? See the 2004-02-18 "rest checking and source path" thread.

  • Add a "disable_transforms" setting? And a dummy Writer subclass that does nothing when its .write() method is called? Would allow for easy syntax checking. See the 2004-02-18 "rest checking and source path" thread.

  • Add a generic meta-stylesheet mechanism? An external file could associate style names ("class" attributes) with specific elements. Could be generalized to arbitrary output attributes; useful for HTML & XMLs. Aahz implemented something like this in sandbox/aahz/Effective/EffMap.py.

  • William Dode suggested that table cells be assigned "class" attributes by columns, so that stylesheets can affect text alignment. Unfortunately, there doesn't seem to be a way (in HTML at least) to leverage the "colspec" elements (HTML "col" tags) by adding classes to them. The resulting HTML is very verbose:

    <td class="col1">111</td>
<td class="col2">222</td>
...

    At the very least, it should be an option. People who don't use it shouldn't be penalized by increases in their HTML file sizes.

    Table rows could also be assigned classes (like odd/even). That would be easier to implement.

    How should it be implemented?

    • There could be writer options (column classes & row classes) with standard values.
    • The table directive could grow some options. Something like ":cell-classes: col1 col2 col3" (either must match the number of columns, or repeat to fill?) and ":row-classes: odd even" (repeat to fill; body rows only, or header rows too?).

    Probably per-table directive options are best. The "class" values could be used by any writer, and applying such classes to all tables in a document with writer options is too broad.

  • Make the csv-table directive work with Python 2.1/2.2. (See the discussion on the mailing list.)

Documentation

User Docs

What else is needed?

Developer Docs

  • Complete Docutils Runtime Settings.
  • @@@ Improve the internal module documentation (docstrings in the code). Specific deficiencies listed below.
    • docutils.parsers.rst.states.State.build_table: data structure required (including StringList).
    • docutils.parsers.rst.states: more complete documentation of parser internals.
  • docs/ref/doctree.txt: DTD element structural relationships, semantics, and attributes. In progress; element descriptions to be completed.
  • Document the pending elements, how they're generated and what they do.
  • Document the transforms (perhaps in docstrings?): how they're used, what they do, dependencies & order considerations.
  • Document the HTML classes used by html4css1.py.
  • Write an overview of the Docutils architecture, as an introduction for developers. What connects to what, why, and how. Either update PEP 258 (see PEPs below) or as a separate doc.
  • Give information about unit tests. Maybe as a howto?

How-Tos

  • Creating Docutils Writers
  • Creating Docutils Readers
  • Creating Docutils Transforms
  • Creating Docutils Parsers
  • Using Docutils as a Library

PEPs

  • Complete PEP 258 Docutils Design Specification.

    • Fill in the blanks in API details.

    • Specify the nodes.py internal data structure implementation?

      [Tibs:] Eventually we need to have direct documentation in there on how it all hangs together - the DTD is not enough (indeed, is it still meant to be correct? [Yes, it is. --DG]).

  • Rework PEP 257, separating style from spec from tools, wrt Docutils? See Doc-SIG from 2001-06-19/20.

Python Source Reader

General:

  • Analyze Tony Ibbs' PySource code.
  • Analyze Doug Hellmann's HappyDoc project.
  • Investigate how POD handles literate programming.
  • Take the best ideas and integrate them into Docutils.

Miscellaneous ideas:

  • If we can detect that a comment block begins with ##, a la JavaDoc, it might be useful to indicate interspersed section headers & explanatory text in a module. For example:

    """Module docstring."""

##
# Constants
# =========

a = 1
b = 2

##
# Exception Classes
# =================

class MyException(Exception): pass

# etc.

  • Should standalone strings also become (module/class) docstrings? Under what conditions? We want to prevent arbitrary strings from becomming docstrings of prior attribute assignments etc. Assume that there must be no blank lines between attributes and attribute docstrings? (Use lineno of NEWLINE token.)

    Triple-quotes are sometimes used for multi-line comments (such as commenting out blocks of code). How to reconcile?

  • HappyDoc's idea of using comment blocks when there's no docstring may be useful to get around the conflict between additional docstrings and from __future__ import for module docstrings. A module could begin like this:

    #!/usr/bin/env python
# :Author: Me
# :Copyright: whatever

"""This is the public module docstring (``__doc__``)."""

# More docs, in comments.
# All comments at the beginning of a module could be
# accumulated as docstrings.
# We can't have another docstring here, because of the
# ``__future__`` statement.

from __future__ import division

    Using the JavaDoc convention of a doc-comment block beginning with ## is useful though. It allows doc-comments and implementation comments.

  • HappyDoc uses an initial comment block to set "parser configuration values". Do the same thing for Docutils, to set runtime settings on a per-module basis? I.e.:

    # Docutils:setting=value

    Could be used to turn on/off function parameter comment recognition & other marginal features. Could be used as a general mechanism to augment config files and command-line options (but which takes precedence?).

  • Multi-file output should be divisible at arbitrary level.

  • Support all forms of import statements:

    • import module: listed as "module"
    • import module as alias: "alias (module)"
    • from module import identifier: "identifier (from module)"
    • from module import identifier as alias: "alias (identifier from module)"
    • from module import *: "all identifiers (*) from module"

  • Have links to colorized Python source files from API docs? And vice-versa: backlinks from the colorized source files to the API docs!

  • In summaries, use the first sentence of a docstring if the first line is not followed by a blank line.

reStructuredText Parser

Also see the ... Or Not To Do? list.

  • Clean up the code; refactor as required.

  • Add motivation sections for constructs in spec.

  • Document title should grow an implicit target.

  • Allow very long titles (on two or more lines)?

  • And for the sake of completeness, should definition list terms be allowed to be very long (two or more lines) also?

  • Support generic hyperlink references to targets in other documents? Not in an HTML-centric way, though (it's trivial to say http://www.example.com/doc#name, and useless in non-HTML contexts). XLink/XPointer? .. baseref::? See Doc-SIG 2001-08-10.

  • In target URLs, it would be useful to not explicitly specify the file extension. If we're generating HTML, then ".html" is appropriate; if PDF, then ".pdf"; etc. How about using ".*" to indicate "choose the most appropriate filename extension? For example:

    .. _Another Document: another.*

    Should the choice be from among existing files only? Documents only, or objects (images, etc.) also? (How to differentiate? Element context [within "image"]?)

    This may not be just a parser issue though; it may need framework support.

    Mailing list thread: Images in both HTML and LaTeX

  • Implement the header row separator modification to table.el. (Wrote to Takaaki Ota & the table.el mailing list on 2001-08-12, suggesting support for "=====" header rows. On 2001-08-17 he replied, saying he'd put it on his to-do list, but "don't hold your breath".)

  • Tony says inline markup rule 7 could do with a little more exposition in the spec, to make clear what is going on for people with head colds.

  • @@ Fix the parser's indentation handling to conform with the stricter definition in the spec. (Explicit markup blocks should be strict or forgiving?)

  • @@ Tighten up the spec for indentation of "constructs using complex markers": field lists and option lists? Bodies may begin on the same line as the marker or on a subsequent line (with blank lines optional). Require that for bodies beginning on the same line as the marker, all lines be in strict alignment. Currently, this is acceptable:

    :Field-name-of-medium-length: Field body beginning on the same
    line as the field name.

    This proposal would make the above example illegal, instead requiring strict alignment. A field body may either begin on the same line:

    :Field-name-of-medium-length: Field body beginning on the same
                              line as the field name.

    Or it may begin on a subsequent line:

    :Field-name-of-medium-length:
    Field body beginning on a line subsequent to that of the
    field name.

    This would be especially relevant in degenerate cases like this:

    :Number-of-African-swallows-requried-to-carry-a-coconut:
    It would be very difficult to align the field body with
    the left edge of the first line if it began on the same
    line as the field name.

  • Allow for variant styles by interpreting indented lists as if they weren't indented? For example, currently the list below will be parsed as a list within a block quote:

    paragraph

  * list item 1
  * list item 2

    But a lot of people seem to write that way, and HTML browsers make it look as if that's the way it should be. The parser could check the contents of block quotes, and if they contain only a single list, remove the block quote wrapper. There would be two problems:

    1. What if we actually do want a list inside a block quote?
    2. What if such a list comes immediately after an indented construct, such as a literal block?

    Both could be solved using empty comments (problem 2 already exists for a block quote after a literal block). But that's a hack.

    Perhaps a runtime setting, allowing or disabling this convenience, would be appropriate. But that raises issues too:

    User A, who writes lists indented (and their config file is set up to allow it), sends a file to user B, who doesn't (and their config file disables indented lists). The result of processing by the two users will be different.

    It may seem minor, but it adds ambiguity to the parser, which is bad.

    See the Doc-SIG discussion starting 2001-04-18 with Ed Loper's "Structuring: a summary; and an attempt at EBNF", item 4 (and follow-ups, here and here). Also docutils-users, 2003-02-17 and beginning 2003-08-04.

  • Make the parser modular. Allow syntax constructs to be added or disabled at run-time. Or is subclassing enough?

  • Continue to report (info, level 1) enumerated lists whose start value is not ordinal-1?

  • Generalize the "doctest block" construct (which is overly Python-centric) to other interactive sessions? "Doctest block" could be renamed to "I/O block" or "interactive block", and each of these could also be recognized as such by the parser:

    • Shell sessions:

      $ cat example1.txt
A block beginning with a "$ " prompt is interpreted as a shell
session interactive block.  As with Doctest blocks, the
interactive block ends with the first blank line, and wouldn't
have to be indented.

    • Root shell sessions:

      # cat example2.txt
A block beginning with a "# " prompt is interpreted as a root
shell session (the user is or has to be logged in as root)
interactive block.  Again, the block ends with a blank line.

    Other standard (and unambiguous) interactive session prompts could easily be added (such as "> " for WinDOS).

    Tony Ibbs spoke out against this idea (2002-06-14 Doc-SIG thread "docutils feedback").

  • Should the "doctest" element go away, and the construct simply be a front-end to generic literal blocks?

  • Add support for pragma (syntax-altering) directives.

    Some pragma directives could be local-scope unless explicitly specified as global/pragma using ":global:" options.

  • Remove leading numbers from section titles for implicit link names? A section titled "3. Conclusion" could then be referred to by "Conclusion_" (i.e., without the "3.").

  • Syntax for line blocks (currently a directive)?

  • Implement auto-enumerated lists? See Auto-Enumerated Lists.

  • Support whitespace in angle-bracketed standalone URLs according to Appendix E ("Recommendations for Delimiting URI in Context") of RFC 2396.

  • Use the vertical spacing of the source text to determine the corresponding vertical spacing of the output?

  • [From Mark Nodine] For cells in simple tables that comprise a single line, the justification can be inferred according to the following rules:

    1. If the text begins at the leftmost column of the cell, then left justification, ELSE
    2. If the text begins at the rightmost column of the cell, then right justification, ELSE
    3. Center justification.

    The onus is on the author to make the text unambiguous by adding blank columns as necessary. There should be a parser setting to turn off justification-recognition (normally on would be fine).

    Decimal justification?

  • Make enumerated list parsing more strict, so that this would parse as a paragraph with an info message:

    1. line one
3. line two

  • Generalize the "target-notes" directive into a command-line option somehow? See docutils-develop 2003-02-13.

  • Include the character entity substitution definition files temporarily stored here, perhaps in a docutils/parsers/rst/includes/ directory. See misc.include below.

  • Should ^L (or something else in reST) be defined to mean force/suggest page breaks in whatever output we have?

    A "break" or "page-break" directive would be easy to add. A new doctree element would be required though (perhaps "break"). The final behavior would be up to the Writer. The directive argument could be one of page/column/recto/verso for added flexibility.

    Currently ^L (Python's "f") characters are treated as whitespace. They're converted to single spaces, actually, as are vertical tabs (^K, Python's "v"). It would be possible to recognize form feeds as markup, but it requires some thought and discussion first. Are there any downsides? Many editing environments do not allow the insertion of control characters. Will it cause any harm? It would be useful as a shorthand for the directive.

    It's common practice to use ^L before Emacs "Local Variables" lists:

    ^L
..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:

    These are already present in many PEPs and Docutils project documents. From the Emacs manual (info):

    A "local variables list" goes near the end of the file, in the last page. (It is often best to put it on a page by itself.)

    It would be unfortunate if this construct caused a final blank page to be generated (for those Writers that recognize the page breaks). We'll have to add a transform that looks for a "break" plus zero or more comments at the end of a document, and removes them.

  • Could the "break" concept above be extended to inline forms? E.g. "^L" in the middle of a sentence could cause a line break. Only recognize it at the end of a line (i.e., "fn")?

    Or is formfeed inappropriate? Perhaps vertical tab ("v"), but even that's a stretch. Can't use carriage returns, since they're commonly used for line endings.

  • Allow a "::"-only paragraph (first line, actually) to introduce a literal block without a blank line? (Idea from Paul Moore.)

    ::
    This is a literal block

    Is indentation enough to make the separation between a paragraph which contains just a :: and the literal text unambiguous? There's one problem with this concession. What if one wants a definition list item which defines the term "::"? We'd have to escape it. Currenty, "::" doesn't work (although it should; bug), and "::" is misinterpreted as a field name (name ""; also a bug). Assuming these bugs are squashed, I suppose it's a useful special case. It would only be reasonable to apply it to "::"-only paragraphs though. I think the blank line is visually necessary if there's text before the "::":

    The text in this paragraph needs separation
from the literal block following::
    This doesn't look right.

    Another idea. Would it be worthwhile to allow literal blocks to begin without a newline after the "::"? Example:

    ::  while True:
        print 'hello world'

    Perhaps. Perhaps not.

  • Add new syntax for nested inline markup? Or extend the parser to parse nested inline markup somehow? See the collected notes.

  • Drop the backticks from embedded URIs with omitted reference text? Should the angle brackets be kept in the output or not?

    <file_name>_

    Probably not worth the trouble.

  • Add ^superscript^ inline markup? The only common non-markup uses of "^" I can think of are as short hand for "superscript" itself and for describing control characters ("^C to cancel"). The former supports the proposed syntax, and it could be argued that the latter ought to be literal text anyhow (e.g. "^C to cancel").

  • Add math markup. We should try for a general solution, that's applicable to any output format. Using a standard, such as MathML, would be best. TeX (or itex) would be acceptable as a front-end to MathML. See the culmination of a relevant discussion.

    Both a directive and an interpreted text role will be necessary (for each markup). Directive example:

    .. itex::
   \alpha_t(i) = P(O_1, O_2, \dots O_t, q_t = S_i \lambda)

    The same thing inline:

    The equation in question is :itex:`\alpha_t(i) = P(O_1, O_2,
\dots O_t, q_t = S_i \lambda)`.

  • How about a syntax for alternative hyperlink behavior, such as "open in a new window" (as in HTML's <a target="_blank">)? Double angle brackets might work for inline targets:

    The `reference docs <<url>>`__ may be handy.

    But what about explicit targets?

    The MoinMoin wiki uses a caret ("^") at the beginning of the URL ("^" is not a legal URI character). That could work for both inline and explicit targets:

    The `reference docs <^url>`__ may be handy.

.. _name: ^url

  • Allow transitions between sections? A transition immediately preceding a section title could be interpreted as a sibling of that section. The context would obviate the need for a special directive. The document model would have to accommodate the change:

    <!ENTITY % structure.model
  " ( ( (%body.elements; | topic | sidebar)+,
        (transition, (%body.elements; | topic | sidebar)+ )*,
        (transition?, (%section.elements;))* )
    | (transition?, (%section.elements;))+ ) ">

  • Add an option to allow arbitrary URI schemes (not just those in urischemes.py)? This would make text like "signal:noise" into a URI.

  • Add an option to add URI schemes at runtime.

  • Segmented lists:

    : segment : segment : segment
: segment : segment : very long
  segment
: segment : segment : segment

    The initial colon (":") can be thought of as a type of bullet

    We could even have segment titles:

    :: title  : title   : title
: segment : segment : segment
: segment : segment : segment

    This would correspond well to DocBook's SegmentedList. Output could be tabular or "name: value" pairs, as described in DocBook's docs.

  • Allow backslash-escaped colons in field names:

    :Case Study\: Event Handling: This chapter will be dropped.

  • footnote spaces:

    When using superscripted footnote references, the space looks ugly. We could automatically activate --trim-footnote-reference-space when --footnote-references=superscript (or --use-latex-footnotes for the LaTeX writer) is supplied.

    There should be also an option to deactivate the space-trimming, e.g. --leave-footnote-reference-space. The implementation logic is a bit tricky: --leave-footnote-reference-space can't just be "settings.trim_footnote_reference_space = False", because of the automatic activation, unless the order of options is significant. If the order is significant, the automatic activation has to take place in the option-handling code. But config files are handled after the fact, so order can't be relied upon there. So they have to be two independent settings.

  • Enable tables inside XML comments, where "--" ends comments. I see three implementation possibilities:

    1. Make the table syntax characters into "table" directive options. This is the most flexible but most difficult, and we probably don't need that much flexibility.
    2. Substitute "~" for "-" with a specialized directive option (e.g. ":tildes:").
    3. Make the standard table syntax recognize "~" as well as "-", even without a directive option. Individual tables would have to be internally consistent.

    Directive options are preferable to configuration settings, because tables are document-specific. A pragma directive would be another approach, to set the syntax once for a whole document.

Directives

Directives below are often referred to as "module.directive", the directive function. The "module." is not part of the directive name when used in a document.

  • Allow directives to be added at run-time?

  • Use the language module for directive option names?

  • Add "substitution_only" and "substitution_ok" function attributes, and automate context checking?

  • Change directive functions to directive classes? Superclass' __init__() could handle all the bookkeeping.

  • Implement options on existing directives:

    • Add a "name" option to directives, to set an author-supplied identifier?

    • All directives that produce titled elements should grow implicit reference names based on the titles.

    • images.image: "border"?

      Units of measure? (See docutils-users, 2003-03-02, and docutils-develop, 2004-04-29.)

    • images.figure: "title" and "number", to indicate a formal figure?

    • parts.sectnum: "local"?, "start", "refnum"

      A "local" option could enable numbering for sections from a certain point down, and sections in the rest of the document are not numbered. For example, a reference section of a manual might be numbered, but not the rest. OTOH, an all-or-nothing approach would probably be enough.

      The "start" option will specify the sequence set to use at the same time as the starting value, for the first part of the section number (i.e., section, not subsection). For example:

      .. sectnum: :start: 1

.. sectnum: :start: A

.. sectnum: :start: 5

.. sectnum: :start: I

      The first one is the default: start at 1, numbered. The second one specifies letters, and start at "A". The third specifies numbers, start at 5. The last example could signal Roman numerals, although I don't know if they'd be applicable here. Enumerated lists already do all this; perhaps that code could be reused.

      Here comes the tricky part. The "sectnum" directive should be usable multiple times in a single document. For example, in a long document with "chapter" and "appendix" sections, there could be a second "sectnum" before the first appendix, changing the sequence used (from 1,2,3... to A,B,C...). This is where the "local" concept comes in. This part of the implementation can be left for later.

      A "refnum" option (better name?) would insert reference names (targets) consisting of the reference number. Then a URL could be of the form http://host/document.html#2.5 (or "2-5"?). Allow internal references by number? Allow name-based and number-based ids at the same time, or only one or the other (which would the table of contents use)? Usage issue: altering the section structure of a document could render hyperlinks invalid.

    • parts.contents: Add a "suppress" or "prune" option? It would suppress contents display for sections in a branch from that point down. Or a new directive, like "prune-contents"?

      Add an option to include topics in the TOC? Another for sidebars? See docutils-develop 2003-01-29.

    • misc.include:

      • @@@ Allow whitespace in paths (just not at line boundaries).

      • "encoding" option? Take default from runtime settings. Use Input component to read it in?

      • Option to select a range of lines?

      • Option to label lines?

      • Default directory for "built-in includes", using the C syntax #include <name>?

        Use C-preprocessor semantics for locating include files? E.g., .. include:: file.txt will read another file into the current one, relative to the current file's directory, and .. include:: <standard> will read a standard include file from docutils/include/. (Should "quotes" be required around non-standard include files?)

        —http://sf.net/mailarchive/message.php?msg_id=1938401

        I now think that docutils/parsers/rst/include/ is a better place for these files, since they're reStructuredText-specific.

        Keeping standard data files together with the package code makes sense to me. It seems much less complex to implement than a separate system data directory, such as /usr/share/docutils. Any reason a system data directory should be used? How does Distutils handle data files?

        How about an environment variable, say RSTINCLUDEPATH or RSTPATH? This could be combined with a setting/option to allow user-defined include directories.

        For a specific application, see the discussion of character entity substitution definition files above.

        Instead of C-include "<syntax>", perhaps a new "include-sys" directive?

    • misc.raw: "encoding" option?

    • misc.class: @@@ Allow multiple classes (whitespace-separated) to be declared.

  • Implement directives. Each of the list items below begins with an identifier of the form, "module_name.directive_function_name". The directive name itself could be the same as the directive_function_name, or it could differ.

    • html.imagemap (Useful outside of HTML? If not, replace with image only in non-HTML writers?)

    • parts.endnotes (or "footnotes"): See Footnote & Citation Gathering.

    • parts.citations: See Footnote & Citation Gathering.

    • misc.exec: Execute Python code & insert the results. Perhaps dangerous? Call it "python" to allow for other languages?

    • misc.system?: Execute an os.system() call, and insert the results (possibly as a literal block). Definitely dangerous! How to make it safe? Perhaps such processing should be left outside of the document, in the user's production system (a makefile or a script or whatever). Or, the directive could be disabled by default and only enabled with an explicit command-line option or config file setting. Even then, an interactive prompt may be useful, such as:

      The file.txt document you are processing contains a "system" directive requesting that the sudo rm -rf / command be executed. Allow it to execute? (y/N)

    • misc.eval: Evaluate an expression & insert the text. At parse time or at substitution time? Dangerous? Perhaps limit to canned macros; see text.date below.

    • misc.encoding: Specify the character encoding of the input data. But there are problems:

      • When it sees the directive, the parser will already have read the input data, and encoding determination will already have been done.
      • If a file with an "encoding" directive is edited and saved with a different encoding, the directive may cause data corruption.

    • misc.language: Specify the language of a document. There is a problem similar to the first problem listed for misc.encoding, although to a lesser degree.

    • misc.settings: Set any Docutils runtime setting from within a document?

    • misc.charents: Equivalent to:

      .. include:: {includepath}/charents.txt

    • Docutils already has the ability to say "use this content for Writer X" (via the "raw" directive), but it doesn't have the ability to say "use this content for any Writer other than X". It wouldn't be difficult to add this ability though.

      My first idea would be to add a set of conditional directives. Let's call them "writer-is" and "writer-is-not" for discussion purposes (don't worry about implemention details). We might have:

      .. writer-is:: text-only

   ::

       +----------+
       |   SNMP   |
       +----------+
       |   UDP    |
       +----------+
       |    IP    |
       +----------+
       | Ethernet |
       +----------+

.. writer-is:: pdf

   .. figure:: protocol_stack.eps

.. writer-is-not:: text-only pdf

   .. figure:: protocol_stack.png

      This could be an interface to the Filter transform (docutils.transforms.components.Filter).

      The ideas in adaptable file extensions above may also be applicable here.

      Here's an example of a directive that could produce multiple outputs (both raw troff pass-through and a GIF, for example) and allow the Writer to select.

      .. eqn::

   .EQ
   delim %%
   .EN
   %sum from i=o to inf c sup i~=~lim from {m -> inf}
   sum from i=0 to m sup i%
   .EQ
   delim off
   .EN

    • body.qa (directive a.k.a. "faq", "questions"): Questions & Answers. Implement as a generic two-column marked list? As a standalone (non-directive) construct? (Is the markup ambiguous?) Add support to parts.contents.

      New elements would be required. Perhaps:

      <!ELEMENT question_list (question_list_item+)>
<!ATTLIST question_list
    numbering  (none | local | global)
                        #IMPLIED
    start     NUMBER    #IMPLIED>
<!ELEMENT question_list_item (question, answer*)>
<!ELEMENT question %text.model;>
<!ELEMENT answer (%body.elements;)+>

      Originally I thought of implementing a Q&A list with special syntax:

      Q: What am I?

A: You are a question-and-answer
   list.

Q: What are you?

A: I am the omniscient "we".

      Where each "Q" and "A" could also be numbered (e.g., "Q1"). However, a simple enumerated or bulleted list will do just fine for syntax. A directive could treat the list specially; e.g. the first paragraph could be treated as a question, the remainder as the answer (multiple answers could be represented by nested lists). Without special syntax, this directive becomes low priority.

    • body.example: Examples; suggested by Simon Hefti. Semantics as per Docbook's "example"; admonition-style, numbered, reference, with a caption/title.

    • body.index: Index targets.

      Were I writing a book with an index, I guess I'd need two different kinds of index targets: inline/implicit and out-of-line/explicit. For example:

      In this `paragraph`:index:, several words are being
`marked`:index: inline as implicit `index`:index:
entries.

.. index:: markup
.. index:: syntax

The explicit index directives above would refer to
this paragraph.  It might also make sense to allow multiple
entries in an ``index`` directive:

.. index::
    markup
    syntax

      The words "paragraph", "marked", and "index" would become index entries pointing at the words in the first paragraph. The index entry words appear verbatim in the text. (Don't worry about the ugly ":index:" part; if indexing is the only/main application of interpreted text in your documents, it can be implicit and omitted.) The two directives provide manual indexing, where the index entry words ("markup" and "syntax") do not appear in the main text. We could combine the two directives into one:

      .. index:: markup; syntax

      Semicolons instead of commas because commas could be part of the index target, like:

      .. index:: van Rossum, Guido

      Another reason for index directives is because other inline markup wouldn't be possible within inline index targets.

      Sometimes index entries have multiple levels. Given:

      .. index:: statement syntax: expression statements

      In a hypothetical index, combined with other entries, it might look like this:

      statement syntax
    expression statements ..... 56
    assignment ................ 57
    simple statements ......... 58
    compound statements ....... 60

      Inline multi-level index targets could be done too. Perhaps something like:

      When dealing with `expression statements <statement syntax:>`,
we must remember ...

      The opposite sense could also be possible:

      When dealing with `index entries <:multi-level>`, there are
many permutations to consider.

      Also "see / see also" index entries.

      Given:

      Here's a paragraph.

.. index:: paragraph

      (The "index" directive above actually targets the preceding object.) The directive should produce something like this XML:

      <paragraph>
<index_entry text="paragraph"/>
Here's a paragraph.
</paragraph>

      This kind of content model would also allow true inline index-entries:

      Here's a `paragraph`:index:.

      If the "index" role were the default for the application, it could be dropped:

      Here's a `paragraph`.

      Both of these would result in this XML:

      <paragraph>
Here's a <index_entry>paragraph</index_entry>.
</paragraph>

    • body.literal: Literal block, possibly "formal" (see object numbering and object references above). Possible options:

      • "highlight" a range of lines
      • "number" or "line-numbers"

      • "styled" could indicate that the directive should check for style comments at the end of lines to indicate styling or markup.

        Specific derivatives (i.e., a "python-interactive" directive) could interpret style based on cues, like the ">>> " prompt and "input()"/"raw_input()" calls.

        See docutils-users 2003-03-03.

    • body.sidebar: Add to the already implemented directive. Allow internal section structure, with adornment styles independent of the main document.

    • body.list-table: See List-Driven Tables.

    • colorize.python: Colorize Python code. Fine for HTML output, but what about other formats? Revert to a literal block? Do we need some kind of "alternate" mechanism? Perhaps use a "pending" transform, which could switch its output based on the "format" in use. Use a factory function "transformFF()" which returns either "HTMLTransform()" instance or "GenericTransform" instance?

      If we take a Python-to-HTML pretty-printer and make it output a Docutils internal doctree (as per nodes.py) instead of HTML, then each output format's stylesheet (or equivalent) mechanism could take care of the rest. The pretty-printer code could turn this doctree fragment:

      <literal_block xml:space="preserve">
print 'This is Python code.'
for i in range(10):
    print i
</literal_block>

      into something like this ("</>" is end-tag shorthand):

      <literal_block xml:space="preserve" class="python">
<keyword>print</> <string>'This is Python code.'</>
<keyword>for</> <identifier>i</> <keyword
>in</> <expression>range(10)</>:
    <keyword>print</> <expression>i</>
</literal_block>

      But I'm leaning toward adding a single new general-purpose element, "phrase", equivalent to HTML's <span>. Here's the example rewritten using the generic "phrase":

      <literal_block xml:space="preserve" class="python">
<phrase class="keyword">print</> <phrase
 class="string">'This is Python code.'</>
<phrase class="keyword">for</> <phrase
 class="identifier">i</> <phrase class="keyword">in</> <phrase
 class="expression">range(10)</>:
    <phrase class="keyword">print</> <phrase
     class="expression">i</>
</literal_block>

      It's more verbose but more easily extensible and more appropriate for the case at hand. It allows us to edit style sheets to add support for new formats, not the Docutils code itself.

      Perhaps a single directive with a format parameter would be better:

      .. colorize:: python

   print 'This is Python code.'
   for i in range(10):
       print i

      But directives can have synonyms for convenience. "format:: python" was suggested, but "format" seems too generic.

    • text.date: Datestamp. For substitutions. The directive could be followed by a formatting string, using strftime codes. Default is "%Y-%m-%d" (ISO 8601 date), but time fields can also be used.

      • Combined with the "include" directive, implement canned macros? E.g.:

        .. include:: <macros>

Today's date is |date|.

        Where "macros" contains .. |date| date::, among others.

    • text.time: Timestamp. For substitutions. Shortcut for .. date:: %H:%M. Date fields can also be used.

    • pysource.usage: Extract a usage message from the program, either by running it at the command line with a --help option or through an exposed API. [Suggestion for Optik.]

Interpreted Text

Interpreted text is entirely a reStructuredText markup construct, a way to get around built-in limitations of the medium. Some roles are intended to introduce new doctree elements, such as "title-reference". Others are merely convenience features, like "RFC".

All supported interpreted text roles must already be known to the Parser when they are encountered in a document. Whether pre-defined in core/client code, or in the document, doesn't matter; the roles just need to have already been declared. Adding a new role often involves adding a new element to the DTD and may require extensive support, therefore such additions should be well thought-out. There should be a limited number of roles.

The only place where no limit is placed on variation is at the start, at the Reader/Parser interface. Transforms are inserted by the Reader into the Transformer's queue, where non-standard elements are converted. Once past the Transformer, no variation from the standard Docutils doctree is possible.

An example is the Python Source Reader, which will use interpreted text extensively. The default role will be "Python identifier", which will be further interpreted by namespace context into <class>, <method>, <module>, <attribute>, etc. elements (see pysource.dtd), which will be transformed into standard hyperlink references, which will be processed by the various Writers. No Writer will need to have any knowledge of the Python-Reader origin of these elements.

  • Alan Jaffray suggested (and I agree) that it would be sensible to:

    • have a directive and/or command-line option to specify a default role for interpreted text
    • allow the reST processor to take an argument for the default role (this will be subsumed by the above via the runtime settings mechanism)
    • issue a warning when processing documents with no default role which contain interpreted text with no explicitly specified role (there will always be a default role, so this won't happen)

  • Add explicit interpreted text roles for the rest of the implicit inline markup constructs: named-reference, anonymous-reference, footnote-reference, citation-reference, substitution-reference, target, uri-reference (& synonyms).

  • Add directives for each role as well? This would allow indirect nested markup:

    This text contains |nested inline markup|.

.. |nested inline markup| emphasis::

   nested ``inline`` markup

  • Implement roles:

    • "acronym" and "abbreviation": Associate the full text with a short form. Jason Diamond's description:

      I want to translate `reST`:acronym: into <acronym title='reStructuredText'>reST</acronym>. The value of the title attribute has to be defined out-of-band since you can't parameterize interpreted text. Right now I have them in a separate file but I'm experimenting with creating a directive that will use some form of reST syntax to let you define them.

      Should Docutils complain about undefined acronyms or abbreviations?

      What to do if there are multiple definitions? How to differentiate between CSS (Content Scrambling System) and CSS (Cascading Style Sheets) in a single document? David Priest responds,

      The short answer is: you don't. Anyone who did such a thing would be writing very poor documentation indeed. (Though I note that somewhere else in the docs, there's mention of allowing replacement text to be associated with the abbreviation. That takes care of the duplicate acronyms/abbreviations problem, though a writer would be foolish to ever need it.)

      How to define the full text? Possibilities:

      1. With a directive and a definition list?

        .. acronyms::

   reST
       reStructuredText
   DPS
       Docstring Processing System

        Would this list remain in the document as a glossary, or would it simply build an internal lookup table? A "glossary" directive could be used to make the intention clear. Acronyms/abbreviations and glossaries could work together.

        Then again, a glossary could be formed by gathering individual definitions from around the document.

      2. Some kind of inline parameter syntax?

        `reST <reStructuredText>`:acronym: is `WYSIWYG <what you
see is what you get>`:acronym: plaintext markup.

      3. A combination of 1 & 2?

        The multiple definitions issue could be handled by establishing rules of priority. For example, directive-based lookup tables have highest priority, followed by the first inline definition. Multiple definitions in directive-based lookup tables would trigger warnings, similar to the rules of implicit hyperlink targets.

    • "annotation": The equivalent of the HTML "title" attribute. This is secondary information that may "pop up" when the pointer hovers over the main text. A corresponding directive would be required to associate annotations with the original text (by name, or positionally as in anonymous targets?).

    • "figure", "table", "listing", "chapter", "page", etc: See object numbering and object references above.

    • "term"?: Unfamiliar or specialized terminology.

    • "glossary-term": This would establish a link to a glossary. It would require an associated "glossary-entry" directive, whose contents could be a definition list:

      .. glossary-entry::

   term1
       definition1
   term2
       definition2

      This would allow entries to be defined anywhere in the document, and collected (via a "glossary" directive perhaps) at one point.

Unimplemented Transforms

  • Footnote & Citation Gathering

    Collect and move footnotes & citations to the end of a document. (Separate transforms.)

  • Hyperlink Target Gathering

    It probably comes in two phases, because in a Python context we need to resolve them on a per-docstring basis [do we? --DG], but if the user is trying to do the callout form of presentation, they would then want to group them all at the end of the document.

  • Reference Merging

    When merging two or more subdocuments (such as docstrings), conflicting references may need to be resolved. There may be:

    • duplicate reference and/or substitution names that need to be made unique; and/or
    • duplicate footnote numbers that need to be renumbered.

    Should this be done before or after reference-resolving transforms are applied? What about references from within one subdocument to inside another?

  • Document Splitting

    If the processed document is written to multiple files (possibly in a directory tree), it will need to be split up. Internal references will have to be adjusted.

    (HTML only? Initially, yes. Eventually, anything should be splittable.)

    Idea: insert a "destination" attribute into the root element of each split-out document, containing the path/filename. The Output object or Writer will recognize this attribute and split out the files accordingly. Must allow for common headers & footers, prev/next, breadcrumbs, etc.

  • Navigation

    If a document is split up, each segment will need navigation links: parent, children (small TOC), previous (preorder), next (preorder). Part of Document Splitting?

  • List of System Messages

    The system_message elements are inserted into the document tree, adjacent to the problems themselves where possible. Some (those generated post-parse) are kept until later, in document.messages, and added as a special final section, "Docutils System Messages".

    Docutils could be made to generate hyperlinks to all known system_messages and add them to the document, perhaps to the end of the "Docutils System Messages" section.

    Fred L. Drake, Jr. wrote:

    I'd like to propose that both parse- and transformation-time messages are included in the "Docutils System Messages" section. If there are no objections, I can make the change.

    The advantage of the current way of doing things is that parse-time system messages don't require a transform; they're already in the document. This is valuable for testing (unit tests, tools/quicktest.py). So if we do decide to make a change, I think the insertion of parse-time system messages ought to remain as-is and the Messages transform ought to move all parse-time system messages (remove from their originally inserted positions, insert in System Messages section).

  • Index Generation

HTML Writer

  • Construct a templating system, as in ht2html/yaptu, using directives and substitutions for dynamic stuff? Or a specialized writer to generate .ht & links.h files for ht2html?

    This is an overly generic, ambitious, and unnecessary idea. The two systems, ht2html & yaptu, represent different approaches to templating. Ht2html stiches bits of HTML together to make unified web pages. Yaptu is for generic programmable text replacement. Templating should simply be left to other systems. Users can choose a templating system to apply to their reStructuredText documents as best serves their interests.

  • Add more support for <link> elements, especially for navigation bars.

  • Make the admonitions more distinctive and varied.

  • Make the "class" attributes optional? Implies no stylesheet?

  • Base list compaction on the spacing of source list? Would require parser support. (Idea: fantasai, 16 Dec 2002, doc-sig.)

  • Add a tool tip ("title" attribute?) to footnote back-links identifying them as such. Text in Docutils language module.

  • Add an option to restrict the document title to <head><title> only, and not include it in the document body. Subtitle?

  • Insert a comment at the top of HTML files that describes how to deal with the broken servers w.r.t. encodings? Perhaps something like this:

    <!-- If your browser is showing gibberish, the server may be broken. Try manually setting the character coding to "UTF-8". In Mozilla/Firefox, do ... In Internet Explorer, do ... For details, see <URL>. -->

Front-End Tools

  • What about if we don't know which Reader and/or Writer we are going to use? If the Reader/Writer is specified on the command-line? (Will this ever happen?)

    Perhaps have different types of front ends:

    1. Fully qualified: Reader and Writer are hard-coded into the front end (e.g. pep2html [options], pysource2pdf [options]).
    2. Partially qualified: Reader is hard-coded, and the Writer is specified a sub-command (e.g. pep2 html [options], pysource2 pdf [options]). The Writer is known before option processing happens, allowing the OptionParser to be built dynamically. Alternatively, the Writer could be hard-coded and the Reader specified as a sub-command (e.g. htmlfrom pep [options]).
    3. Unqualified: Reader and Writer are specified as subcommands (e.g. publish pep html [options], publish pysource pdf [options]). A single front end would be sufficient, but probably only useful for testing purposes.
    4. Dynamic: Reader and/or Writer are specified by options, with defaults if unspecified (e.g. publish --writer pdf [options]). Is this possible? The option parser would have to be told about new options it needs to handle, on the fly. Component-specific options would have to be specified after the component-specifying option.

    Allow common options before subcommands, as in CVS? Or group all options together? In the case of the fully qualified front ends, all the options will have to be grouped together anyway, so there's no advantage (we can't use it to avoid conflicts) to splitting common and component-specific options apart.

  • Parameterize help text & defaults somehow? Perhaps a callback? Or initialize settings_spec in __init__ or init_options?

  • Disable common options that don't apply?

  • Implement the "sectnum" directive as a command-line option also?

  • @@@ Come up with better names for the most-used tools, and install them as scripts.

  • Create a single dynamic or unqualified front end that can be installed?

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Docutils Web Site

Author: | David Goodger; open to all Docutils developers
Contact: | goodger@python.org
Date: | 2004-06-03
Revision: | 1.1
Copyright: | This document has been placed in the public domain.

The Docutils web site, <http://docutils.sourceforge.net/>, is maintained automatically by the docutils-update script, run as an hourly cron job (by user "goodger"). The script will process any .txt file which is newer than the corresponding .html file in the project's web directory on SourceForge (/home/groups/d/do/docutils/htdocs/). For a new .txt file, just SSH to SourceForge and

touch filename.html
chmod g+w filename.html
sleep 1
touch filename.txt

The script will take care of the rest within an hour. Thereafter whenever the .txt file is modified (checked in to CVS), the .html will be regenerated automatically.

After adding directories to CVS, allow the script to run once to create the directories in the filesystem before preparing for HTML processing as described above.

The docutils-update script is in CVS as sandbox/davidg/infrastructure/docutils-update.

View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


A Record of reStructuredText Syntax Alternatives

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.33
Date: | 2004-06-14
Copyright: | This document has been placed in the public domain.

The following are ideas, alternatives, and justifications that were considered for reStructuredText syntax, which did not originate with Setext or StructuredText. For an analysis of constructs which did originate with StructuredText or Setext, please see Problems With StructuredText. See the reStructuredText Markup Specification for full details of the established syntax.

Contents

  • Implemented
    • Field Lists
    • Interpreted Text "Roles"
    • Comments
    • Anonymous Hyperlinks
    • Reworking Explicit Markup (Round 1)
    • Backquotes in Phrase-Links
    • Substitution Mechanism
    • Inline External Targets
    • Doctree Representation of Transitions
  • Not Implemented
    • Reworking Footnotes
  • Tabled
    • Reworking Explicit Markup (Round 2)
  • To Do
    • Auto-Enumerated Lists
    • Nested Inline Markup
    • Syntax for Line Blocks
      • Syntax
      • Internal Representation
      • Output
      • Implementation Plan
    • List-Driven Tables
  • ... Or Not To Do?
    • Compound Enumerated Lists
    • Sloppy Indentation of List Items
    • Lazy Indentation of List Items
      • David's Idea for Lazy Indentation
    • Multiple Roles in Interpreted Text
    • Parameterized Interpreted Text
    • Syntax for Interpreted Text Role Bindings
    • Character Processing

Implemented

Field Lists

Prior to the syntax for field lists being finalized, several alternatives were proposed.

  1. Unadorned RFC822 everywhere:

    Author: Me
Version: 1

    Advantages: clean, precedent (RFC822-compliant). Disadvantage: ambiguous (these paragraphs are a prime example).

    Conclusion: rejected.

  2. Special case: use unadorned RFC822 for the very first or very last text block of a document:

    """
Author: Me
Version: 1

The rest of the document...
"""

    Advantages: clean, precedent (RFC822-compliant). Disadvantages: special case, flat (unnested) field lists only, still ambiguous:

    """
Usage: cmdname [options] arg1 arg2 ...

We obviously *don't* want the like above to be interpreted as a
field list item.  Or do we?
"""

    Conclusion: rejected for the general case, accepted for specific contexts (PEPs, email).

  3. Use a directive:

    .. fields::

   Author: Me
   Version: 1

    Advantages: explicit and unambiguous, RFC822-compliant. Disadvantage: cumbersome.

    Conclusion: rejected for the general case (but such a directive could certainly be written).

  4. Use Javadoc-style:

    @Author: Me
@Version: 1
@param a: integer

    Advantages: unambiguous, precedent, flexible. Disadvantages: non-intuitive, ugly, not RFC822-compliant.

    Conclusion: rejected.

  5. Use leading colons:

    :Author: Me
:Version: 1

    Advantages: unambiguous, obvious (almost RFC822-compliant), flexible, perhaps even elegant. Disadvantages: no precedent, not quite RFC822-compliant.

    Conclusion: accepted!

  6. Use double colons:

    Author:: Me
Version:: 1

    Advantages: unambiguous, obvious? (almost RFC822-compliant), flexible, similar to syntax already used for literal blocks and directives. Disadvantages: no precedent, not quite RFC822-compliant, similar to syntax already used for literal blocks and directives.

    Conclusion: rejected because of the syntax similarity & conflicts.

Why is RFC822 compliance important? It's a universal Internet standard, and super obvious. Also, I'd like to support the PEP format (ulterior motive: get PEPs to use reStructuredText as their standard). But it would be easy to get used to an alternative (easy even to convert PEPs; probably harder to convert python-deviants ;-).

Unfortunately, without well-defined context (such as in email headers: RFC822 only applies before any blank lines), the RFC822 format is ambiguous. It is very common in ordinary text. To implement field lists unambiguously, we need explicit syntax.

The following question was posed in a footnote:

Should "bibliographic field lists" be defined at the parser level, or at the DPS transformation level? In other words, are they reStructuredText-specific, or would they also be applicable to another (many/every other?) syntax?

The answer is that bibliographic fields are a reStructuredText-specific markup convention. Other syntaxes may implement the bibliographic elements explicitly. For example, there would be no need for such a transformation for an XML-based markup syntax.

Interpreted Text "Roles"

The original purpose of interpreted text was as a mechanism for descriptive markup, to describe the nature or role of a word or phrase. For example, in XML we could say "<function>len</function>" to mark up "len" as a function. It is envisaged that within Python docstrings (inline documentation in Python module source files, the primary market for reStructuredText) the role of a piece of interpreted text can be inferred implicitly from the context of the docstring within the program source. For other applications, however, the role may have to be indicated explicitly.

Interpreted text is enclosed in single backquotes (`).

  1. Initially, it was proposed that an explicit role could be indicated as a word or phrase within the enclosing backquotes:

    • As a prefix, separated by a colon and whitespace:

      `role: interpreted text`

    • As a suffix, separated by whitespace and a colon:

      `interpreted text :role`

    There are problems with the initial approach:

    • There could be ambiguity with interpreted text containing colons. For example, an index entry of "Mission: Impossible" would require a backslash-escaped colon.
    • The explicit role is descriptive markup, not content, and will not be visible in the processed output. Putting it inside the backquotes doesn't feel right; the role isn't being quoted.

  2. Tony Ibbs suggested that the role be placed outside the backquotes:

    role:`prefix` or `suffix`:role

    This removes the embedded-colons ambiguity, but limits the role identifier to be a single word (whitespace would be illegal). Since roles are not meant to be visible after processing, the lack of whitespace support is not important.

    The suggested syntax remains ambiguous with respect to ratios and some writing styles. For example, suppose there is a "signal" identifier, and we write:

    ...calculate the `signal`:noise ratio.

    "noise" looks like a role.

  3. As an improvement on #2, we can bracket the role with colons:

    :role:`prefix` or `suffix`:role:

    This syntax is similar to that of field lists, which is fine since both are doing similar things: describing.

    This is the syntax chosen for reStructuredText.

  4. Another alternative is two colons instead of one:

    role::`prefix` or `suffix`::role

    But this is used for analogies ("A:B::C:D": "A is to B as C is to D").

    Both alternative #2 and #4 lack delimiters on both sides of the role, making it difficult to parse (by the reader).

  5. Some kind of bracketing could be used:

    • Parentheses:

      (role)`prefix` or `suffix`(role)

    • Braces:

      {role}`prefix` or `suffix`{role}

    • Square brackets:

      [role]`prefix` or `suffix`[role]

    • Angle brackets:

      <role>`prefix` or `suffix`<role>

      (The overlap of *ML tags with angle brackets would be too confusing and precludes their use.)

Syntax #3 was chosen for reStructuredText.

Comments

A problem with comments (actually, with all indented constructs) is that they cannot be followed by an indented block -- a block quote -- without swallowing it up.

I thought that perhaps comments should be one-liners only. But would this mean that footnotes, hyperlink targets, and directives must then also be one-liners? Not a good solution.

Tony Ibbs suggested a "comment" directive. I added that we could limit a comment to a single text block, and that a "multi-block comment" could use "comment-start" and "comment-end" directives. This would remove the indentation incompatibility. A "comment" directive automatically suggests "footnote" and (hyperlink) "target" directives as well. This could go on forever! Bad choice.

Garth Kidd suggested that an "empty comment", a ".." explicit markup start with nothing on the first line (except possibly whitespace) and a blank line immediately following, could serve as an "unindent". An empty comment does not swallow up indented blocks following it, so block quotes are safe. "A tiny but practical wart." Accepted.

Anonymous Hyperlinks

Alan Jaffray came up with this idea, along with the following syntax:

Search the `Python DOC-SIG mailing list archives`{}_.

.. _: http://mail.python.org/pipermail/doc-sig/

The idea is sound and useful. I suggested a "double underscore" syntax:

Search the `Python DOC-SIG mailing list archives`__.

.. __: http://mail.python.org/pipermail/doc-sig/

But perhaps single underscores are okay? The syntax looks better, but the hyperlink itself doesn't explicitly say "anonymous":

Search the `Python DOC-SIG mailing list archives`_.

.. _: http://mail.python.org/pipermail/doc-sig/

Mixing anonymous and named hyperlinks becomes confusing. The order of targets is not significant for named hyperlinks, but it is for anonymous hyperlinks:

Hyperlinks: anonymous_, named_, and another anonymous_.

.. _named: named
.. _: anonymous1
.. _: anonymous2

Without the extra syntax of double underscores, determining which hyperlink references are anonymous may be difficult. We'd have to check which references don't have corresponding targets, and match those up with anonymous targets. Keeping to a simple consistent ordering (as with auto-numbered footnotes) seems simplest.

reStructuredText will use the explicit double-underscore syntax for anonymous hyperlinks. An alternative (see Reworking Explicit Markup (Round 1) below) for the somewhat awkward ".. __:" syntax is "__":

An anonymous__ reference.

__ http://anonymous

Reworking Explicit Markup (Round 1)

Alan Jaffray came up with the idea of anonymous hyperlinks, added to reStructuredText. Subsequently it was asserted that hyperlinks (especially anonymous hyperlinks) would play an increasingly important role in reStructuredText documents, and therefore they require a simpler and more concise syntax. This prompted a review of the current and proposed explicit markup syntaxes with regards to improving usability.

  1. Original syntax:

    .. _blah:                     internal hyperlink target
.. _blah: http://somewhere    external hyperlink target
.. _blah: blahblah_           indirect hyperlink target
.. __:                        anonymous internal target
.. __: http://somewhere       anonymous external target
.. __: blahblah_              anonymous indirect target
.. [blah] http://somewhere    footnote
.. blah:: http://somewhere    directive
.. blah: http://somewhere     comment

    Note

    The comment text was intentionally made to look like a hyperlink target.

    Origins:

    • Except for the colon (a delimiter necessary to allow for phrase-links), hyperlink target .. _blah: comes from Setext.
    • Comment syntax from Setext.
    • Footnote syntax from StructuredText ("named links").
    • Directives and anonymous hyperlinks original to reStructuredText.

    Advantages:

    • Consistent explicit markup indicator: "..".
    • Consistent hyperlink syntax: ".. _" & ":".

    Disadvantages:

    • Anonymous target markup is awkward: ".. __:".
    • The explicit markup indicator ("..") is excessively overloaded?
    • Comment text is limited (can't look like a footnote, hyperlink, or directive). But this is probably not important.

  2. Alan Jaffray's proposed syntax #1:

    __ _blah                      internal hyperlink target
__ blah: http://somewhere     external hyperlink target
__ blah: blahblah_            indirect hyperlink target
__                            anonymous internal target
__ http://somewhere           anonymous external target
__ blahblah_                  anonymous indirect target
__ [blah] http://somewhere    footnote
.. blah:: http://somewhere    directive
.. blah: http://somewhere     comment

    The hyperlink-connoted underscores have become first-level syntax.

    Advantages:

    • Anonymous targets are simpler.
    • All hyperlink targets are one character shorter.

    Disadvantages:

    • Inconsistent internal hyperlink targets. Unlike all other named hyperlink targets, there's no colon. There's an extra leading underscore, but we can't drop it because without it, "blah" looks like a relative URI. Unless we restore the colon:

      __ blah:                      internal hyperlink target

    • Obtrusive markup?

  3. Alan Jaffray's proposed syntax #2:

    .. _blah                      internal hyperlink target
.. blah: http://somewhere     external hyperlink target
.. blah: blahblah_            indirect hyperlink target
..                            anonymous internal target
.. http://somewhere           anonymous external target
.. blahblah_                  anonymous indirect target
.. [blah] http://somewhere    footnote
!! blah: http://somewhere     directive
## blah: http://somewhere     comment

    Leading underscores have been (almost) replaced by "..", while comments and directives have gained their own syntax.

    Advantages:

    • Anonymous hyperlinks are simpler.
    • Unique syntax for comments. Connotation of "comment" from some programming languages (including our favorite).
    • Unique syntax for directives. Connotation of "action!".

    Disadvantages:

    • Inconsistent internal hyperlink targets. Again, unlike all other named hyperlink targets, there's no colon. There's a leading underscore, matching the trailing underscores of references, which no other hyperlink targets have. We can't drop that one leading underscore though: without it, "blah" looks like a relative URI. Again, unless we restore the colon:

      .. blah:                      internal hyperlink target

    • All (except for internal) hyperlink targets lack their leading underscores, losing the "hyperlink" connotation.

    • Obtrusive syntax for comments. Alternatives:

      ;; blah: http://somewhere
   (also comment syntax in Lisp & others)
,, blah: http://somewhere
   ("comma comma": sounds like "comment"!)

    • Iffy syntax for directives. Alternatives?

  4. Tony Ibbs' proposed syntax:

    .. _blah:                     internal hyperlink target
.. _blah: http://somewhere    external hyperlink target
.. _blah: blahblah_           indirect hyperlink target
..                            anonymous internal target
.. http://somewhere           anonymous external target
.. blahblah_                  anonymous indirect target
.. [blah] http://somewhere    footnote
.. blah:: http://somewhere    directive
.. blah: http://somewhere     comment

    This is the same as the current syntax, except for anonymous targets which drop their "__: ".

    Advantage:

    • Anonymous targets are simpler.

    Disadvantages:

    • Anonymous targets lack their leading underscores, losing the "hyperlink" connotation.
    • Anonymous targets are almost indistinguishable from comments. (Better to know "up front".)

  5. David Goodger's proposed syntax: Perhaps going back to one of Alan's earlier suggestions might be the best solution. How about simply adding "__ " as a synonym for ".. __: " in the original syntax? These would become equivalent:

    .. __:                        anonymous internal target
.. __: http://somewhere       anonymous external target
.. __: blahblah_              anonymous indirect target

__                            anonymous internal target
__ http://somewhere           anonymous external target
__ blahblah_                  anonymous indirect target

Alternative 5 has been adopted.

Backquotes in Phrase-Links

[From a 2001-06-05 Doc-SIG post in reply to questions from Doug Hellmann.]

The first draft of the spec, posted to the Doc-SIG in November 2000, used square brackets for phrase-links. I changed my mind because:

  1. In the first draft, I had already decided on single-backquotes for inline literal text.
  2. However, I wanted to minimize the necessity for backslash escapes, for example when quoting Python repr-equivalent syntax that uses backquotes.
  3. The processing of identifiers (function/method/attribute/module etc. names) into hyperlinks is a useful feature. PyDoc recognizes identifiers heuristically, but it doesn't take much imagination to come up with counter-examples where PyDoc's heuristics would result in embarassing failure. I wanted to do it deterministically, and that called for syntax. I called this construct "interpreted text".
  4. Leveraging off the *emphasis*/**strong** syntax, lead to the idea of using double-backquotes as syntax.
  5. I worked out some rules for inline markup recognition.
  6. In combination with #5, double backquotes lent themselves to inline literals, neatly satisfying #2, minimizing backslash escapes. In fact, the spec says that no interpretation of any kind is done within double-backquote inline literal text; backslashes do no escaping within literal text.
  7. Single backquotes are then freed up for interpreted text.
  8. I already had square brackets required for footnote references.
  9. Since interpreted text will typically turn into hyperlinks, it was a natural fit to use backquotes as the phrase-quoting syntax for trailing-underscore hyperlinks.

The original inspiration for the trailing underscore hyperlink syntax was Setext. But for phrases Setext used a very cumbersome underscores_between_words_like_this_ syntax.

The underscores can be viewed as if they were right-pointing arrows: -->. So hyperlink_ points away from the reference, and .. _hyperlink: points toward the target.

Substitution Mechanism

Substitutions arose out of a Doc-SIG thread begun on 2001-10-28 by Alan Jaffray, "reStructuredText inline markup". It reminded me of a missing piece of the reStructuredText puzzle, first referred to in my contribution to "Documentation markup & processing / PEPs" (Doc-SIG 2001-06-21).

Substitutions allow the power and flexibility of directives to be shared by inline text. They are a way to allow arbitrarily complex inline objects, while keeping the details out of the flow of text. They are the equivalent of SGML/XML's named entities. For example, an inline image (using reference syntax alternative 4d (vertical bars) and definition alternative 3, the alternatives chosen for inclusion in the spec):

The |biohazard| symbol must be used on containers used to dispose
of medical waste.

.. |biohazard| image:: biohazard.png
   [height=20 width=20]

The |biohazard| substitution reference will be replaced in-line by whatever the .. |biohazard| substitution definition generates (in this case, an image). A substitution definition contains the substitution text bracketed with vertical bars, followed by a an embedded inline-compatible directive, such as "image". A transform is required to complete the substitution.

Syntax alternatives for the reference:

  1. Use the existing interpreted text syntax, with a predefined role such as "sub":

    The `biohazard`:sub: symbol...

    Advantages: existing syntax, explicit. Disadvantages: verbose, obtrusive.

  2. Use a variant of the interpreted text syntax, with a new suffix akin to the underscore in phrase-link references:

    (a) `name`@
(b) `name`#
(c) `name`&
(d) `name`/
(e) `name`<
(f) `name`::
(g) `name`:

    Due to incompatibility with other constructs and ordinary text usage, (f) and (g) are not possible.

  3. Use interpreted text syntax with a fixed internal format:

    (a) `:name:`
(b) `name:`
(c) `name::`
(d) `::name::`
(e) `%name%`
(f) `#name#`
(g) `/name/`
(h) `&name&`
(i) `|name|`
(j) `[name]`
(k) `<name>`
(l) `&name;`
(m) `'name'`

    To avoid ML confusion (k) and (l) are definitely out. Square brackets (j) won't work in the target (the substitution definition would be indistinguishable from a footnote).

    The `/name/` syntax (g) is reminiscent of "s/find/sub" substitution syntax in ed-like languages. However, it may have a misleading association with regexps, and looks like an absolute POSIX path. (i) is visually equivalent and lacking the connotations.

    A disadvantage of all of these is that they limit interpreted text, albeit only slightly.

  4. Use specialized syntax, something new:

    (a) #name#
(b) @name@
(c) /name/
(d) |name|
(e) <<name>>
(f) //name//
(g) ||name||
(h) ^name^
(i) [[name]]
(j) ~name~
(k) !name!
(l) =name=
(m) ?name?
(n) >name<

    "#" (a) and "@" (b) are obtrusive. "/" (c) without backquotes looks just like a POSIX path; it is likely for such usage to appear in text.

    "|" (d) and "^" (h) are feasible.

  5. Redefine the trailing underscore syntax. See definition syntax alternative 4, below.

Syntax alternatives for the definition:

  1. Use the existing directive syntax, with a predefined directive such as "sub". It contains a further embedded directive resolving to an inline-compatible object:

    .. sub:: biohazard
   .. image:: biohazard.png
      [height=20 width=20]

.. sub:: parrot
   That bird wouldn't *voom* if you put 10,000,000 volts
   through it!

    The advantages and disadvantages are the same as in inline alternative 1.

  2. Use syntax as in #1, but with an embedded directivecompressed:

    .. sub:: biohazard image:: biohazard.png
   [height=20 width=20]

    This is a bit better than alternative 1, but still too much.

  3. Use a variant of directive syntax, incorporating the substitution text, obviating the need for a special "sub" directive name. If we assume reference alternative 4d (vertical bars), the matching definition would look like this:

    .. |biohazard| image:: biohazard.png
   [height=20 width=20]

  4. (Suggested by Alan Jaffray on Doc-SIG from 2001-11-06.)

    Instead of adding new syntax, redefine the trailing underscore syntax to mean "substitution reference" instead of "hyperlink reference". Alan's example:

    I had lunch with Jonathan_ today.  We talked about Zope_.

.. _Jonathan: lj [user=jhl]
.. _Zope: http://www.zope.org/

    A problem with the proposed syntax is that URIs which look like simple reference names (alphanum plus ".", "-", "_") would be indistinguishable from substitution directive names. A more consistent syntax would be:

    I had lunch with Jonathan_ today.  We talked about Zope_.

.. _Jonathan: lj:: user=jhl
.. _Zope: http://www.zope.org/

    (:: after .. _Jonathan: lj.)

    The "Zope" target is a simple external hyperlink, but the "Jonathan" target contains a directive. Alan proposed is that the reference text be replaced by whatever the referenced directive (the "directive target") produces. A directive reference becomes a hyperlink reference if the contents of the directive target resolve to a hyperlink. If the directive target resolves to an icon, the reference is replaced by an inline icon. If the directive target resolves to a hyperlink, the directive reference becomes a hyperlink reference.

    This seems too indirect and complicated for easy comprehension.

    The reference in the text will sometimes become a link, sometimes not. Sometimes the reference text will remain, sometimes not. We don't know at the reference:

    This is a `hyperlink reference`_; its text will remain.
This is an `inline icon`_; its text will disappear.

    That's a problem.

The syntax that has been incorporated into the spec and parser is reference alternative 4d with definition alternative 3:

The |biohazard| symbol...

.. |biohazard| image:: biohazard.png
   [height=20 width=20]

We can also combine substitution references with hyperlink references, by appending a "_" (named hyperlink reference) or "__" (anonymous hyperlink reference) suffix to the substitution reference. This allows us to click on an image-link:

The |biohazard|_ symbol...

.. |biohazard| image:: biohazard.png
   [height=20 width=20]
.. _biohazard: http://www.cdc.gov/

There have been several suggestions for the naming of these constructs, originally called "substitution references" and "substitutions".

  1. Candidate names for the reference construct:
    1. substitution reference
    2. tagging reference
    3. inline directive reference
    4. directive reference
    5. indirect inline directive reference
    6. inline directive placeholder
    7. inline directive insertion reference
    8. directive insertion reference
    9. insertion reference
    10. directive macro reference
    11. macro reference
    12. substitution directive reference
  2. Candidate names for the definition construct:
    1. substitution
    2. substitution directive
    3. tag
    4. tagged directive
    5. directive target
    6. inline directive
    7. inline directive definition
    8. referenced directive
    9. indirect directive
    10. indirect directive definition
    11. directive definition
    12. indirect inline directive
    13. named directive definition
    14. inline directive insertion definition
    15. directive insertion definition
    16. insertion definition
    17. insertion directive
    18. substitution definition
    19. directive macro definition
    20. macro definition
    21. substitution directive definition
    22. substitution definition

"Inline directive reference" (1c) seems to be an appropriate term at first, but the term "inline" is redundant in the case of the reference. Its counterpart "inline directive definition" (2g) is awkward, because the directive definition itself is not inline.

"Directive reference" (1d) and "directive definition" (2k) are too vague. "Directive definition" could be used to refer to any directive, not just those used for inline substitutions.

One meaning of the term "macro" (1k, 2s, 2t) is too programming-language-specific. Also, macros are typically simple text substitution mechanisms: the text is substituted first and evaluated later. reStructuredText substitution definitions are evaluated in place at parse time and substituted afterwards.

"Insertion" (1h, 1i, 2n-2q) is almost right, but it implies that something new is getting added rather than one construct being replaced by another.

Which brings us back to "substitution". The overall best names are "substitution reference" (1a) and "substitution definition" (2v). A long way to go to add one word!

Inline External Targets

Currently reStructuredText has two hyperlink syntax variations:

  • Named hyperlinks:

    This is a named reference_ of one word ("reference").  Here is
a `phrase reference`_.  Phrase references may even cross `line
boundaries`_.

.. _reference: http://www.example.org/reference/
.. _phrase reference: http://www.example.org/phrase_reference/
.. _line boundaries: http://www.example.org/line_boundaries/
    • Advantages:
      • The plaintext is readable.
      • Each target may be reused multiple times (e.g., just write "reference_" again).
      • No syncronized ordering of references and targets is necessary.
    • Disadvantages:
      • The reference text must be repeated as target names; could lead to mistakes.
      • The target URLs may be located far from the references, and hard to find in the plaintext.

  • Anonymous hyperlinks (in current reStructuredText):

    This is an anonymous reference__.  Here is an anonymous
`phrase reference`__.  Phrase references may even cross `line
boundaries`__.

__ http://www.example.org/reference/
__ http://www.example.org/phrase_reference/
__ http://www.example.org/line_boundaries/
    • Advantages:
      • The plaintext is readable.
      • The reference text does not have to be repeated.
    • Disadvantages:
      • References and targets must be kept in sync.
      • Targets cannot be reused.
      • The target URLs may be located far from the references.

For comparison and historical background, StructuredText also has two syntaxes for hyperlinks:

  • First, "reference text":URL:

    This is a "reference":http://www.example.org/reference/
of one word ("reference").  Here is a "phrase
reference":http://www.example.org/phrase_reference/.

  • Second, "reference text", http://example.com/absolute_URL:

    This is a "reference", http://www.example.org/reference/
of one word ("reference").  Here is a "phrase reference",
http://www.example.org/phrase_reference/.

Both syntaxes share advantages and disadvantages:

  • Advantages:
    • The target is specified immediately adjacent to the reference.
  • Disadvantages:
    • Poor plaintext readability.
    • Targets cannot be reused.
    • Both syntaxes use double quotes, common in ordinary text.
    • In the first syntax, the URL and the last word are stuck together, exacerbating the line wrap problem.
    • The second syntax is too magical; text could easily be written that way by accident (although only absolute URLs are recognized here, perhaps because of the potential for ambiguity).

A new type of "inline external hyperlink" has been proposed.

  1. On 2002-06-28, Simon Budig proposed a new syntax for reStructuredText hyperlinks:

    This is a reference_(http://www.example.org/reference/) of one
word ("reference").  Here is a `phrase
reference`_(http://www.example.org/phrase_reference/).  Are
these examples, (single-underscore), named?  If so, `anonymous
references`__(http://www.example.org/anonymous/) using two
underscores would probably be preferable.

    The syntax, advantages, and disadvantages are similar to those of StructuredText.

    • Advantages:
      • The target is specified immediately adjacent to the reference.
    • Disadvantages:
      • Poor plaintext readability.
      • Targets cannot be reused (unless named, but the semantics are unclear).
    • Problems:
      • The "`ref`_(URL)" syntax forces the last word of the reference text to be joined to the URL, making a potentially very long word that can't be wrapped (URLs can be very long). The reference and the URL should be separate. This is a symptom of the following point:
      • The syntax produces a single compound construct made up of two equally important parts, with syntax in the middle, between the reference and the target. This is unprecedented in reStructuredText.
      • The "inline hyperlink" text is not a named reference (there's no lookup by name), so it shouldn't look like one.
      • According to the IETF standards RFC 2396 and RFC 2732, parentheses are legal URI characters and curly braces are legal email characters, making their use prohibitively difficult.
      • The named/anonymous semantics are unclear.

  2. After an analysis of the syntax of (1) above, we came up with the following compromise syntax:

    This is an anonymous reference__
__<http://www.example.org/reference/> of one word
("reference").  Here is a `phrase reference`__
__<http://www.example.org/phrase_reference/>.  `Named
references`_ _<http://www.example.org/anonymous/> use single
underscores.

    The syntax builds on that of the existing "inline internal targets": an _`inline internal target`.

    • Advantages:
      • The target is specified immediately adjacent to the reference, improving maintainability:
        • References and targets are easily kept in sync.
        • The reference text does not have to be repeated.
      • The construct is executed in two parts: references identical to existing references, and targets that are new but not too big a stretch from current syntax.
      • There's overwhelming precedent for quoting URLs with angle brackets 1.
    • Disadvantages:
      • Poor plaintext readability.
      • Lots of "line noise".
      • Targets cannot be reused (unless named; see below).

    To alleviate the readability issue slightly, we could allow the target to appear later, such as after the end of the sentence:

    This is a named reference__ of one word ("reference").
__<http://www.example.org/reference/>  Here is a `phrase
reference`__.  __<http://www.example.org/phrase_reference/>

    Problem: this could only work for one reference at a time (reference/target pairs must be proximate [refA trgA refB trgB], not interleaved [refA refB trgA trgB] or nested [refA refB trgB trgA]). This variation is too problematic; references and inline external targets will have to be kept imediately adjacent (see (3) below).

    The "reference__ __<target>" syntax is actually for "anonymous inline external targets", emphasized by the double underscores. It follows that single trailing and leading underscores would lead to implicitly named inline external targets. This would allow the reuse of targets by name. So after "reference_ _<target>", another "reference_" would point to the same target.

    [1]

    From RFC 2396 (URI syntax):

    The angle-bracket "<" and ">" and double-quote (") characters are excluded [from URIs] because they are often used as the delimiters around URI in text documents and protocol fields.

    Using <> angle brackets around each URI is especially recommended as a delimiting style for URI that contain whitespace.

    From RFC 822 (email headers):

    Angle brackets ("<" and ">") are generally used to indicate the presence of a one machine-usable reference (e.g., delimiting mailboxes), possibly including source-routing to the machine.

  3. If it is best for references and inline external targets to be immediately adjacent, then they might as well be integrated. Here's an alternative syntax embedding the target URL in the reference:

    This is an anonymous `reference <http://www.example.org
/reference/>`__ of one word ("reference").  Here is a `phrase
reference <http://www.example.org/phrase_reference/>`__.

    Advantages and disadvantages are similar to those in (2). Readability is still an issue, but the syntax is a bit less heavyweight (reduced line noise). Backquotes are required, even for one-word references; the target URL is included within the reference text, forcing a phrase context.

    We'll call this variant "embedded URIs".

    Problem: how to refer to a title like "HTML Anchors: <a>" (which ends with an HTML/SGML/XML tag)? We could either require more syntax on the target (like "`reference text __<http://example.com/>`__"), or require the odd conflicting title to be escaped (like "`HTML Anchors: \<a>`__"). The latter seems preferable, and not too onerous.

    Similarly to (2) above, a single trailing underscore would convert the reference & inline external target from anonymous to implicitly named, allowing reuse of targets by name.

    I think this is the least objectionable of the syntax alternatives.

Other syntax variations have been proposed (by Brett Cannon and Benja Fallenstein):

`phrase reference`->http://www.example.com

`phrase reference`@http://www.example.com

`phrase reference`__ ->http://www.example.com

`phrase reference` [-> http://www.example.com]

`phrase reference`__ [-> http://www.example.com]

`phrase reference` <http://www.example.com>_

None of these variations are clearly superior to #3 above. Some have problems that exclude their use.

With any kind of inline external target syntax it comes down to the conflict between maintainability and plaintext readability. I don't see a major problem with reStructuredText's maintainability, and I don't want to sacrifice plaintext readability to "improve" it.

The proponents of inline external targets want them for easily maintainable web pages. The arguments go something like this:

  • Named hyperlinks are difficult to maintain because the reference text is duplicated as the target name.

    To which I said, "So use anonymous hyperlinks."

  • Anonymous hyperlinks are difficult to maintain becuase the references and targets have to be kept in sync.

    "So keep the targets close to the references, grouped after each paragraph. Maintenance is trivial."

  • But targets grouped after paragraphs break the flow of text.

    "Surely less than URLs embedded in the text! And if the intent is to produce web pages, not readable plaintext, then who cares about the flow of text?"

Many participants have voiced their objections to the proposed syntax:

Garth Kidd: "I strongly prefer the current way of doing it. Inline is spectactularly messy, IMHO."

Tony Ibbs: "I vehemently agree... that the inline alternatives being suggested look messy - there are/were good reasons they've been taken out... I don't believe I would gain from the new syntaxes."

Paul Moore: "I agree as well. The proposed syntax is far too punctuation-heavy, and any of the alternatives discussed are ambiguous or too subtle."

Others have voiced their support:

fantasai: "I agree with Simon. In many cases, though certainly not in all, I find parenthesizing the url in plain text flows better than relegating it to a footnote."

Ken Manheimer: "I'd like to weigh in requesting some kind of easy, direct inline reference link."

(Interesting that those against the proposal have been using reStructuredText for a while, and those for the proposal are either new to the list ["fantasai", background unknown] or longtime StructuredText users [Ken Manheimer].)

I was initially ambivalent/against the proposed "inline external targets". I value reStructuredText's readability very highly, and although the proposed syntax offers convenience, I don't know if the convenience is worth the cost in ugliness. Does the proposed syntax compromise readability too much, or should the choice be left up to the author? Perhaps if the syntax is allowed but its use strongly discouraged, for aesthetic/readability reasons?

After a great deal of thought and much input from users, I've decided that there are reasonable use cases for this construct. The documentation should strongly caution against its use in most situations, recommending independent block-level targets instead. Syntax #3 above ("embedded URIs") will be used.

Doctree Representation of Transitions

(Although not reStructuredText-specific, this section fits best in this document.)

Having added the "horizontal rule" construct to the reStructuredText Markup Specification, a decision had to be made as to how to reflect the construct in the implementation of the document tree. Given this source:

Document
========

Paragraph 1

--------

Paragraph 2

The horizontal rule indicates a "transition" (in prose terms) or the start of a new "division". Before implementation, the parsed document tree would be:

<document>
    <section name="document">
        <title>
            Document
        <paragraph>
            Paragraph 1
        --------               <--- error here
        <paragraph>
            Paragraph 2

There are several possibilities for the implementation:

  1. Implement horizontal rules as "divisions" or segments. A "division" is a title-less, non-hierarchical section. The first try at an implementation looked like this:

    <document>
    <section name="document">
        <title>
            Document
        <paragraph>
            Paragraph 1
        <division>
            <paragraph>
                Paragraph 2

    But the two paragraphs are really at the same level; they shouldn't appear to be at different levels. There's really an invisible "first division". The horizontal rule splits the document body into two segments, which should be treated uniformly.

  2. Treating "divisions" uniformly brings us to the second possibility:

    <document>
    <section name="document">
        <title>
            Document
        <division>
            <paragraph>
                Paragraph 1
        <division>
            <paragraph>
                Paragraph 2

    With this change, documents and sections will directly contain divisions and sections, but not body elements. Only divisions will directly contain body elements. Even without a horizontal rule anywhere, the body elements of a document or section would be contained within a division element. This makes the document tree deeper. This is similar to the way HTML treats document contents: grouped within a <body> element.

  3. Implement them as "transitions", empty elements:

    <document>
    <section name="document">
        <title>
            Document
        <paragraph>
            Paragraph 1
        <transition>
        <paragraph>
            Paragraph 2

    A transition would be a "point element", not containing anything, only identifying a point within the document structure. This keeps the document tree flatter, but the idea of a "point element" like "transition" smells bad. A transition isn't a thing itself, it's the space between two divisions. However, transitions are a practical solution.

Solution 3 was chosen for incorporation into the document tree model.

Not Implemented

Reworking Footnotes

As a further wrinkle (see Reworking Explicit Markup (Round 1) above), in the wee hours of 2002-02-28 I posted several ideas for changes to footnote syntax:

  • Change footnote syntax from .. [1] to _[1]? ...
  • Differentiate (with new DTD elements) author-date "citations" ([GVR2002]) from numbered footnotes? ...
  • Render footnote references as superscripts without "[]"? ...

These ideas are all related, and suggest changes in the reStructuredText syntax as well as the docutils tree model.

The footnote has been used for both true footnotes (asides expanding on points or defining terms) and for citations (references to external works). Rather than dealing with one amalgam construct, we could separate the current footnote concept into strict footnotes and citations. Citations could be interpreted and treated differently from footnotes. Footnotes would be limited to numerical labels: manual ("1") and auto-numbered (anonymous "#", named "#label").

The footnote is the only explicit markup construct (starts with ".. ") that directly translates to a visible body element. I've always been a little bit uncomfortable with the ".. " marker for footnotes because of this; ".. " has a connotation of "special", but footnotes aren't especially "special". Printed texts often put footnotes at the bottom of the page where the reference occurs (thus "foot note"). Some HTML designs would leave footnotes to be rendered the same positions where they're defined. Other online and printed designs will gather footnotes into a section near the end of the document, converting them to "endnotes" (perhaps using a directive in our case); but this "special processing" is not an intrinsic property of the footnote itself, but a decision made by the document author or processing system.

Citations are almost invariably collected in a section at the end of a document or section. Citations "disappear" from where they are defined and are magically reinserted at some well-defined point. There's more of a connection to the "special" connotation of the ".. " syntax. The point at which the list of citations is inserted could be defined manually by a directive (e.g., ".. citations::"), and/or have default behavior (e.g., a section automatically inserted at the end of the document) that might be influenced by options to the Writer.

Syntax proposals:

  • Footnotes:

    • Current syntax:

      .. [1] Footnote 1
.. [#] Auto-numbered footnote.
.. [#label] Auto-labeled footnote.

    • The syntax proposed in the original 2002-02-28 Doc-SIG post: remove the ".. ", prefix a "_":

      _[1] Footnote 1
_[#] Auto-numbered footnote.
_[#label] Auto-labeled footnote.

      The leading underscore syntax (earlier dropped because .. _[1]: was too verbose) is a useful reminder that footnotes are hyperlink targets.

    • Minimal syntax: remove the ".. [" and "]", prefix a "_", and suffix a ".":

      _1. Footnote 1.
_#. Auto-numbered footnote.
_#label. Auto-labeled footnote.

         ``_1.``, ``_#.``, and ``_#label.`` are markers,
         like list markers.

      Footnotes could be rendered something like this in HTML

      | 1. This is a footnote. The brackets could be dropped | from the label, and a vertical bar could set them | off from the rest of the document in the HTML.

      Two-way hyperlinks on the footnote marker ("1." above) would also help to differentiate footnotes from enumerated lists.

      If converted to endnotes (by a directive/transform), a horizontal half-line might be used instead. Page-oriented output formats would typically use the horizontal line for true footnotes.

  • Footnote references:

    • Current syntax:

      [1]_, [#]_, [#label]_

    • Minimal syntax to match the minimal footnote syntax above:

      1_, #_, #label_

      As a consequence, pure-numeric hyperlink references would not be possible; they'd be interpreted as footnote references.

  • Citation references: no change is proposed from the current footnote reference syntax:

    [GVR2001]_

  • Citations:

    • Current syntax (footnote syntax):

      .. [GVR2001] Python Documentation; van Rossum, Drake, et al.;
   http://www.python.org/doc/

    • Possible new syntax:

      _[GVR2001] Python Documentation; van Rossum, Drake, et al.;
           http://www.python.org/doc/

_[DJG2002]
    Docutils: Python Documentation Utilities project; Goodger
    et al.; http://docutils.sourceforge.net/

      Without the ".. " marker, subsequent lines would either have to align as in one of the above, or we'd have to allow loose alignment (I'd rather not):

      _[GVR2001] Python Documentation; van Rossum, Drake, et al.;
    http://www.python.org/doc/

I proposed adopting the "minimal" syntax for footnotes and footnote references, and adding citations and citation references to reStructuredText's repertoire. The current footnote syntax for citations is better than the alternatives given.

From a reply by Tony Ibbs on 2002-03-01:

However, I think easier with examples, so let's create one:

Fans of Terry Pratchett are perhaps more likely to use
footnotes [1]_ in their own writings than other people
[2]_.  Of course, in *general*, one only sees footnotes
in academic or technical writing - it's use in fiction
and letter writing is not normally considered good
style [4]_, particularly in emails (not a medium that
lends itself to footnotes).

.. [1] That is, little bits of referenced text at the
   bottom of the page.
.. [2] Because Terry himself does, of course [3]_.
.. [3] Although he has the distinction of being
   *funny* when he does it, and his fans don't always
   achieve that aim.
.. [4] Presumably because it detracts from linear
   reading of the text - this is, of course, the point.

and look at it with the second syntax proposal:

Fans of Terry Pratchett are perhaps more likely to use
footnotes [1]_ in their own writings than other people
[2]_.  Of course, in *general*, one only sees footnotes
in academic or technical writing - it's use in fiction
and letter writing is not normally considered good
style [4]_, particularly in emails (not a medium that
lends itself to footnotes).

_[1] That is, little bits of referenced text at the
     bottom of the page.
_[2] Because Terry himself does, of course [3]_.
_[3] Although he has the distinction of being
     *funny* when he does it, and his fans don't always
     achieve that aim.
_[4] Presumably because it detracts from linear
     reading of the text - this is, of course, the point.

(I note here that if I have gotten the indentation of the footnotes themselves correct, this is clearly not as nice. And if the indentation should be to the left margin instead, I like that even less).

and the third (new) proposal:

Fans of Terry Pratchett are perhaps more likely to use
footnotes 1_ in their own writings than other people
2_.  Of course, in *general*, one only sees footnotes
in academic or technical writing - it's use in fiction
and letter writing is not normally considered good
style 4_, particularly in emails (not a medium that
lends itself to footnotes).

_1. That is, little bits of referenced text at the
    bottom of the page.
_2. Because Terry himself does, of course 3_.
_3. Although he has the distinction of being
    *funny* when he does it, and his fans don't always
    achieve that aim.
_4. Presumably because it detracts from linear
    reading of the text - this is, of course, the point.

I think I don't, in practice, mind the targets too much (the use of a dot after the number helps a lot here), but I do have a problem with the body text, in that I don't naturally separate out the footnotes as different than the rest of the text - instead I keep wondering why there are numbers interspered in the text. The use of brackets around the numbers ([ and ]) made me somehow parse the footnote references as "odd" - i.e., not part of the body text - and thus both easier to skip, and also (paradoxically) easier to pick out so that I could follow them.

Thus, for the moment (and as always susceptable to argument), I'd say -1 on the new form of footnote reference (i.e., I much prefer the existing [1]_ over the proposed 1_), and ambivalent over the proposed target change.

That leaves David's problem of wanting to distinguish footnotes and citations - and the only thing I can propose there is that footnotes are numeric or # and citations are not (which, as a human being, I can probably cope with!).

From a reply by Paul Moore on 2002-03-01:

I think the current footnote syntax [1]_ is exactly the right balance of distinctness vs unobtrusiveness. I very definitely don't think this should change.

On the target change, it doesn't matter much to me.

From a further reply by Tony Ibbs on 2002-03-01, referring to the "[1]" form and actual usage in email:

Clearly this is a form people are used to, and thus we should consider it strongly (in the same way that the usage of *..* to mean emphasis was taken partly from email practise).

Equally clearly, there is something "magical" for people in the use of a similar form (i.e., [1]) for both footnote reference and footnote target - it seems natural to keep them similar.

...

I think that this established plaintext usage leads me to strongly believe we should retain square brackets at both ends of a footnote. The markup of the reference end (a single trailing underscore) seems about as minimal as we can get away with. The markup of the target end depends on how one envisages the thing - if ".." means "I am a target" (as I tend to see it), then that's good, but one can also argue that the "_[1]" syntax has a neat symmetry with the footnote reference itself, if one wishes (in which case ".." presumably means "hidden/special" as David seems to think, which is why one needs a ".." and a leading underline for hyperlink targets.

Given the persuading arguments voiced, we'll leave footnote & footnote reference syntax alone. Except that these discussions gave rise to the "auto-symbol footnote" concept, which has been added. Citations and citation references have also been added.

Tabled

Reworking Explicit Markup (Round 2)

See Reworking Explicit Markup (Round 1) for an earlier discussion.

In April 2004, a new thread becan on docutils-develop: Inconsistency in RST markup. Several arguments were made; the first argument begat later arguments. Below, the arguments are paraphrased "in quotes", with responses.

  1. References and targets take this form:

    targetname_

.. _targetname: stuff

    But footnotes, "which generate links just like targets do", are written as:

    [1]_

.. [1] stuff

    "Footnotes should be written as":

    [1]_

.. _[1]: stuff

    But they're not the same type of animal. That's not a "footnote target", it's a footnote. Being a target is not a footnote's primary purpose (an arguable point). It just happens to grow a target automatically, for convenience. Just as a section title:

    Title
=====

    isn't a "title target", it's a title, which happens to grow a target automatically. The consistency is there, it's just deeper than at first glance.

    Also, ".. [1]" was chosen for footnote syntax because it closely resembles one form of actual footnote rendering. ".. _[1]:" is too verbose; excessive punctuation is required to get the job done.

    For more of the reasoning behind the syntax, see Problems With StructuredText (Hyperlinks) and Reworking Footnotes.

  2. "I expect directives to also look like .. this: [one colon] because that also closely parallels the link and footnote target markup."

    There are good reasons for the two-colon syntax:

    Two colons are used after the directive type for these reasons:

    • Two colons are distinctive, and unlikely to be used in common text.

    • Two colons avoids clashes with common comment text like:

      .. Danger: modify at your own risk!

    • If an implementation of reStructuredText does not recognize a directive (i.e., the directive-handler is not installed), a level-3 (error) system message is generated, and the entire directive block (including the directive itself) will be included as a literal block. Thus "::" is a natural choice.

    —restructuredtext.html#directives

    The last reason is not particularly compelling; it's more of a convenient coincidence or mnemonic.

  3. "Comments always seemed too easy. I almost never write comments. I'd have no problem writing '.. comment:' in front of my comments. In fact, it would probably be more readable, as comments should be set off strongly, because they are very different from normal text."

    Many people do use comments though, and some applications of reStructuredText require it. For example, all reStructuredText PEPs (and this document!) have an Emacs stanza at the bottom, in a comment. Having to write ".. comment::" would be very obtrusive.

    Comments should be dirt-easy to do. It should be easy to "comment out" a block of text. Comments in programming languages and other markup languages are invariably easy.

    Any author is welcome to preface their comments with "Comment:" or "Do Not Print" or "Note to Editor" or anything they like. A "comment" directive could easily be implemented. It might be confused with admonition directives, like "note" and "caution" though. In unrelated (and unpublished and unfinished) work, adding a "comment" directive as a true document element was considered:

    If structure is necessary, we could use a "comment" directive
(to avoid nonsensical DTD changes, the "comment" directive
could produce an untitled topic element).

  4. "One of the goals of reStructuredText is to be readable by people who don't know it. This construction violates that: it is not at all obvious to the uninitiated that text marked by '..' is a comment. On the other hand, '.. comment:' would be totally transparent."

    Totally transparent, perhaps, but also very obtrusive. Another of reStructuredText's goals is to be unobtrusive, and ".. comment::" would violate that. The goals of reStructuredText are many, and they conflict. Determining the right set of goals and finding solutions that best fit is done on a case-by-case basis.

    Even readability is has two aspects. Being readable without any prior knowledge is one. Being as easily read in raw form as in processed form is the other. ".." may not contribute to the former aspect, but ".. comment::" would certainly detract from the latter.

  5. "Recently I sent someone an rst document, and they got confused; I had to explain to them that '..' marks comments, unless it's a directive, etc..."

    The explanation of directives is roundabout, defining comments in terms of not being other things. That's definitely a wart.

  6. "Under the current system, a mistyped directive (with ':' instead of '::') will be silently ignored. This is an error that could easily go unnoticed."

    A parser option/setting like "--comments-on-stderr" would help.

  7. "I'd prefer to see double-dot-space / command / double-colon as the standard Docutils markup-marker. It's unusual enough to avoid being accidently used. Everything that starts with a double-dot should end with a double-colon."

    That would increase the punctuation verbosity of some constructs considerably.

  8. Edward Loper proposed the following plan for backwards compatibility:

    1. ".. foo" will generate a deprecation warning to stderr, and nothing in the output (no system messages).
    2. ".. foo: bar" will be treated as a directive foo. If there is no foo directive, then do the normal error output.
    3. ".. foo:: bar" will generate a deprecation warning to stderr, and be treated as a directive. Or leave it valid?

    So some existing documents might start printing deprecation warnings, but the only existing documents that would break would be ones that say something like:

    .. warning: this should be a comment

    instead of:

    .. warning:: this should be a comment

    Here, we're trading fairly common a silent error (directive falsely treated as a comment) for a fairly uncommon explicitly flagged error (comment falsely treated as directive). To make things even easier, we could add a sentence to the unknown-directive error. Something like "If you intended to create a comment, please use '.. comment:' instead".

On one hand, I understand and sympathize with the points raised. On the other hand, I think the current syntax strikes the right balance (but I acknowledge a possible lack of objectivity). On the gripping hand, the comment and directive syntax has become well established, so even if it's a wart, it may be a wart we have to live with.

Making any of these changes would cause a lot of breakage or at least deprecation warnings. I'm not sure the benefit is worth the cost.

For now, we'll treat this as an unresolved legacy issue.

To Do

Auto-Enumerated Lists

The advantage of auto-numbered enumerated lists would be similar to that of auto-numbered footnotes: lists could be written and rearranged without having to manually renumber them. The disadvantages are also the same: input and output wouldn't match exactly; the markup may be ugly or confusing (depending on which alternative is chosen).

  1. Use the "#" symbol. Example:

    #. Item 1.
#. Item 2.
#. Item 3.

    Advantages: simple, explicit. Disadvantage: enumeration sequence cannot be specified (limited to arabic numerals); ugly.

  2. As a variation on #1, first initialize the enumeration sequence? For example:

    a) Item a.
#) Item b.
#) Item c.

    Advantages: simple, explicit, any enumeration sequence possible. Disadvantages: ugly; perhaps confusing with mixed concrete/abstract enumerators.

  3. Alternative suggested by Fred Bremmer, from experience with MoinMoin:

    1. Item 1.
1. Item 2.
1. Item 3.

    Advantages: enumeration sequence is explicit (could be multiple "a." or "(I)" tokens). Disadvantages: perhaps confusing; otherwise erroneous input (e.g., a duplicate item "1.") would pass silently, either causing a problem later in the list (if no blank lines between items) or creating two lists (with blanks).

    Take this input for example:

    1. Item 1.

1. Unintentional duplicate of item 1.

2. Item 2.

    Currently the parser will produce two list, "1" and "1,2" (no warnings, because of the presence of blank lines). Using Fred's notation, the current behavior is "1,1,2 -> 1 1,2" (without blank lines between items, it would be "1,1,2 -> 1 [WARNING] 1,2"). What should the behavior be with auto-numbering?

    Fred has produced a patch, whose initial behavior is as follows:

    1,1,1   -> 1,2,3
1,2,2   -> 1,2,3
3,3,3   -> 3,4,5
1,2,2,3 -> 1,2,3 [WARNING] 3
1,1,2   -> 1,2 [WARNING] 2

    (After the "[WARNING]", the "3" would begin a new list.)

    I have mixed feelings about adding this functionality to the spec & parser. It would certainly be useful to some users (myself included; I often have to renumber lists). Perhaps it's too clever, asking the parser to guess too much. What if you do want three one-item lists in a row, each beginning with "1."? You'd have to use empty comments to force breaks. Also, I question whether "1,2,2 -> 1,2,3" is optimal behavior.

    In response, Fred came up with "a stricter and more explicit rule [which] would be to only auto-number silently if all the enumerators of a list were identical". In that case:

    1,1,1   -> 1,2,3
1,2,2   -> 1,2 [WARNING] 2
3,3,3   -> 3,4,5
1,2,2,3 -> 1,2 [WARNING] 2,3
1,1,2   -> 1,2 [WARNING] 2

    Should any start-value be allowed ("3,3,3"), or should auto-numbered lists be limited to begin with ordinal-1 ("1", "A", "a", "I", or "i")?

  4. Alternative proposed by Tony Ibbs:

    #1. First item.
#3. Aha - I edited this in later.
#2. Second item.

    The initial proposal required unique enumerators within a list, but this limits the convenience of a feature of already limited applicability and convenience. Not a useful requirement; dropped.

    Instead, simply prepend a "#" to a standard list enumerator to indicate auto-enumeration. The numbers (or letters) of the enumerators themselves are not significant, except:

    • as a sequence indicator (arabic, roman, alphabetic; upper/lower),
    • and perhaps as a start value (first list item).

    Advantages: explicit, any enumeration sequence possible. Disadvantages: a bit ugly.

Nested Inline Markup

These are collected notes on a long-discussed issue. The original mailing list messages should be referred to for details.

  • In a 2001-10-31 discussion I wrote:

    Try, for example, Ed Loper's 2001-03-21 post, which details some rules for nested inline markup. I think the complexity is prohibitive for the marginal benefit. (And if you can understand that tree without going mad, you're a better man than I. ;-)

    Inline markup is already fragile. Allowing nested inline markup would only be asking for trouble IMHO. If it proves absolutely necessary, it can be added later. The rules for what can appear inside what must be well thought out first though.

    —http://mail.python.org/pipermail/doc-sig/2001-October/002354.html

  • In a 2001-11-09 Doc-SIG post, I wrote:

    The problem is that in the what-you-see-is-more-or-less-what-you-get markup language that is reStructuredText, the symbols used for inline markup ("*", "**", "`", "``", etc.) may preclude nesting.

    I've rethought this position. Nested markup is not precluded, just tricky. People and software parse "double and 'single' quotes" all the time. Continuing,

    I've thought over how we might implement nested inline markup. The first algorithm ("first identify the outer inline markup as we do now, then recursively scan for nested inline markup") won't work; counterexamples were given in my last post.

    The second algorithm makes my head hurt:

    while 1:
    scan for start-string
    if found:
        push on stack
        scan for start or end string
        if new start string found:
            recurse
        elif matching end string found:
            pop stack
        elif non-matching end string found:
            if its a markup error:
                generate warning
            elif the initial start-string was misinterpreted:
                # e.g. in this case: ***strong** in emphasis*
                restart with the other interpretation
                # but it might be several layers back ...
    ...

    This is similar to how the parser does section title recognition, but sections are much more regular and deterministic.

    Bottom line is, I don't think the benefits are worth the effort, even if it is possible. I'm not going to try to write the code, at least not now. If somebody codes up a consistent, working, general solution, I'll be happy to consider it.

    —http://mail.python.org/pipermail/doc-sig/2001-November/002388.html

  • In a 2003-05-06 Docutils-Users post Paul Tremblay proposed a new syntax to allow for easier nesting. It eventually evolved into this:

    :role:[inline text]

    The duplication with the existing interpreted text syntax is problematic though.

  • Could the parser be extended to parse nested interpreted text?

    :emphasis:`Some emphasized text with :strong:`some more
emphasized text` in it and **perhaps** :reference:`a link``

  • In a 2003-06-18 Docutils-Develop post, Mark Nodine reported on his implementation of a form of nested inline markup in his Perl-based parser (unpublished). He brought up some interesting ideas. The implementation was flawed, however, by the change in semantics required for backslash escapes.

  • Docutils-develop threads between David Abrahams, David Goodger, and Mark Nodine (beginning 2004-01-16 and 2004-01-19) hashed out many of the details of a potentially successful implementation, as described below. David Abrahams checked in code to the "nesting" branch of CVS, awaiting thorough review.

It may be possible to accomplish nested inline markup in general with a more powerful inline markup parser. There may be some issues, but I'm not averse to the idea of nested inline markup in general. I just don't have the time or inclination to write a new parser now. Of course, a good patch would be welcome!

I envisage something like this. Explicit-role interpreted text must be nestable. Prefix-based is probably preferred, since suffix-based will look like inline literals:

``text`:role1:`:role2:

But it can be disambiguated, so it ought to be left up to the author:

`\ `text`:role1:`:role2:

In addition, other forms of inline markup may be nested if unambiguous:

*emphasized ``literal`` and |substitution ref| and link_*

IOW, the parser ought to be as permissive as possible.

Syntax for Line Blocks

  • An early idea: How about a literal-block-like prefix, perhaps ";;"? (It is, after all, a semi-literal literal block, no?) Example:

    Take it away, Eric the Orchestra Leader!  ;;

    A one, two, a one two three four

    Half a bee, philosophically,
    must, *ipso facto*, half not be.
    But half the bee has got to be,
    *vis a vis* its entity.  D'you see?

    But can a bee be said to be
    or not to be an entire bee,
    when half the bee is not a bee,
    due to some ancient injury?

    Singing...

    Kinda lame.

  • Another idea: in an ordinary paragraph, if the first line ends with a backslash (escaping the newline), interpret the entire paragraph as a verse block? For example:

    Add just one backslash\
And this paragraph becomes
An awful haiku

    (Awful, and arguably invalid, since in Japanese the word "haiku" contains three syllables not two.)

    This idea was superceded by the rules for escaped whitespace, useful for character-level inline markup.

  • In a 2004-02-22 docutils-develop message, Jarno Elonen proposed a "plain list" syntax (and also provided a patch):

    | John Doe
| President, SuperDuper Corp.
| jdoe@example.org

    This syntax is very natural. However, these "plain lists" seem very similar to line blocks, and I see so little intrinsic "list-ness" that I'm loathe to add a new object. I used the term "blurbs" to remove the "list" connotation from the originally proposed name. Perhaps line blocks could be refined to add the two properties they currently lack:

    1. long lines wrap nicely
    2. HTML output doesn't look like program code in non-CSS web browsers

    (A) is an issue of all 3 aspects of Docutils: syntax (construct behaviour), internal representation, and output. (B) is partly an issue of internal representation but mostly of output.

ReStructuredText will redefine line blocks with the "|"-quoting syntax. The following is my current thinking.

Syntax

Perhaps line block syntax like this would do:

| M6: James Bond
| MIB: Mr. J.
| IMF: not decided yet, but probably one of the following:
|   Ethan Hunt
|   Jim Phelps
|   Claire Phelps
| CIA: Felix Leiter

Note that the "nested" list does not have nested syntax (the "|" are not further indented); the leading whitespace would still be significant somehow (more below). As for long lines in the input, this could suffice:

| John Doe
| Founder, President, Chief Executive Officer, Cook, Bottle
  Washer, and All-Round Great Guy
| SuperDuper Corp.
| jdoe@example.org

The lack of "|" on the third line indicates that it's a continuation of the second line, wrapped.

I don't see much point in allowing arbitrary nested content. Multiple paragraphs or bullet lists inside a "blurb" doesn't make sense to me. Simple nested line blocks should suffice.

Internal Representation

Line blocks are currently represented as text blobs as follows:

<!ELEMENT line_block %text.model;>
<!ATTLIST line_block
    %basic.atts;
    %fixedspace.att;>

Instead, we could represent each line by a separate element:

<!ELEMENT line_block (line+)>
<!ATTLIST line_block %basic.atts;>

<!ELEMENT line %text.model;>
<!ATTLIST line %basic.atts;>

We'd keep the significance of the leading whitespace of each line either by converting it to non-breaking spaces at output, or with a per-line margin. Non-breaking spaces are simpler (for HTML, anyway) but kludgey, and wouldn't support indented long lines that wrap. But should inter-word whitespace (i.e., not leading whitespace) be preserved? Currently it is preserved in line blocks.

Representing a more complex line block may be tricky:

| But can a bee be said to be
|     or not to be an entire bee,
|         when half the bee is not a bee,
|             due to some ancient injury?

Perhaps the representation could allow for nested line blocks:

<!ELEMENT line_block (line | line_block)+>

With this model, leading whitespace would no longer be significant. Instead, left margins are implied by the nesting. The example above could be represented as follows:

<line_block>
    <line>
        But can a bee be said to be
    <line_block>
        <line>
             or not to be an entire bee,
        <line_block>
            <line>
                when half the bee is not a bee,
            <line_block>
                <line>
                    due to some ancient injury?

I wasn't sure what to do about even more complex line blocks:

|     Indented
| Not indented
|   Indented a bit
|     A bit more
|  Only one space

How should that be parsed and nested? Should the first line have the same nesting level (== indentation in the output) as the fourth line, or the same as the last line? Mark Nodine suggested that such line blocks be parsed similarly to complexly-nested block quotes, which seems reasonable. In the example above, this would result in the nesting of first line matching the last line's nesting. In other words, the nesting would be relative to neighboring lines only.

Output

In HTML, line blocks are currently output as "<pre>" blocks, which gives us significant whitespace and line breaks, but doesn't allow long lines to wrap and causes monospaced output without stylesheets. Instead, we could output "<div>" elements parallelling the representation above, where each nested <div class="line_block"> would have an increased left margin (specified in the stylesheet).

Jarno suggested the following HTML output:

<div class="line_block">
   <span class="line">First, top level line</span><br class="hidden"/>
   <div class="line_block"><span class="hidden">&nbsp;</span>
      <span class="line">Second, once nested</span><br class="hidden"/>
      <span class="line">Third, once nested</span><br class="hidden"/>
      ...
   </div>
   ...
</div>

The <br class="hidden" /> and <span class="hidden">&nbsp;</span> are meant to support non-CSS and non-graphical browsers. I understand the case for "br", but I'm not so sure about hidden "&nbsp;". I question how much effort should be put toward supporting non-graphical and especially non-CSS browsers, at least for html4css1.py output.

Should the lines themselves be <span> or <div>? I don't like mixing inline and block-level elements.

Implementation Plan

We'll leave the old implementation in place (via the "line-block" directive only) until all Writers have been updated to support the new syntax & implementation. The "line-block" directive can then be updated to use the new internal representation, and its documentation will be updated to recommend the new syntax.

List-Driven Tables

The original idea came from Dylan Jay:

... to use a two level bulleted list with something to indicate it should be rendered as a table ...

It's an interesting idea. It could be implemented in as a directive which transforms a uniform two-level list into a table. Using a directive would allow the author to explicitly set the table's orientation (by column or by row), the presence of row headers, etc.

Alternatives:

  1. Bullet-list-tables might look like this:

    .. list-table::

   * - Treat
     - Quantity
     - Description
   * - Albatross!
     - 299
     - On a stick!
   * - Crunchy Frog!
     - 1499
     - If we took the bones out, it wouldn't be crunchy,
       now would it?
   * - Gannet Ripple!
     - 199
     - On a stick!

    This list must be written in two levels. This wouldn't work:

    .. list-table::

   * Treat
   * Albatross!
   * Gannet!
   * Crunchy Frog!

   * Quantity
   * 299
   * 199
   * 1499

   * Description
   * On a stick!
   * On a stick!
   * If we took the bones out...

    The above is a single list of 12 items. The blank lines are not significant to the markup. We'd have to explicitly specify how many columns or rows to use, which isn't a good idea.

  2. Beni Cherniavsky suggested a field list alternative. It could look like this:

    .. field-list-table::
   :headrows: 1

   - :treat: Treat
     :quantity: Quantity
     :descr: Description

   - :treat: Albatross!
     :quantity: 299
     :descr: On a stick!

   - :treat: Crunchy Frog!
     :quantity: 1499
     :descr: If we took the bones out, it wouldn't be
             crunchy, now would it?

    Column order is determined from the order of fields in the first row. Field order in all other rows is ignored. As a side-effect, this allows trivial re-arrangement of columns. By using named fields, it becomes possible to omit fields in some rows without losing track of things, which is important for spans.

  3. An alternative to two-level bullet lists would be to use enumerated lists for the table cells:

    .. list-table::

    * 1. Treat
      2. Quantity
      3. Description
    * 1. Albatross!
      2. 299
      3. On a stick!
    * 1. Crunchy Frog!
      2. 1499
      3. If we took the bones out, it wouldn't be crunchy,
         now would it?

    That provides better correspondence between cells in the same column than does bullet-list syntax, but not as good as field list syntax. I think that were only field-list-tables available, a lot of users would use the equivalent degenerate case:

    .. field-list-table::
    - :1: Treat
      :2: Quantity
      :3: Description
    ...

  4. Another natural variant is to allow a description list with field lists as descriptions:

    .. list-table::
    :headrows: 1

    Treat
        :quantity: Quantity
        :descr: Description
    Albatross!
        :quantity: 299
        :descr: On a stick!
    Crunchy Frog!
        :quantity: 1499
        :descr: If we took the bones out, it wouldn't be
                crunchy, now would it?

    This would make the whole first column a header column ("stub"). It's limited to a single column and a single paragraph fitting on one source line. Also it wouldn't allow for empty cells or row spans in the first column. But these are limitations that we could live with, like those of simple tables.

The List-driven table feature could be done in many ways. Each user will have their preferred usage. Perhaps a single "list-table" directive could handle them all, depending on which options and content are present.

Issues:

  • How to indicate that there's 1 header row? Perhaps two lists?

    .. list-table::

   + - Treat
     - Quantity
     - Description

   * - Albatross!
     - 299
     - On a stick!

    This is probably too subtle though. Better would be a directive option, like :headrows: 1. An early suggestion for the header row(s) was to use a directive option:

    .. field-list-table::
   :header:
       - :treat: Treat
         :quantity: Quantity
         :descr: Description
   - :treat: Albatross!
     :quantity: 299
     :descr: On a stick!

    But the table data is at two levels and looks inconsistent.

    In general, we cannot extract the header row from field lists' field names because field names cannot contain everything one might put in a table cell. A separate header row also allows shorter field names and doesn't force one to rewrite the whole table when the header text changes. But for simpler cases, we can offer a ":header: fields" option, which does extract header cells from field names:

    .. field-list-table::
    :header: fields

    - :Treat: Albatross!
      :Quantity: 299
      :Description: On a stick!

  • How to indicate the column widths? A directive option?

    .. list-table::
   :widths: 15 10 35

    Automatic defaults from the text used?

  • How to handle row and/or column spans?

    In a field list, column-spans can be indicated by specifying the first and last fields, separated by space-dash-space or ellipsis:

    - :foo - baz: quuux
- :foo ... baz: quuux

    Commas were proposed for column spans:

    - :foo, bar: quux

    But non-adjacent columns become problematic. Should we report an error, or duplicate the value into each span of adjacent columns (as was suggested)? The latter suggestion is appealing but may be too clever. Best perhaps to simply specify the two ends.

    It was suggested that comma syntax should be allowed, too, in order to allow the user to avoid trouble when changing the column order. But changing the column order of a table with spans is not trivial; we shouldn't make it easier to mess up.

    One possible syntax for row-spans is to simply treat any row where a field is missing as a row-span from the last row where it appeared. Leaving a field empty would still be possible by writing a field with empty content. But this is too implicit.

    Another way would be to require an explicit continuation marker (.../-"-/"?) in all but the first row of a spanned field. Empty comments could work (".."). If implemented, the same marker could also be supported in simple tables, which lack row-spanning abilities.

    Explicit markup like ":rowspan:" and ":colspan:" was also suggested.

    Sometimes in a table, the first header row contains spans. It may be necessary to provide a way to specify the column field names independently of data rows. A directive option would do it.

  • We could specify "column-wise" or "row-wise" ordering, with the same markup structure. For example, with definition data:

    .. list-table::
   :column-wise:

   Treat
       - Albatross!
       - Crunchy Frog!
   Quantity
       - 299
       - 1499
   Description
       - On a stick!
       - If we took the bones out, it wouldn't be
         crunchy, now would it?

  • A syntax for stubs in ASCII-art tables is easy to imagine:

    +------------------------++------------+----------+
| Header row, column 1   || Header 2   | Header 3 |
+========================++============+==========+
| body row 1, column 1   || column 2   | column 3 |
+------------------------++------------+----------+

... Or Not To Do?

This is the realm of the possible but questionably probable. These ideas are kept here as a record of what has been proposed, for posterity and in case any of them prove to be useful.

Compound Enumerated Lists

Allow for compound enumerators, such as "1.1." or "1.a." or "1(a)", to allow for nested enumerated lists without indentation?

Sloppy Indentation of List Items

Perhaps the indentation shouldn't be so strict. Currently, this is required:

1. First line,
   second line.

Anything wrong with this?

1. First line,
 second line.

Problem?

1. First para.

   Block quote.  (no good: requires some indent relative to first
   para)

 Second Para.

2. Have to carefully define where the literal block ends::

     Literal block

   Literal block?

Hmm... Non-strict indentation isn't such a good idea.

Lazy Indentation of List Items

Another approach: Going back to the first draft of reStructuredText (2000-11-27 post to Doc-SIG):

- This is the fourth item of the main list (no blank line above).
The second line of this item is not indented relative to the
bullet, which precludes it from having a second paragraph.

Change that to require a blank line above and below, to reduce ambiguity. This "loosening" may be added later, once the parser's been nailed down. However, a serious drawback of this approach is to limit the content of each list item to a single paragraph.

David's Idea for Lazy Indentation

Consider a paragraph in a word processor. It is a single logical line of text which ends with a newline, soft-wrapped arbitrarily at the right edge of the page or screen. We can think of a plaintext paragraph in the same way, as a single logical line of text, ending with two newlines (a blank line) instead of one, and which may contain arbitrary line breaks (newlines) where it was accidentally hard-wrapped by an application. We can compensate for the accidental hard-wrapping by "unwrapping" every unindented second and subsequent line. The indentation of the first line of a paragraph or list item would determine the indentation for the entire element. Blank lines would be required between list items when using lazy indentation.

The following example shows the lazy indentation of multiple body elements:

- This is the first paragraph
of the first list item.

  Here is the second paragraph
of the first list item.

- This is the first paragraph
of the second list item.

  Here is the second paragraph
of the second list item.

A more complex example shows the limitations of lazy indentation:

- This is the first paragraph
of the first list item.

  Next is a definition list item:

  Term
      Definition.  The indentation of the term is
required, as is the indentation of the definition's
first line.

      When the definition extends to more than
one line, lazy indentation may occur.  (This is the second
paragraph of the definition.)

- This is the first paragraph
of the second list item.

  - Here is the first paragraph of
the first item of a nested list.

  So this paragraph would be outside of the nested list,
but inside the second list item of the outer list.

But this paragraph is not part of the list at all.

And the ambiguity remains:

- Look at the hyphen at the beginning of the next line
- is it a second list item marker, or a dash in the text?

Similarly, we may want to refer to numbers inside enumerated
lists:

1. How many socks in a pair? There are
2. How many pants in a pair? Exactly
1. Go figure.

Literal blocks and block quotes would still require consistent indentation for all their lines. For block quotes, we might be able to get away with only requiring that the first line of each contained element be indented. For example:

Here's a paragraph.

    This is a paragraph inside a block quote.
Second and subsequent lines need not be indented at all.

    - A bullet list inside
the block quote.

      Second paragraph of the
bullet list inside the block quote.

Although feasible, this form of lazy indentation has problems. The document structure and hierarchy is not obvious from the indentation, making the source plaintext difficult to read. This will also make keeping track of the indentation while writing difficult and error-prone. However, these problems may be acceptable for Wikis and email mode, where we may be able to rely on less complex structure (few nested lists, for example).

Multiple Roles in Interpreted Text

In reStructuredText, inline markup cannot be nested (yet; see above). This also applies to interpreted text. In order to simultaneously combine multiple roles for a single piece of text, a syntax extension would be necessary. Ideas:

  1. Initial idea:

    `interpreted text`:role1,role2:

  2. Suggested by Jason Diamond:

    `interpreted text`:role1:role2:

If a document is so complex as to require nested inline markup, perhaps another markup system should be considered. By design, reStructuredText does not have the flexibility of XML.

Parameterized Interpreted Text

In some cases it may be expedient to pass parameters to interpreted text, analogous to function calls. Ideas:

  1. Parameterize the interpreted text role itself (suggested by Jason Diamond):

    `interpreted text`:role1(foo=bar):

    Positional parameters could also be supported:

    `CSS`:acronym(Cascading Style Sheets): is used for HTML, and
`CSS`:acronym(Content Scrambling System): is used for DVDs.

    Technical problem: current interpreted text syntax does not recognize roles containing whitespace. Design problem: this smells like programming language syntax, but reStructuredText is not a programming language.

  2. Put the parameters inside the interpreted text:

    `CSS (Cascading Style Sheets)`:acronym: is used for HTML, and
`CSS (Content Scrambling System)`:acronym: is used for DVDs.

    Although this could be defined on an individual basis (per role), we ought to have a standard. Hyperlinks with embedded URIs already use angle brackets; perhaps they could be used here too:

    `CSS <Cascading Style Sheets>`:acronym: is used for HTML, and
`CSS <Content Scrambling System>`:acronym: is used for DVDs.

    Do angle brackets connote URLs too much for this to be acceptable? How about the "tag" connotation -- does it save them or doom them?

Does this push inline markup too far? Readability becomes a serious issue. Substitutions may provide a better alternative (at the expense of verbosity and duplication) by pulling the details out of the text flow:

|CSS| is used for HTML, and |CSS-DVD| is used for DVDs.

.. |CSS| acronym:: Cascading Style Sheets
.. |CSS-DVD| acronym:: Content Scrambling System
   :text: CSS

This whole idea may be going beyond the scope of reStructuredText. Documents requiring this functionality may be better off using XML or another markup system.

This argument comes up regularly when pushing the envelope of reStructuredText syntax. I think it's a useful argument in that it provides a check on creeping featurism. In many cases, the resulting verbosity produces such unreadable plaintext that there's a natural desire not to use it unless absolutely necessary. It's a matter of finding the right balance.

Syntax for Interpreted Text Role Bindings

The following syntax (idea from Jeffrey C. Jacobs) could be used to associate directives with roles:

.. :rewrite: class:: rewrite

`She wore ribbons in her hair and it lay with streaks of
grey`:rewrite:

The syntax is similar to that of substitution declarations, and the directive/role association may resolve implementation issues. The semantics, ramifications, and implementation details would need to be worked out.

The example above would implement the "rewrite" role as adding a class="rewrite" attribute to the interpreted text ("inline" element). The stylesheet would then pick up on the "class" attribute to do the actual formatting.

The advantage of the new syntax would be flexibility. Uses other than "class" may present themselves. The disadvantage is complexity: having to implement new syntax for a relatively specialized operation, and having new semantics in existing directives ("class::" would do something different).

The "role" directive has been implemented.

Character Processing

Several people have suggested adding some form of character processing to reStructuredText:

  • Some sort of automated replacement of ASCII sequences:
    • -- to em-dash (or -- to en-dash, and --- to em-dash).
    • Convert quotes to curly quote entities. (Essentially impossible for HTML? Unnecessary for TeX.)
    • Various forms of :-) to smiley icons.
    • "\ " to &nbsp;. Problem with line-wrapping though: it could end up escaping the newline.
    • Escaped newlines to <BR>.
    • Escaped period or quote or dash as a disappearing catalyst to allow character-level inline markup?
  • XML-style character entities, such as "&copy;" for the copyright symbol.

Docutils has no need of a character entity subsystem. Supporting Unicode and text encodings, character entities should be directly represented in the text: a copyright symbol should be represented by the copyright symbol character. If this is not possible in an authoring environment, a pre-processing stage can be added, or a table of substitution definitions can be devised.

A "unicode" directive has been implemented to allow direct specification of esoteric characters. In combination with the substitution construct, "include" files defining common sets of character entities can be defined and used. A set of character entity set definition files have been defined (tarball). There's also a description and instructions for use.

To allow for character-level inline markup, a limited form of character processing has been added to the spec and parser: escaped whitespace characters are removed from the processed document. Any further character processing will be of this functional type, rather than of the character-encoding type.

  • Directive idea:

    .. text-replace:: "pattern" "replacement"
    • Support Unicode "U+XXXX" codes.
    • Support regexps, perhaps with alternative "regexp-replace" directive.
    • Flags for regexps; ":flags:" option, or individuals.
    • Specifically, should the default be case-sensistive or -insensitive?
View document source. Generated on: 2004-07-01 22:44 UTC. Generated by Docutils from reStructuredText source.


Problems With StructuredText

Author: | David Goodger
Contact: | goodger@users.sourceforge.net
Revision: | 1.10
Date: | 2004-06-03
Copyright: | This document has been placed in the public domain.

There are several problems, unresolved issues, and areas of controversy within StructuredText (Classic and Next Generation). In order to resolve all these issues, this analysis brings all of the issues out into the open, enumerates all the alternatives, and proposes solutions to be incorporated into the reStructuredText specification.

Contents

  • Formal Specification
  • Understanding and Extending the Code
  • Section Structure via Indentation
  • Character Escaping Mechanism
  • Blank Lines in Lists
  • Bullet List Markup
  • Enumerated List Markup
  • Definition List Markup
  • Literal Blocks
  • Tables
  • Delimitation of Inline Markup
  • Underlining
  • Inline Literals
  • Hyperlinks

Formal Specification

The description in the original StructuredText.py has been criticized for being vague. For practical purposes, "the code is the spec." Tony Ibbs has been working on deducing a detailed description from the documentation and code of StructuredTextNG. Edward Loper's STMinus is another attempt to formalize a spec.

For this kind of a project, the specification should always precede the code. Otherwise, the markup is a moving target which can never be adopted as a standard. Of course, a specification may be revised during lifetime of the code, but without a spec there is no visible control and thus no confidence.

Understanding and Extending the Code

The original StructuredText is a dense mass of sparsely commented code and inscrutable regular expressions. It was not designed to be extended and is very difficult to understand. StructuredTextNG has been designed to allow input (syntax) and output extensions, but its documentation (both internal [comments & docstrings], and external) is inadequate for the complexity of the code itself.

For reStructuredText to become truly useful, perhaps even part of Python's standard library, it must have clear, understandable documentation and implementation code. For the implementation of reStructuredText to be taken seriously, it must be a sterling example of the potential of docstrings; the implementation must practice what the specification preaches.

Section Structure via Indentation

Setext required that body text be indented by 2 spaces. The original StructuredText and StructuredTextNG require that section structure be indicated through indentation, as "inspired by Python". For certain structures with a very limited, local extent (such as lists, block quotes, and literal blocks), indentation naturally indicates structure or hierarchy. For sections (which may have a very large extent), structure via indentation is unnecessary, unnatural and ambiguous. Rather, the syntax of the section title itself should indicate that it is a section title.

The original StructuredText states that "A single-line paragraph whose immediately succeeding paragraphs are lower level is treated as a header." Requiring indentation in this way is:

  • Unnecessary. The vast majority of docstrings and standalone documents will have no more than one level of section structure. Requiring indentation for such docstrings is unnecessary and irritating.
  • Unnatural. Most published works use title style (type size, face, weight, and position) and/or section/subsection numbering rather than indentation to indicate hierarchy. This is a tradition with a very long history.
  • Ambiguous. A StructuredText header is indistinguishable from a one-line paragraph followed by a block quote (precluding the use of block quotes). Enumerated section titles are ambiguous (is it a header? is it a list item?). Some additional adornment must be required to confirm the line's role as a title, both to a parser and to the human reader of the source text.

Python's use of significant whitespace is a wonderful (if not original) innovation, however requiring indentation in ordinary written text is hypergeneralization.

reStructuredText indicates section structure through title adornment style (as exemplified by this document). This is far more natural. In fact, it is already in widespread use in plain text documents, including in Python's standard distribution (such as the toplevel README file).

Character Escaping Mechanism

No matter what characters are chosen for markup, some day someone will want to write documentation about that markup or using markup characters in a non-markup context. Therefore, any complete markup language must have an escaping or encoding mechanism. For a lightweight markup system, encoding mechanisms like SGML/XML's '&ast;' are out. So an escaping mechanism is in. However, with carefully chosen markup, it should be necessary to use the escaping mechanism only infrequently.

reStructuredText needs an escaping mechanism: a way to treat markup-significant characters as the characters themselves. Currently there is no such mechanism (although ZWiki uses '!'). What are the candidates?

  1. ! (http://dev.zope.org/Members/jim/StructuredTextWiki/NGEscaping)
  2. \
  3. ~
  4. doubling of characters

The best choice for this is the backslash (\). It's "the single most popular escaping character in the world!", therefore familiar and unsurprising. Since characters only need to be escaped under special circumstances, which are typically those explaining technical programming issues, the use of the backslash is natural and understandable. Python docstrings can be raw (prefixed with an 'r', as in 'r""'), which would obviate the need for gratuitous doubling-up of backslashes.

(On 2001-03-29 on the Doc-SIG mailing list, GvR endorsed backslash escapes, saying, "'nuff said. Backslash it is." Although neither legally binding nor irrevocable nor any kind of guarantee of anything, it is a good sign.)

The rule would be: An unescaped backslash followed by any markup character escapes the character. The escaped character represents the character itself, and is prevented from playing a role in any markup interpretation. The backslash is removed from the output. A literal backslash is represented by an "escaped backslash," two backslashes in a row.

A carefully constructed set of recognition rules for inline markup will obviate the need for backslash-escapes in almost all cases; see Delimitation of Inline Markup below.

When an expression (requiring backslashes and other characters used for markup) becomes too complicated and therefore unreadable, a literal block may be used instead. Inside literal blocks, no markup is recognized, therefore backslashes (for the purpose of escaping markup) become unnecessary.

We could allow backslashes preceding non-markup characters to remain in the output. This would make describing regular expressions and other uses of backslashes easier. However, this would complicate the markup rules and would be confusing.

Blank Lines in Lists

Oft-requested in Doc-SIG (the earliest reference is dated 1996-08-13) is the ability to write lists without requiring blank lines between items. In docstrings, space is at a premium. Authors want to convey their API or usage information in as compact a form as possible. StructuredText requires blank lines between all body elements, including list items, even when boundaries are obvious from the markup itself.

In reStructuredText, blank lines are optional between list items. However, in order to eliminate ambiguity, a blank line is required before the first list item and after the last. Nested lists also require blank lines before the list start and after the list end.

Bullet List Markup

StructuredText includes 'o' as a bullet character. This is dangerous and counter to the language-independent nature of the markup. There are many languages in which 'o' is a word. For example, in Spanish:

Llamame a la casa
o al trabajo.

(Call me at home or at work.)

And in Japanese (when romanized):

Senshuu no doyoubi ni tegami
o kakimashita.

([I] wrote a letter on Saturday last week.)

If a paragraph containing an 'o' word wraps such that the 'o' is the first text on a line, or if a paragraph begins with such a word, it could be misinterpreted as a bullet list.

In reStructuredText, 'o' is not used as a bullet character. '-', '*', and '+' are the possible bullet characters.

Enumerated List Markup

StructuredText enumerated lists are allowed to begin with numbers and letters followed by a period or right-parenthesis, then whitespace. This has surprising consequences for writing styles. For example, this is recognized as an enumerated list item by StructuredText:

Mr. Creosote.

People will write enumerated lists in all different ways. It is folly to try to come up with the "perfect" format for an enumerated list, and limit the docstring parser's recognition to that one format only.

Rather, the parser should recognize a variety of enumerator styles. It is also recommended that the enumerator of the first list item be ordinal-1 ('1', 'A', 'a', 'I', or 'i'), as output formats may not be able to begin a list at an arbitrary enumeration.

An initial idea was to require two or more consistent enumerated list items in a row. This idea proved impractical and was dropped. In practice, the presence of a proper enumerator is enough to reliably recognize an enumerated list item; any ambiguities are reported by the parser. Here's the original idea for posterity:

The parser should recognize a variety of enumerator styles, mark each block as a potential enumerated list item (PELI), and interpret the enumerators of adjacent PELIs to decide whether they make up a consistent enumerated list.

If a PELI is labeled with a "1.", and is immediately followed by a PELI labeled with a "2.", we've got an enumerated list. Or "(A)" followed by "(B)". Or "i)" followed by "ii)", etc. The chances of accidentally recognizing two adjacent and consistently labeled PELIs, are acceptably small.

For an enumerated list to be recognized, the following must be true:

  • the list must consist of multiple adjacent list items (2 or more)
  • the enumerators must all have the same format
  • the enumerators must be sequential

Definition List Markup

StructuredText uses ' -- ' (whitespace, two hyphens, whitespace) on the first line of a paragraph to indicate a definition list item. The ' -- ' serves to separate the term (on the left) from the definition (on the right).

Many people use ' -- ' as an em-dash in their text, conflicting with the StructuredText usage. Although the Chicago Manual of Style says that spaces should not be used around an em-dash, Peter Funk pointed out that this is standard usage in German (according to the Duden, the official German reference), and possibly in other languages as well. The widespread use of ' -- ' precludes its use for definition lists; it would violate the "unsurprising" criterion.

A simpler, and at least equally visually distinctive construct (proposed by Guido van Rossum, who incidentally is a frequent user of ' -- ') would do just as well:

term 1
    Definition.

term 2
    Definition 2, paragraph 1.

    Definition 2, paragraph 2.

A reStructuredText definition list item consists of a term and a definition. A term is a simple one-line paragraph. A definition is a block indented relative to the term, and may contain multiple paragraphs and other body elements. No blank line precedes a definition (this distinguishes definition lists from block quotes).

Literal Blocks

The StructuredText specification has literal blocks indicated by 'example', 'examples', or '::' ending the preceding paragraph. STNG only recognizes '::'; 'example'/'examples' are not implemented. This is good; it fixes an unnecessary language dependency. The problem is what to do with the sometimes- unwanted '::'.

In reStructuredText '::' at the end of a paragraph indicates that subsequent indented blocks are treated as literal text. No further markup interpretation is done within literal blocks (not even backslash-escapes). If the '::' is preceded by whitespace, '::' is omitted from the output; if '::' was the sole content of a paragraph, the entire paragraph is removed (no 'empty' paragraph remains). If '::' is preceded by a non-whitespace character, '::' is replaced by ':' (i.e., the extra colon is removed).

Thus, a section could begin with a literal block as follows:

Section Title
-------------

::

    print "this is example literal"

Tables

The table markup scheme in classic StructuredText was horrible. Its omission from StructuredTextNG is welcome, and its markup will not be repeated here. However, tables themselves are useful in documentation. Alternatives:

  1. This format is the most natural and obvious. It was independently invented (no great feat of creation!), and later found to be the format supported by the Emacs table mode:

    +------------+------------+------------+--------------+
|  Header 1  |  Header 2  |  Header 3  |  Header 4    |
+============+============+============+==============+
|  Column 1  |  Column 2  | Column 3 & 4 span (Row 1) |
+------------+------------+------------+--------------+
|    Column 1 & 2 span    |  Column 3  | - Column 4   |
+------------+------------+------------+ - Row 2 & 3  |
|      1     |      2     |      3     | - span       |
+------------+------------+------------+--------------+

    Tables are described with a visual outline made up of the characters '-', '=', '|', and '+':

    • The hyphen ('-') is used for horizontal lines (row separators).
    • The equals sign ('=') is optionally used as a header separator (as of version 1.5.24, this is not supported by the Emacs table mode).
    • The vertical bar ('|') is used for for vertical lines (column separators).
    • The plus sign ('+') is used for intersections of horizontal and vertical lines.

    Row and column spans are possible simply by omitting the column or row separators, respectively. The header row separator must be complete; in other words, a header cell may not span into the table body. Each cell contains body elements, and may have multiple paragraphs, lists, etc. Initial spaces for a left margin are allowed; the first line of text in a cell determines its left margin.

  2. Below is a simpler table structure. It may be better suited to manual input than alternative #1, but there is no Emacs editing mode available. One disadvantage is that it resembles section titles; a one-column table would look exactly like section & subsection titles.

    ============ ============ ============ ==============
  Header 1     Header 2     Header 3     Header 4
============ ============ ============ ==============
  Column 1     Column 2    Column 3 & 4 span (Row 1)
------------ ------------ ---------------------------
    Column 1 & 2 span       Column 3    - Column 4
------------------------- ------------  - Row 2 & 3
      1            2            3       - span
============ ============ ============ ==============

    The table begins with a top border of equals signs with a space at each column boundary (regardless of spans). Each row is underlined. Internal row separators are underlines of '-', with spaces at column boundaries. The last of the optional head rows is underlined with '=', again with spaces at column boundaries. Column spans have no spaces in their underline. Row spans simply lack an underline at the row boundary. The bottom boundary of the table consists of '=' underlines. A blank line is required following a table.

  3. A minimalist alternative is as follows:

    ====  =====  ========  ========  =======  ====  =====  =====
Old State    Input     Action             New State    Notes
-----------  --------  -----------------  -----------
ids   types  new type  sys.msg.  dupname  ids   types
====  =====  ========  ========  =======  ====  =====  =====
--    --     explicit  --        --       new   True
--    --     implicit  --        --       new   False
None  False  explicit  --        --       new   True
old   False  explicit  implicit  old      new   True
None  True   explicit  explicit  new      None  True
old   True   explicit  explicit  new,old  None  True   [1]
None  False  implicit  implicit  new      None  False
old   False  implicit  implicit  new,old  None  False
None  True   implicit  implicit  new      None  True
old   True   implicit  implicit  new      old   True
====  =====  ========  ========  =======  ====  =====  =====

    The table begins with a top border of equals signs with one or more spaces at each column boundary (regardless of spans). There must be at least two columns in the table (to differentiate it from section headers). Each line starts a new row. The rightmost column is unbounded; text may continue past the edge of the table. Each row/line must contain spaces at column boundaries, except for explicit column spans. Underlines of '-' can be used to indicate column spans, but should be used sparingly if at all. Lines containing column span underlines may not contain any other text. The last of the optional head rows is underlined with '=', again with spaces at column boundaries. The bottom boundary of the table consists of '=' underlines. A blank line is required following a table.

    This table sums up the features. Using all the features in such a small space is not pretty though:

    ========  ========  ========
          Header 2 & 3 Span
          ------------------
Header 1  Header 2  Header 3
========  ========  ========
Each      line is   a new row.
Each row  consists  of one line only.
Row       spans     are not possible.
The last  column    may spill over to the right.
Column spans are possible with an underline joining columns.
----------------------------
The span  is        limited to the row above the underline.
========  ========  ========

  4. As a variation of alternative 3, bullet list syntax in the first column could be used to indicate row starts. Multi-line rows are possible, but row spans are not. For example:

    ===== =====
col 1 col 2
===== =====
- 1   Second column of row 1.
- 2   Second column of row 2.
      Second line of paragraph.
- 3   Second column of row 3.

      Second paragraph of row 3,
      column 2
===== =====

    Column spans would be indicated on the line after the last line of the row. To indicate a real bullet list within a first-column cell, simply nest the bullets.

  5. In a further variation, we could simply assume that whitespace in the first column implies a multi-line row; the text in other columns is continuation text. For example:

    ===== =====
col 1 col 2
===== =====
1     Second column of row 1.
2     Second column of row 2.
      Second line of paragraph.
3     Second column of row 3.

      Second paragraph of row 3,
      column 2
===== =====

    Limitations of this approach:

    • Cells in the first column are limited to one line of text.
    • Cells in the first column must contain some text; blank cells would lead to a misinterpretation. An empty comment ("..") is sufficient.

  6. Combining alternative 3 and 4, a bullet list in the first column could mean multi-line rows, and no bullet list means single-line rows only.

Alternatives 1 and 5 has been adopted by reStructuredText.

Delimitation of Inline Markup

StructuredText specifies that inline markup must begin with whitespace, precluding such constructs as parenthesized or quoted emphatic text:

"**What?**" she cried.  (*exit stage left*)

The reStructuredText markup specification allows for such constructs and disambiguates inline markup through a set of recognition rules. These recognition rules define the context of markup start-strings and end-strings, allowing markup characters to be used in most non-markup contexts without a problem (or a backslash). So we can say, "Use asterisks (*) around words or phrases to emphasisze them." The '(*)' will not be recognized as markup. This reduces the need for markup escaping to the point where an escape character is almost (but not quite!) unnecessary.

Underlining

StructuredText uses '_text_' to indicate underlining. To quote David Ascher in his 2000-01-21 Doc-SIG mailing list post, "Docstring grammar: a very revised proposal":

The tagging of underlined text with _'s is suboptimal. Underlines shouldn't be used from a typographic perspective (underlines were designed to be used in manuscripts to communicate to the typesetter that the text should be italicized -- no well-typeset book ever uses underlines), and conflict with double-underscored Python variable names (__init__ and the like), which would get truncated and underlined when that effect is not desired. Note that while complete markup would prevent that truncation ('__init__'), I think of docstring markups much like I think of type annotations -- they should be optional and above all do no harm. In this case the underline markup does harm.

Underlining is not part of the reStructuredText specification.

Inline Literals

StructuredText's markup for inline literals (text left as-is, verbatim, usually in a monospaced font; as in HTML <TT>) is single quotes ('literals'). The problem with single quotes is that they are too often used for other purposes:

  • Apostrophes: "Don't blame me, 'cause it ain't mine, it's Chris'.";

  • Quoting text:

    First Bruce: "Well Bruce, I heard the prime minister use it. 'S'hot enough to boil a monkey's bum in 'ere your Majesty,' he said, and she smiled quietly to herself."

    In the UK, single quotes are used for dialogue in published works.

  • String literals: s = ''

Alternatives:

'text'    \'text\'    ''text''    "text"    \"text\"    ""text""
#text#     @text@      `text`     ^text^    ``text''    ``text``

The examples below contain inline literals, quoted text, and apostrophes. Each example should evaluate to the following HTML:

Some <TT>code</TT>, with a 'quote', "double", ain't it grand?
Does <TT>a[b] = 'c' + "d" + `2^3`</TT> work?

0. Some code, with a quote, double, ain't it grand?
   Does a[b] = 'c' + "d" + `2^3` work?
1. Some 'code', with a \'quote\', "double", ain\'t it grand?
   Does 'a[b] = \'c\' + "d" + `2^3`' work?
2. Some \'code\', with a 'quote', "double", ain't it grand?
   Does \'a[b] = 'c' + "d" + `2^3`\' work?
3. Some ''code'', with a 'quote', "double", ain't it grand?
   Does ''a[b] = 'c' + "d" + `2^3`'' work?
4. Some "code", with a 'quote', \"double\", ain't it grand?
   Does "a[b] = 'c' + "d" + `2^3`" work?
5. Some \"code\", with a 'quote', "double", ain't it grand?
   Does \"a[b] = 'c' + "d" + `2^3`\" work?
6. Some ""code"", with a 'quote', "double", ain't it grand?
   Does ""a[b] = 'c' + "d" + `2^3`"" work?
7. Some #code#, with a 'quote', "double", ain't it grand?
   Does #a[b] = 'c' + "d" + `2^3`# work?
8. Some @code@, with a 'quote', "double", ain't it grand?
   Does @a[b] = 'c' + "d" + `2^3`@ work?
9. Some `code`, with a 'quote', "double", ain't it grand?
   Does `a[b] = 'c' + "d" + \`2^3\`` work?
10. Some ^code^, with a 'quote', "double", ain't it grand?
    Does ^a[b] = 'c' + "d" + `2\^3`^ work?
11. Some ``code'', with a 'quote', "double", ain't it grand?
    Does ``a[b] = 'c' + "d" + `2^3`'' work?
12. Some ``code``, with a 'quote', "double", ain't it grand?
    Does ``a[b] = 'c' + "d" + `2^3\``` work?

Backquotes (#9 & #12) are the best choice. They are unobtrusive and relatviely rarely used (more rarely than ' or ", anyhow). Backquotes have the connotation of 'quotes', which other options (like carets, #10) don't.

Analogously with *emph* & **strong**, double-backquotes (#12) could be used for inline literals. If single-backquotes are used for 'interpreted text' (context-sensitive domain-specific descriptive markup) such as function name hyperlinks in Python docstrings, then double-backquotes could be used for absolute-literals, wherein no processing whatsoever takes place. An advantage of double-backquotes would be that backslash-escaping would no longer be necessary for embedded single-backquotes; however, embedded double-backquotes (in an end-string context) would be illegal. See Backquotes in Phrase-Links in Record of reStructuredText Syntax Alternatives.

Alternative choices are carets (#10) and TeX-style quotes (#11). For examples of TeX-style quoting, see http://www.zope.org/Members/jim/StructuredTextWiki/CustomizingTheDocumentProcessor.

Some existing uses of backquotes:

  1. As a synonym for repr() in Python.
  2. For command-interpolation in shell scripts.
  3. Used as open-quotes in TeX code (and carried over into plaintext by TeXies).

The inline markup start-string and end-string recognition rules defined by the reStructuredText markup specification would allow all of these cases inside inline literals, with very few exceptions. As a fallback, literal blocks could handle all cases.

Outside of inline literals, the above uses of backquotes would require backslash-escaping. However, these are all prime examples of text that should be marked up with inline literals.

If either backquotes or straight single-quotes are used as markup, TeX-quotes are too troublesome to support, so no special-casing of TeX-quotes should be done (at least at first). If TeX-quotes have to be used outside of literals, a single backslash-escaped would suffice: ``TeX quote''. Ugly, true, but very infrequently used.

Using literal blocks is a fallback option which removes the need for backslash-escaping:

like this::

    Here, we can do ``absolutely'' anything `'`'\|/|\ we like!

No mechanism for inline literals is perfect, just as no escaping mechanism is perfect. No matter what we use, complicated inline expressions involving the inline literal quote and/or the backslash will end up looking ugly. We can only choose the least often ugly option.

reStructuredText will use double backquotes for inline literals, and single backqoutes for interpreted text.

Hyperlinks

There are three forms of hyperlink currently in StructuredText:

  1. (Absolute & relative URIs.) Text enclosed by double quotes followed by a colon, a URI, and concluded by punctuation plus white space, or just white space, is treated as a hyperlink:

    "Python":http://www.python.org/

  2. (Absolute URIs only.) Text enclosed by double quotes followed by a comma, one or more spaces, an absolute URI and concluded by punctuation plus white space, or just white space, is treated as a hyperlink:

    "mail me", mailto:me@mail.com

  3. (Endnotes.) Text enclosed by brackets link to an endnote at the end of the document: at the beginning of the line, two dots, a space, and the same text in brackets, followed by the end note itself:

    Please refer to the fine manual [GVR2001].

.. [GVR2001] Python Documentation, Release 2.1, van Rossum,
   Drake, et al., http://www.python.org/doc/

The problem with forms 1 and 2 is that they are neither intuitive nor unobtrusive (they break design goals 5 & 2). They overload double-quotes, which are too often used in ordinary text (potentially breaking design goal 4). The brackets in form 3 are also too common in ordinary text (such as [nested] asides and Python lists like [12]).

Alternatives:

  1. Have no special markup for hyperlinks.

    1. Interpret and mark up hyperlinks as any contiguous text containing '://' or ':...@' (absolute URI) or '@' (email address) after an alphanumeric word. To de-emphasize the URI, simply enclose it in parentheses:

      Python (http://www.python.org/)

    2. Leave special hyperlink markup as a domain-specific extension. Hyperlinks in ordinary reStructuredText documents would be required to be standalone (i.e. the URI text inline in the document text). Processed hyperlinks (where the URI text is hidden behind the link) are important enough to warrant syntax.

  3. The original Setext introduced a mechanism of indirect hyperlinks. A source link word ('hot word') in the text was given a trailing underscore:

    Here is some text with a hyperlink_ built in.

    The hyperlink itself appeared at the end of the document on a line by itself, beginning with two dots, a space, the link word with a leading underscore, whitespace, and the URI itself:

    .. _hyperlink http://www.123.xyz

    Setext used underscores_instead_of_spaces_ for phrase links.

With some modification, alternative 3 best satisfies the design goals. It has the advantage of being readable and relatively unobtrusive. Since each source link must match up to a target, the odd variable ending in an underscore can be spared being marked up (although it should generate a "no such link target" warning). The only disadvantage is that phrase-links aren't possible without some obtrusive syntax.

We could achieve phrase-links if we enclose the link text:

  1. in double quotes:

    "like this"_

  2. in brackets:

    [like this]_

  3. or in backquotes:

    `like this`_

Each gives us somewhat obtrusive markup, but that is unavoidable. The bracketed syntax (#2) is reminiscent of links on many web pages (intuitive), although it is somewhat obtrusive. Alternative #3 is much less obtrusive, and is consistent with interpreted text: the trailing underscore indicates the interpretation of the phrase, as a hyperlink. #3 also disambiguates hyperlinks from footnote references. Alternative #3 wins.

The same trailing underscore markup can also be used for footnote and citation references, removing the problem with ordinary bracketed text and Python lists:

Please refer to the fine manual [GVR2000]_.

.. [GVR2000] Python Documentation, van Rossum, Drake, et al.,
   http://www.python.org/doc/

The two-dots-and-a-space syntax was generalized by Setext for comments, which are removed from the (visible) processed output. reStructuredText uses this syntax for comments, footnotes, and link target, collectively termed "explicit markup". For link targets, in order to eliminate ambiguity with comments and footnotes, reStructuredText specifies that a colon always follow the link target word/phrase. The colon denotes 'maps to'. There is no reason to restrict target links to the end of the document; they could just as easily be interspersed.

Internal hyperlinks (links from one point to another within a single document) can be expressed by a source link as before, and a target link with a colon but no URI. In effect, these targets 'map to' the element immediately following.

As an added bonus, we now have a perfect candidate for reStructuredText directives, a simple extension mechanism: explicit markup containing a single word followed by two colons and whitespace. The interpretation of subsequent data on the directive line or following is directive-dependent.

To summarize:

.. This is a comment.

.. The line below is an example of a directive.
.. version:: 1

This is a footnote [1]_.

This internal hyperlink will take us to the footnotes_ area below.

Here is a one-word_ external hyperlink.

Here is `a hyperlink phrase`_.

.. _footnotes:
.. [1] Footnote text goes here.

.. external hyperlink target mappings:
.. _one-word: http://www.123.xyz
.. _a hyperlink phrase: http://www.123.xyz

The presence or absence of a colon after the target link differentiates an indirect hyperlink from a footnote, respectively. A footnote requires brackets. Backquotes around a target link word or phrase are required if the phrase contains a colon, optional otherwise.

Below are examples using no markup, the two StructuredText hypertext styles, and the reStructuredText hypertext style. Each example contains an indirect link, a direct link, a footnote/endnote, and bracketed text. In HTML, each example should evaluate to:

<P>A <A HREF="http://spam.org">URI</A>, see <A HREF="#eggs2000">
[eggs2000]</A> (in Bacon [Publisher]).  Also see
<A HREF="http://eggs.org">http://eggs.org</A>.</P>

<P><A NAME="eggs2000">[eggs2000]</A> "Spam, Spam, Spam, Eggs,
Bacon, and Spam"</P>

  1. No markup:

    A URI http://spam.org, see eggs2000 (in Bacon [Publisher]).
Also see http://eggs.org.

eggs2000 "Spam, Spam, Spam, Eggs, Bacon, and Spam"

  2. StructuredText absolute/relative URI syntax ("text":http://www.url.org):

    A "URI":http://spam.org, see [eggs2000] (in Bacon [Publisher]).
Also see "http://eggs.org":http://eggs.org.

.. [eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam"

    Note that StructuredText does not recognize standalone URIs, forcing doubling up as shown in the second line of the example above.

  3. StructuredText absolute-only URI syntax ("text", mailto:you@your.com):

    A "URI", http://spam.org, see [eggs2000] (in Bacon
[Publisher]).  Also see "http://eggs.org", http://eggs.org.

.. [eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam"

  4. reStructuredText syntax:

    4. A URI_, see [eggs2000]_ (in Bacon [Publisher]).
   Also see http://eggs.org.

   .. _URI: http:/spam.org
   .. [eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam"

The bracketed text '[Publisher]' may be problematic with StructuredText (syntax 2 & 3).

reStructuredText's syntax (#4) is definitely the most readable. The text is separated from the link URI and the footnote, resulting in cleanly readable text.

View document source. Generated on: 2004-07-01 22:45 UTC. Generated by Docutils from reStructuredText source.


================
 Docutils_ Bugs
================

:Author: David Goodger; open to all Docutils developers
:Contact: goodger@python.org
:Date: $Date: 2004/06/29 02:44:04 $
:Revision: $Revision: 1.12 $
:Copyright: This document has been placed in the public domain.

.. _Docutils: http://docutils.sourceforge.net/

Bugs in Docutils?!?  Yes, we do have a few.  Some are old-timers that
tend to stay in the shadows and don't bother anybody.  Once in a while
new bugs are born.  From time to time some bugs (new and old) crawl
out into the light and must be dealt with.  Icky.

This document describes how to report a bug, and lists known bugs.

.. contents::

How To Report A Bug
===================

If you think you've discovered a bug, please read through these
guidelines before reporting it.

First, make sure it's a new bug:

* Please check the list of `known bugs`_ below and the `SourceForge
  Bug Tracker`_ to see if it has already been reported.

* Are you using the very latest version of Docutils?  The bug may have
  already been fixed.  Please get the latest version of Docutils from
  CVS_ or from the `development snapshot`_ and check again.  Even if
  your bug has not been fixed, others probably have, and you're better
  off with the most up-to-date code.

  If you don't have time to check the latest snapshot, please report
  the bug anyway.  We'd rather tell you that it's already fixed than
  miss reports of unfixed bugs.

* If Docutils does not behave the way you expect, look in the
  documentation_ (don't forget the FAQ_!) and `mailing list archives`_
  for evidence that it should behave the way you expect.

If you're not sure, please ask on the
docutils-users@lists.sourceforge.net [1]_ mailing list first.

If it's a new bug, the most important thing you can do is to write a
simple description and a recipe that reproduces the bug.  Try to
create a minimal document that demonstrates the bug.  The easier you
make it to understand and track down the bug, the more likely a fix
will be.

Now you're ready to write the bug report.  Please include:

* A clear description of the bug.  Describe how you expected Docutils
  to behave, and contrast that with how it actually behaved.  While
  the bug may seem obvious to you, it may not be so obvious to someone
  else, so it's best to avoid a guessing game.

* A complete description of the environment in which you reproduced
  the bug:

  - Your operating system & version.
  - The version of Python (``python -V``).
  - The version of Docutils (use the "-V" option to most Docutils
    front-end tools).
  - Any private modifications you made to Docutils.
  - Anything else that could possibly be relevant.  Err on the side
    of too much information, rather than too little.

* A literal transcript of the *exact* command you ran, and the *exact*
  output.  Use the "--traceback" option to get a complete picture.

* The exact input and output files.  Better to attach complete files
  to your bug report than to include just a summary or excerpt.

* If you also want to include speculation as to the cause, and even a
  patch to fix the bug, that would be great!

The best place to send your bug report is to the `SourceForge Bug
Tracker`_.  That way, it won't be misplaced or forgotten.  In fact, an
open bug report on SourceForge is a constant irritant that begs to be
squashed.

Thank you!

(This section was inspired by the `Subversion project's`__ BUGS__
file.)

.. [1] Due to overwhelming amounts of spam, the
   docutils-users@lists.sourceforge.net mailing list has been set up
   for subscriber posting only.  Non-subscribers who post to
   docutils-users will receive a message with "Subject: Your message
   to Docutils-users awaits moderator approval".  Legitimate messages
   are accepted and posted as soon as possible (a list administrator
   must verify the message manually).  If you'd like to subscribe to
   docutils-users, please visit
   <http://lists.sourceforge.net/lists/listinfo/docutils-users>.

__ http://subversion.tigris.org/
__ http://svn.collab.net/viewcvs/svn/trunk/BUGS?view=markup

.. _CVS: http://sourceforge.net/cvs/?group_id=38414
.. _development snapshot: http://docutils.sf.net/#development-snapshot
.. _documentation: docs/
.. _FAQ: FAQ.html
.. _mailing list archives: http://docutils.sf.net/#mailing-lists
.. _SourceForge Bug Tracker:
   http://sourceforge.net/tracker/?group_id=38414&atid=422030

Known Bugs
==========

Also see the `SourceForge Bug Tracker`_.

* The parser doesn't know anything about double-width characters such
  as Chinese hanza & Japanese kanji/kana.  Also, it's dependent on
  whitespace and punctuation as markup delimiters, which may not be
  applicable in these languages.

* Line numbers in system messages are inconsistent in the parser.

  - In text inserted by the "include" directive, errors are often not
    reported with the correct "source" or "line" numbers.  Perhaps all
    Reporter calls need "source" and "line" keyword arguments.
    Elements' .line assignments should be checked.  (Assign to .source
    too?  Add a set_info method?  To what?)  There's a test in
    test/test_parsers/test_rst/test_directives/test_include.py.

  - Some line numbers in elements are not being set properly
    (explicitly), just implicitly/automatically.  See rev. 1.74 of
    docutils/parsers/rst/states.py for an example of how to set.

* Quite a few nodes are getting a "None" source attribute as well.  In
  particular, see the bodies of definition lists.

* Explicit targets are sometimes mis-located.  In particular, placing
  a target before a section header puts the target at the end of the
  previous section instead of the start of the next section.  The code
  in docutils.transforms.misc.ClassAttribute could be used to fix
  this.  (Reported by David Priest.)

* David Abrahams pointed out that doubly-indirect substitutions have a
  bug, but only when there's multiple references::

      |substitute| my coke for gin
      |substitute| you for my mum
      at least I'll get my washing done

      .. |substitute| replace:: |replace|
      .. |replace| replace:: swap

  This is tricky.  Substitutions have to propagate back completely.

* Another bug from David Abrahams (run with ``rst2html.py --traceback``)::

      .. [#crtp] See |runtime|

      foo [#tag_dispatching]_

      .. [#tag_dispatching] See  |runtime|

      .. |runtime| replace:: 7__
      __ runtime.html

      __ reference/__main.html

  Change the references.Substitutions tranform's priority from 220 to
  680, so it happens after reference resolution?  Then we have to deal
  with multiple IDs.  Perhaps the Substitution transform should remove
  all IDs from definitions after the first substitution reference is
  processed.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


==================
 Copying Docutils
==================

:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Date: $Date: 2004/06/22 20:13:36 $
:Web site: http://docutils.sourceforge.net/
:Copyright: This document has been placed in the public domain.

Most of the files included in this project have been placed in the
public domain, and therefore have no license requirements and no
restrictions on copying or usage; see the `Public Domain Dedication`_
below.  There are a few exceptions_, listed below.

One goal of the Docutils project is to be included in the Python
standard library distribution, at which time it is expected that
copyright will be asserted by the `Python Software Foundation
<http://www.python.org/psf/>`_.

Public Domain Dedication
========================

The persons who have associated their work with this project (the
"Dedicator": David Goodger and the many contributors to the Docutils
project) hereby dedicate the entire copyright, less the exceptions_
listed below, in the work of authorship known as "Docutils" identified
below (the "Work") to the public domain.

The primary repository for the Work is the Internet World Wide Web
site <http://docutils.sourceforge.net/>.  The Work consists of the
files within the "docutils" module of the Docutils project CVS
repository (Internet host cvs.sourceforge.net, filesystem path
/cvsroot/docutils), whose Internet web interface is located at
<http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/docutils/docutils/>.
Files dedicated to the public domain may be identified by the
inclusion, near the beginning of each file, of a declaration of the
form::

    Copyright: This document/module/DTD/stylesheet/file/etc. has been
               placed in the public domain.

Dedicator makes this dedication for the benefit of the public at large
and to the detriment of Dedicator's heirs and successors.  Dedicator
intends this dedication to be an overt act of relinquishment in
perpetuity of all present and future rights under copyright law,
whether vested or contingent, in the Work.  Dedicator understands that
such relinquishment of all rights includes the relinquishment of all
rights to enforce (by lawsuit or otherwise) those copyrights in the
Work.

Dedicator recognizes that, once placed in the public domain, the Work
may be freely reproduced, distributed, transmitted, used, modified,
built upon, or otherwise exploited by anyone for any purpose,
commercial or non-commercial, and in any way, including by methods
that have not yet been invented or conceived.

(This dedication is derived from the text of the `Creative Commons
Public Domain Dedication
<http://creativecommons.org/licenses/publicdomain>`_.)

Exceptions
==========

The exceptions to the `Public Domain Dedication`_ above are:

* extras/optparse.py, copyright by Gregory P. Ward, released under a
  BSD-style license (which can be found in the module's source code).

* extras/textwrap.py, copyright by Gregory P. Ward and the Python
  Software Foundation, released under the `Python 2.3 license`_
  (`local copy`__).

  __ licenses/python-2-3.txt

* extras/roman.py, copyright by Mark Pilgrim, released under the
  `Python 2.1.1 license`_ (`local copy`__).

  __ licenses/python-2-1-1.txt

* test/docutils_difflib.py, copyright by the Python Software
  Foundation, released under the `Python 2.2 license`_ (`local
  copy`__).  This file is included for compatibility with Python
  versions less than 2.2.  (It's only used to report test failures
  anyhow; it isn't installed anywhere.  The included file is a
  pre-generator version of the difflib.py module included in Python
  2.2.)

  __ licenses/python-2-2.txt

* tools/pep2html.py, copyright by the Python Software Foundation,
  released under the `Python 2.2 license`_ (`local copy`__).

  __ licenses/python-2-2.txt

* tools/editors/emacs/rst-html.el, copyright by Martin Blais, released
  under the `GNU General Public License`_ (`local copy`__).

  __ licenses/gpl.txt

* tools/editors/emacs/rst-mode.el, copyright by Stefan Merten,
  released under the `GNU General Public License`_ (`local copy`__).

  __ licenses/gpl.txt

(Disclaimer: I am not a lawyer.)  The BSD license and the Python
licenses are OSI-approved_ and GPL-compatible_.  Although complicated
by multiple owners and lots of legalese, the Python license basically
lets you copy, use, modify, and redistribute files as long as you keep
the copyright attribution intact, note any changes you make, and don't
use the owner's name in vain.  The BSD license is similar.

Plaintext versions of all the linked-to licenses are provided in the
licenses_ directory.

.. _licenses: licenses/
.. _Python 2.1.1 license: http://www.python.org/2.1.1/license.html
.. _Python 2.2 license: http://www.python.org/2.2/license.html
.. _Python 2.3 license: http://www.python.org/2.3/license.html
.. _GNU General Public License: http://www.gnu.org/copyleft/gpl.html
.. _OSI-approved: http://opensource.org/licenses/
.. _GPL-compatible: http://www.gnu.org/philosophy/license-list.html


===================================== Docutils Frequently Asked Questions ===================================== :Date: $Date: 2004/06/29 02:44:14 $ :Web site: http://docutils.sourceforge.net/ :Copyright: This document has been placed in the public domain. .. Please note that until there's a Q&A-specific construct available, this FAQ will use section titles for questions. Therefore questions must fit on one line. The title may be a summary of the question, with the full question in the section body. .. contents:: .. sectnum:: This is a work in progress. Please feel free to ask questions and/or provide answers; `send email`__ to the `Docutils-Users mailing list`__ [1]_. Project members should feel free to edit the source text file directly. .. [1] Due to overwhelming amounts of spam, the docutils-users@lists.sourceforge.net mailing list has been set up for subscriber posting only. Non-subscribers who post to docutils-users will receive a message with "Subject: Your message to Docutils-users awaits moderator approval". Legitimate messages are accepted and posted as soon as possible (a list administrator must verify the message manually). If you'd like to subscribe to docutils-users, please visit . .. _let us know: __ mailto:docutils-users@lists.sourceforge.net __ http://lists.sourceforge.net/lists/listinfo/docutils-users Docutils ======== What is Docutils? ----------------- Docutils_ is a system for processing plaintext documentation into useful formats, such as HTML, XML, and TeX. It supports multiple types of input, such as standalone files (implemented), inline documentation from Python modules and packages (under development), `PEPs (Python Enhancement Proposals)`_ (implemented), and others as discovered. For an overview of the Docutils project implementation, see `PEP 258`_, "Docutils Design Specification". Docutils is implemented in Python_. .. _Docutils: http://docutils.sourceforge.net/ .. _PEPs (Python Enhancement Proposals): http://www.python.org/peps/pep-0012.html .. _PEP 258: docs/peps/pep-0258.html .. _Python: http://www.python.org/ Why is it called "Docutils"? ---------------------------- Docutils is short for "Python Documentation Utilities". The name "Docutils" was inspired by "Distutils", the Python Distribution Utilities architected by Greg Ward, a component of Python's standard library. The earliest known use of the term "docutils" in a Python context was a `fleeting reference`__ in a message by Fred Drake on 1999-12-02 in the Python Doc-SIG mailing list. It was suggested `as a project name`__ on 2000-11-27 on Doc-SIG, again by Fred Drake, in response to a question from Tony "Tibs" Ibbs: "What do we want to *call* this thing?". This was shortly after David Goodger first `announced reStructuredText`__ on Doc-SIG. Tibs used the name "Docutils" for `his effort`__ "to document what the Python docutils package should support, with a particular emphasis on documentation strings". Tibs joined the current project (and its predecessors) and graciously donated the name. For more history of reStructuredText and the Docutils project, see `An Introduction to reStructuredText`_. Please note that the name is "Docutils", not "DocUtils" or "Doc-Utils" or any other variation. .. _An Introduction to reStructuredText: ref/rst/introduction.html __ http://mail.python.org/pipermail/doc-sig/1999-December/000878.html __ http://mail.python.org/pipermail/doc-sig/2000-November/001252.html __ http://mail.python.org/pipermail/doc-sig/2000-November/001239.html __ http://homepage.ntlworld.com/tibsnjoan/docutils/STpy.html Is there a GUI authoring environment for Docutils? -------------------------------------------------- DocFactory_ is under development. It uses wxPython and looks very promising. .. _DocFactory: http://docutils.sf.net/sandbox/gschwant/docfactory/doc/ What is the status of the Docutils project? ------------------------------------------- Although useful and relatively stable, Docutils is experimental code, with APIs and architecture subject to change. Our highest priority is to fix bugs as they are reported. So the latest code from CVS (or `development snapshots`_) is almost always the most stable (bug-free) as well as the most featureful. What is the Docutils project release policy? -------------------------------------------- It ought to be "release early & often", but official releases are a significant effort and aren't done that often. We have automatically-generated `development snapshots`_ which always contain the latest code from CVS. As the project matures, we may formalize on a stable/development-branch scheme, but we're not using anything like that yet. If anyone would like to volunteer as a release coordinator, please `contact the project coordinator`_. .. _development snapshots: http://docutils.sf.net/#development-snapshots .. _contact the project coordinator: mailto:goodger@python.org reStructuredText ================ What is reStructuredText? ------------------------- reStructuredText_ is an easy-to-read, what-you-see-is-what-you-get plaintext markup syntax and parser system. The reStructuredText parser is a component of Docutils_. reStructuredText is a revision and reinterpretation of the StructuredText_ and Setext_ lightweight markup systems. If you are reading this on the web, you can see for yourself. `The source for this FAQ `_ is written in reStructuredText; open it in another window and compare them side by side. `A ReStructuredText Primer `_ and the `Quick reStructuredText `_ user reference are a good place to start. The `reStructuredText Markup Specification `_ is a detailed technical specification. .. _reStructuredText: http://docutils.sourceforge.net/rst.html .. _StructuredText: http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage/ .. _Setext: mirror/setext.html Why is it called "reStructuredText"? ------------------------------------ The name came from a combination of "StructuredText", one of reStructuredText's predecessors, with "re": "revised", "reworked", and "reinterpreted", and as in the ``re.py`` regular expression module. For a detailed history of reStructuredText and the Docutils project, see `An Introduction to reStructuredText`_. What's the standard abbreviation for "reStructuredText"? -------------------------------------------------------- "RST" and "ReST" (or "reST") are both acceptable. Care should be taken with capitalization, to avoid confusion with "REST__", an acronym for "Representational State Transfer". The abbreviations "reSTX" and "rSTX"/"rstx" should **not** be used; they overemphasize reStructuredText's precedessor, Zope's StructuredText. __ http://www.xml.com/pub/a/2002/02/06/rest.html What's the standard filename extension for a reStructuredText file? ------------------------------------------------------------------- It's ".txt". Some people would like to use ".rest" or ".rst" or ".restx", but why bother? ReStructuredText source files are meant to be readable as plaintext, and most operating systems already associate ".txt" with text files. Using a specialized filename extension would require that users alter their OS settings, which is something that many users will not be willing or able to do. Are there any reStructuredText editor extensions? ------------------------------------------------- See `Editor Support for reStructuredText`__. __ http://docutils.sf.net/tools/editors/README.html How can I indicate the document title? Subtitle? ------------------------------------------------- A uniquely-adorned section title at the beginning of a document is treated specially, as the document title. Similarly, a uniquely-adorned section title immediately after the document title becomes the document subtitle. For example:: This is the Document Title ========================== This is the Document Subtitle ----------------------------- Here's an ordinary paragraph. Counterexample:: Here's an ordinary paragraph. This is *not* a Document Title ============================== The "ordinary paragraph" above the section title prevents it from becoming the document title. Another counterexample:: This is not the Document Title, because... =========================================== Here's an ordinary paragraph. ... the title adornment is not unique ===================================== Another ordinary paragraph. How can I represent esoteric characters (e.g. character entities) in a document? -------------------------------------------------------------------------------- For example, say you want an em-dash (XML character entity —, Unicode character ``\u2014``) in your document: use a real em-dash. Insert concrete characters (e.g. type a *real* em-dash) into your input file, using whatever encoding suits your application, and tell Docutils the input encoding. Docutils uses Unicode internally, so the em-dash character is a real em-dash internally. ReStructuredText has no character entity subsystem; it doesn't know anything about XML charents. To Docutils, "—" in input text is 7 discrete characters; no interpretation happens. When writing HTML, the "&" is converted to "&", so in the raw output you'd see "&mdash;". There's no difference in interpretation for text inside or outside inline literals or literal blocks -- there's no character entity interpretation in either case. If you can't use a Unicode-compatible encoding and must rely on 7-bit ASCII, there is a workaround. Files containing character entity set substitution definitions using the "unicode" directive `are available `__ (`tarball `__). A description and instructions for use `are here `__. Thanks to David Priest for the original idea. Incorporating these files into Docutils is on the `to-do list `_. If you insist on using XML-style charents, you'll have to implement a pre-processing system to convert to UTF-8 or something. That introduces complications though; you can no longer *write* about charents naturally; instead of writing "—" you'd have to write "&mdash;". How can I generate backticks using a Scandinavian keyboard? ----------------------------------------------------------- The use of backticks in reStructuredText is a bit awkward with Scandinavian keyboards, where the backtick is a "dead" key. To get one ` character one must press SHIFT-` + SPACE. Unfortunately, with all the variations out there, there's no way to please everyone. For Scandinavian programmers and technical writers, this is not limited to reStructuredText but affects many languages and environments. Possible solutions include * If you have to input a lot of backticks, simply type one in the normal/awkward way, select it, copy and then paste the rest (CTRL-V is a lot faster than SHIFT-` + SPACE). * Use keyboard macros. * Remap the keyboard. The Scandinavian keyboard layout is awkward for other programming/technical characters too; for example, []{} etc. are a bit awkward compared to US keyboards. According to Axel Kollmorgen, Under Windows, you can use the `Microsoft Keyboard Layout Creator `__ to easily map the backtick key to a real backtick (no dead key). took me five minutes to load my default (german) keyboard layout, untick "Dead Key?" from the backtick key properties ("in all shift states"), "build dll and setup package", install the generated .msi, and add my custom keyboard layout via Control Panel > Regional and Language Options > Languages > Details > Add Keyboard layout (and setting it as default "when you start your computer"). If anyone knows of other/better solutions, please `let us know`_. Are there any tools for HTML/XML-to-reStructuredText? (Round-tripping) ----------------------------------------------------------------------- People have tossed the idea around, but little if any actual work has ever been done. There's no reason why reStructuredText should not be round-trippable to/from XML; any technicalities which prevent round-tripping would be considered bugs. Whitespace would not be identical, but paragraphs shouldn't suffer. The tricky parts would be the smaller details, like links and IDs and other bookkeeping. For HTML, true round-tripping may not be possible. Even adding lots of extra "class" attributes may not be enough. A "simple HTML" to RST filter is possible -- for some definition of "simple HTML" -- but HTML is used as dumb formatting so much that such a filter may not be particularly useful. No general-purpose filter exists. An 80/20 approach should work though: build a tool that does 80% of the work automatically, leaving the other 20% for manual tweaks. Are there any Wikis that use reStructuredText syntax? ----------------------------------------------------- There are several, with various degrees of completeness. With no implied endorsement or recommendation, and in no particular order: * `Ian Bicking's experimental code `__ * `MoinMoin `__ has some support; `here's a sample `__ * Zope-based `Zwiki `__ * Zope3-based Zwiki (in the Zope 3 source tree as ``zope.products.zwiki``) * `StikiWiki `__ * `Trac `__ `supports using reStructuredText `__ as an alternative to wiki markup. This includes support for `TracLinks `__ from within RST text via a custom RST reference-directive or, even easier, an interpreted text role 'trac' Please `let us know`_ of any other reStructuredText Wikis. The example application for the `Web Framework Shootout `__ article is a Wiki using reStructuredText. Are there any Weblog (Blog) projects that use reStructuredText syntax? ---------------------------------------------------------------------- With no implied endorsement or recommendation, and in no particular order: * `Python Desktop Server `__ * `PyBloxsom `__ * `rst2ht `__ * `htmlnav `__ * `restblog `__ * `ReSTWeb `__ * `Lino WebMan `__ Please `let us know`_ of any other reStructuredText Blogs. Can lists be indented without generating block quotes? ------------------------------------------------------ Some people like to write lists with indentation, without intending a block quote context, like this:: paragraph * list item 1 * list item 2 There has been a lot of discussion about this, but there are some issues that would need to be resolved before it could be implemented. There is a summary of the issues and pointers to the discussions in `the to-do list `__. Could the requirement for blank lines around lists be relaxed? -------------------------------------------------------------- Short answer: no. In reStructuredText, it would be impossible to unambigously mark up and parse lists without blank lines before and after. Deeply nested lists may look ugly with so many blank lines, but it's a price we pay for unambiguous markup. Some other plaintext markup systems do not require blank lines in nested lists, but they have to compromise somehow, either accepting ambiguity or requiring extra complexity. For example, `Epytext `__ does not require blank lines around lists, but it does require that lists be indented and that ambiguous cases be escaped. How can I include mathematical equations in documents? ------------------------------------------------------ There is no elegant built-in way, yet. There are several ideas, but no obvious winner. This issue requires a champion to solve the technical and aesthetic issues and implement a generic solution. Here's the `to-do list entry `__. There are several quick & dirty ways to include equations in documents: * For HTML there is MathML but its rendering is still quite broken (or absent) on most browsers. An example of raw MathML is `here `__. * If you use LaTeX output only, a simple way is to use the "raw" directive; examples are `here `__. * Images of the equations can be used, typically generated by TeX. Beni Cherniavsky has written some `preprocessing scripts`__ for making inline TeX (for LaTeX and other [HTML etc.] output) math much easier; `instructions here`__. __ http://docutils.sourceforge.net/sandbox/cben/rolehack/ __ http://docutils.sourceforge.net/sandbox/cben/rolehack/README.html * Here is an example of potential `itex math markup `__. Is nested inline markup possible? --------------------------------- Not currently, no. It's on the `to-do list`__ (`details here`__), and hopefully will be part of the reStructuredText parser soon. At that time, markup like this will become possible:: Here is some *emphasized text containing a `hyperlink`_ and ``inline literals``*. __ http://docutils.sf.net/docs/dev/todo.html#nested-inline-markup __ http://docutils.sf.net/docs/dev/rst/alternatives.html#nested-inline-markup There are workarounds, but they are either convoluted or ugly or both. They are not recommended. * Inline markup can be combined with hyperlinks using `substitution definitions`__ and references__ with the `"replace" directive`__. For example:: Here is an |emphasized hyperlink|_. .. |emphasized hyperlink| replace:: *emphasized hyperlink* .. _emphasized hyperlink: http://example.org It is not possible for just a portion of the replacement text to be a hyperlink; it's the entire replacement text or nothing. __ http://docutils.sf.net/docs/ref/rst/restructuredtext.html#substitution-definitions __ http://docutils.sf.net/docs/ref/rst/restructuredtext.html#substitution-references __ http://docutils.sf.net/docs/ref/rst/directives.html#replace * The `"raw" directive`__ can be used to insert raw HTML into HTML output:: Here is some |stuff|. .. |stuff| raw:: html emphasized text containing a hyperlink and inline literals Raw LaTeX is supported for LaTeX output, etc. __ http://docutils.sf.net/docs/ref/rst/directives.html#raw How to indicate a line break or a significant newline? ------------------------------------------------------ The `"line-block" directive`__ is designed for address blocks, verse, and other cases where line breaks are significant and must be preserved. Unlike literal blocks, the typeface is not changed, and inline markup is recognized. For example:: .. line-block:: A one, two, a one two three four Half a bee, philosophically, must, *ipso facto*, half not be. But half the bee has got to be, *vis a vis* its entity. D'you see? But can a bee be said to be or not to be an entire bee, when half the bee is not a bee, due to some ancient injury? Singing... __ http://docutils.sf.net/docs/ref/rst/directives.html#line-block In the near future, we will be implementing `new syntax for line blocks`__, like this:: | Lend us a couple of bob till Thursday. | I'm absolutely skint. | But I'm expecting a postal order and I can pay you back as soon as it comes. | Love, Ewan. __ http://docutils.sf.net/docs/dev/rst/alternatives.html#syntax-for-line-blocks Here's a workaround for manually inserting explicit line breaks in HTML output:: .. |br| raw:: html
I want to break this line here: |br| this is after the break. If the extra whitespace bothers you, |br|\ backslash-escape it. A URL containing asterisks doesn't work. What to do? ----------------------------------------------------- Asterisks are valid URL characters (see :RFC:`2396`), sometimes used in URLs. For example:: http://cvs.example.org/viewcvs.py/*checkout*/module/file Unfortunately, the parser thinks the asterisks are indicating emphasis. The slashes serve as delineating punctuation, allowing the asterisks to be recognized as markup. The example above is separated by the parser into a truncated URL, an emphasized word, and some regular text:: http://cvs.example.org/viewcvs.py/ *checkout* /module/file To turn off markup recognition, use a backslash to escape at least the first asterisk, like this:: http://cvs.example.org/viewcvs.py/\*checkout*/module/file Escaping the second asterisk doesn't hurt, but it isn't necessary. HTML Writer =========== What is the status of the HTML Writer? -------------------------------------- The HTML Writer module, ``docutils/writers/html4css1.py``, is a proof-of-concept reference implementation. While it is a complete implementation, some aspects of the HTML it produces may be incompatible with older browsers or specialized applications (such as web templating). Alternate implementations are welcome. What kind of HTML does it produce? ---------------------------------- It produces XHTML compatible with the `HTML 4.01`_ and `XHTML 1.0`_ specifications. A cascading style sheet ("default.css" by default) is required for proper viewing with a modern graphical browser. Correct rendering of the HTML produced depends on the CSS support of the browser. .. _HTML 4.01: http://www.w3.org/TR/html4/ .. _XHTML 1.0: http://www.w3.org/TR/xhtml1/ What browsers are supported? ---------------------------- No specific browser is targeted; all modern graphical browsers should work. Some older browsers, text-only browsers, and browsers without full CSS support are known to produce inferior results. Mozilla (version 1.0 and up) and MS Internet Explorer (version 5.0 and up) are known to give good results. Reports of experiences with other browsers are welcome. Unexpected results from tools/html.py: H1, H1 instead of H1, H2. Why? ---------------------------------------------------------------------- Here's the question in full: I have this text:: Heading 1 ========= All my life, I wanted to be H1. Heading 1.1 ----------- But along came H1, and so shouldn't I be H2? No! I'm H1! Heading 1.1.1 ************* Yeah, imagine me, I'm stuck at H3! No?!? When I run it through tools/html.py, I get unexpected results (below). I was expecting H1, H2, then H3; instead, I get H1, H1, H2:: ... ... Heading 1

Heading 1

<-- first H1

All my life, I wanted to be H1.

Heading 1.1

<-- H1

But along came H1, and so now I must be H2.

Heading 1.1.1

Yeah, imagine me, I'm stuck at H3!

... What gives? Check the "class" attribute on the H1 tags, and you will see a difference. The first H1 is actually ``

``; this is the document title, and the default stylesheet renders it centered. There can also be an ``

`` for the document subtitle. If there's only one highest-level section title at the beginning of a document, it is treated specially, as the document title. (Similarly, a lone second-highest-level section title may become the document subtitle.) See `How can I indicate the document title? Subtitle?`_ for details. Rather than use a plain H1 for the document title, we use ``

`` so that we can use H1 again within the document. Why do we do this? HTML only has H1-H6, so by making H1 do double duty, we effectively reserve these tags to provide 6 levels of heading beyond the single document title. HTML is being used for dumb formatting for nothing but final display. A stylesheet *is required*, and one is provided: tools/stylesheets/default.css. Of course, you're welcome to roll your own. The default stylesheet provides rules to format ``

`` and ``

`` differently from ordinary ``

`` and ``

``:: h1.title { text-align: center } h2.subtitle { text-align: center } If you don't want the top section heading to be interpreted as a title at all, disable the `doctitle_xform`_ setting (``--no-doc-title`` option). This will interpret your document differently from the standard settings, which might not be a good idea. If you don't like the reuse of the H1 in the HTML output, you can tweak the `initial_header_level`_ setting (``--initial-header-level`` option) -- but unless you match its value to your specific document, you might end up with bad HTML (e.g. H3 without H2). .. _doctitle_xform: docs/user/config.html#doctitle-xform .. _initial_header_level: docs/user/config.html#initial-header-level (Thanks to Mark McEahern for the question and much of the answer.) Why do enumerated lists only use numbers (no letters or roman numerals)? ------------------------------------------------------------------------ The rendering of enumerators (the numbers or letters acting as list markers) is completely governed by the stylesheet, so either the browser can't find the stylesheet (try using the "--embed-stylesheet" option), or the browser can't understand it (try a recent Firefox, Mozilla, Konqueror, Opera, Safari, or even MSIE). Python Source Reader ==================== Can I use Docutils for Python auto-documentation? ------------------------------------------------- Yes, in conjunction with other projects. Docstring extraction functionality from within Docutils is still under development. There is most of a source code parsing module in docutils/readers/python/moduleparser.py. We do plan to finish it eventually. Ian Bicking wrote an initial front end for the moduleparser.py module, in sandbox/ianb/extractor/extractor.py. Ian also did some work on the Python Source Reader (docutils.readers.python) component at PyCon DC 2004. Version 2.0 of Ed Loper's `Epydoc `_ supports reStructuredText-format docstrings for HTML output. Docutils 0.3 or newer is required. Development of a Docutils-specific auto-documentation tool will continue. Epydoc works by importing Python modules to be documented, whereas the Docutils-specific tool, described above, will parse modules without importing them (as with `HappyDoc `_, which doesn't support reStructuredText). The advantages of parsing over importing are security and flexibility; the disadvantage is complexity/difficulty. * Security: untrusted code that shouldn't be executed can be parsed; importing a module executes its top-level code. * Flexibility: comments and unofficial docstrings (those not supported by Python syntax) can only be processed by parsing. * Complexity/difficulty: it's a lot harder to parse and analyze a module than it is to ``import`` and analyze one. For more details, please see "Docstring Extraction Rules" in `PEP 258`_, item 3 ("How"). Miscellaneous ============= Is the Docutils document model based on any existing XML models? ---------------------------------------------------------------- Not directly, no. It borrows bits from DocBook, HTML, and others. I (David Goodger) have designed several document models over the years, and have my own biases. The Docutils document model is designed for simplicity and extensibility, and has been influenced by the needs of the reStructuredText markup.


==================
 Docutils History
==================

:Author: David Goodger; open to all Docutils developers
:Contact: goodger@python.org
:Date: $Date: 2004/06/27 03:48:03 $
:Web site: http://docutils.sourceforge.net/
:Copyright: This document has been placed in the public domain.

.. contents::

Changes Since 0.3.3
===================

General:

* _`Documentation cleanup/reorganization`.

  - Created new subdirectories of docs/:

    * ``docs/user/``: introductory/tutorial material for end-users
    * ``docs/dev/``: for core-developers (development notes, plans, etc.)
    * ``docs/api/``: API reference material for client-developers
    * ``docs/ref/``: reference material for all groups
    * ``docs/howto/``: for component-developers and core-developers
    * ``docs/peps/``: Python Enhancement Proposals

  - Moved ``docs/*`` to ``docs/user/``.
  - Moved ``pysource.dtd``, ``pysource.txt``, ``semantics.txt`` from
    ``spec/`` to ``docs/dev``.
  - Moved ``doctree.txt``, ``docutils.dtd``, ``soextblx.dtd``,
    ``transforms.txt`` from ``spec/`` to ``docs/ref/``.
  - Moved ``alternatives.txt``, and ``problems.txt`` from
    ``spec/rst/`` to ``docs/dev/rst/``.
  - Moved ``reStructuredText.txt``, ``directives.txt``,
    ``interpreted.txt``, and ``introduction.txt`` from ``spec/rst/``
    to ``docs/ref/rst/``.  Renamed ``interpreted.txt`` to
    ``roles.txt``, ``reStructuredText.txt`` to
    ``restructuredtext.txt``.
  - Moved ``spec/howto/`` to ``docs/howto``.

  In order to keep the CVS history of moved files, we supplied
  SourceForge with a `script for modifying the Docutils CVS
  repository`__.

  __ http://cvs.sourceforge.net/viewcvs.py/*checkout*/docutils/sandbox/davidg/infrastructure/cvs-reorg.sh?content-type=text/plain&rev=1.5

  After running the cleanup script:

  - Added ``docs/index.txt``.
  - Added a ``.htaccess`` file to the ``web`` module, containing
    redirects for all old paths to new paths.  They'll preserve
    fragments (the "#name" part of a URL), and won't clutter up the
    file system, and will correct the URL in the user's browser.
  - Added ``BUGS.txt``, ``docs/dev/policies.txt``,
    ``docs/dev/website.txt``, ``docs/dev/release.txt`` from all but
    the "To Do" list itself in ``docs/dev/todo.txt``.
  - Moved "Future Plans" from ``HISTORY.txt`` to new "Priorities"
    section of ``docs/dev/todo.txt``.
  - Added ``THANKS.txt`` from "Acknowledgements" in ``HISTORY.txt``.
  - Added "How To Report Bugs" to ``BUGS.txt``.
  - Went through all the sources and docs (including under web/) and
    updated links.  Mostly done by Felix Wiemann; thanks Felix!
    (Still need to update links in the sandboxes.)

Specific:

* BUGS.txt: Added to project.

* THANKS.txt: Added to project.

* docutils/__init__.py:

  - 0.3.4: Post-release.

* docutils/core.py:

  - Added special error handling & advice for UnicodeEncodeError.
  - Refactored Publisher.publish (simplified exception handling &
    extracted debug dumps).
  - Renamed "enable_exit" parameter of convenience functions to
    "enable_exit_status".
  - Enabled traceback (exception propagation) by default in
    programmatic convenience functions.
  - Now publish_file and publish_cmdline convenience functions return
    the encoded string results in addition to their regular I/O.
  - Extracted common code from publish_file, publish_string, and
    publish_parts, into new publish_programmatically.  Extracted
    settings code to ``Publisher.process_programmatic_settings``.
  - In Publisher.publish, disabled ``settings_overrides`` when
    ``settings`` is supplied; redundant.

* docutils/frontend.py:

  - Added help text for "--output-encoding-error-handler" and
    "--error-encoding-error-handler".
  - Renamed "--exit" to "--exit-status".
  - Simplified default-setting code.

* docutils/parsers/rst/__init__.py:

  - Added "--pep-base-url" and "--rfc-base-url" options.

* docutils/parsers/rst/directives/__init__.py:

  - Added several directive option conversion functions.

* docutils/parsers/rst/directives/body.py:

  - Moved "table" directive to tables.py.

* docutils/parsers/rst/directives/tables.py: Table-related directives,
  added to project.

* docutils/writers/latex2e.py:

  - Added "--table-style=(standard|booktabs|nolines)"
  - figures get "here" option (LaTeX per default puts them at bottom),
    and figure content is centered.
  - Rowspan support for tables.
  - Fix: admonition titles before first section.
  - Replace ``--`` in literal by ``-{}-`` because fontencoding T1 has endash.
  - Replave ``_`` in literal by an underlined blank, because it has the correct
    width.
  - Fix: encode pdfbookmark titles, ``#`` broke pdflatex.
  - A few unicode replacements, if output_encoding != utf

* docs/index.txt: "Docutils Project Documentation Overview", added to
  project.

* docs/api/cmdline-tool.txt: "Inside A Docutils Command-Line Front-End
  Tool", added to project.

* docs/api/publisher.txt: "The Docutils Publisher", added to project.

* docs/api/runtime-settings.txt: "Docutils Runtime Settings", added to project.

* docs/dev/policies.txt: Added to project (extracted from
  ``docs/dev/todo.txt``, formerly ``spec/notes.txt``).

* docs/dev/release.txt: Added to project (extracted from
  ``docs/dev/todo.txt``, formerly ``spec/notes.txt``).

* docs/dev/testing.txt: Added to project.

* docs/dev/website.txt: Added to project (extracted from
  ``docs/dev/todo.txt``, formerly ``spec/notes.txt``).

* docs/ref/rst/directives.txt:

  - Added directives: "table", "csv-table".

* docs/user/rst/cheatsheet.txt: "The reStructuredText Cheat Sheet"
  added to project.  1 page for syntax, and a 1 page reference for
  directives and roles.  Source text to be used as-is; not meant to be
  converted to HTML.

* test/functional/, contents, and test/test_functional.py: Added to
  project.

Release 0.3.3 (2004-05-09)
==========================

* docutils/__init__.py:

  - 0.3.1: Reorganized config file format (multiple sections); see
    docs/config.txt.
  - Added unknown_reference_resolvers attribute to TransformSpec.
  - 0.3.2: Interpreted text reorganization.
  - 0.3.3: Released.

* docutils/core.py:

  - Catch system messages to stop tracebacks from parsing errors.
  - Catch exceptions during processing report & exit without
    tracebacks, except when "--traceback" used.
  - Reordered components for OptionParser; application comes last.
  - Added "config_section" parameter to several methods and functions,
    allowing front ends to easily specify their config file sections.
  - Added publish_parts convenience function to allow access to individual
    parts of a document.

* docutils/examples.py: Added to project; practical examples of
  Docutils client code, to be used as-is or as models for variations.

* docutils/frontend.py:

  - Added "--traceback" & "--no-traceback" options ("traceback"
    setting).
  - Implemented support for config file reorganization:
    ``standard_config_files`` moved from ``ConfigParser`` to
    ``OptionParser``; added
    ``OptionParser.get_config_file_settings()`` and
    ``.get_standard_config_settings()``; support for old "[options]"
    section (with deprecation warning) and mapping from old to new
    settings.
  - Reimplemented setting validation.
  - Enabled flexible boolean values: yes/no, true/false, on/off.
  - Added ``Values``, a subclass of ``optparse.Values``, with support
    for list setting attributes.
  - Added support for new ``DOCUTILSCONFIG`` environment variable;
    thanks to Beni Cherniavsky.
  - Added "--no-section-numbering" option.

* docutils/io.py:

  - Catch IOErrors when opening source & destination files, report &
    exit without tracebacks.  Added ``handle_io_errors`` parameter to
    ``FileInput`` & ``FileOutput`` to enable caller error handling.

* docutils/nodes.py:

  - Changed ``SparseNodeVisitor`` and ``GenericNodeVisitor`` dynamic
    method definitions (via ``exec``) to dynamic assignments (via
    ``setattr``); thanks to Roman Suzi.
  - Encapsulated visitor dynamic assignments in a function; thanks to
    Ian Bicking.
  - Added indirect_reference_name attribute to the Targetable
    class. This attribute holds the whitespace_normalized_name
    (contains mixed case) of a target.

* docutils/statemachine.py:

  - Renamed ``StringList.strip_indent`` to ``.trim_left``.
  - Added ``StringList.get_2D_block``.

* docutils/utils.py:

  - Added "level" attribute to SystemMessage exceptions.

* docutils/languages/af.py: Added to project; Afrikaans mappings by
  Jannie Hofmeyr.

* docutils/languages/cs.py: Added to project; Czech mappings by Marek
  Blaha.

* docutils/languages/eo.py: Added to project; Esperanto mappings by
  Marcelo Huerta San Martin.

* docutils/languages/pt_br.py: Added to project; Brazilian Portuguese
  mappings by Lalo Martins.

* docutils/languages/ru.py: Added to project; Russian mappings by
  Roman Suzi.

* docutils/parsers/rst/roles.py: Added to project.  Contains
  interpreted text role functions, a registry for interpreted text
  roles, and an API for adding to and retrieving from the registry.
  Contributed by Edward Loper.

* docutils/parsers/rst/states.py:

  - Updated ``RSTState.nested_parse`` for "include" in table cells.
  - Allowed true em-dash character and "---" as block quote
    attribution marker.
  - Added support for <angle-bracketed> complex option arguments
    (option lists).
  - Fixed handling of backslashes in substitution definitions.
  - Fixed off-by-1 error with extra whitespace after substitution
    definition directive.
  - Added inline markup parsing to field lists' field names.
  - Added support for quoted (and unindented) literal blocks.
    Driven in part by a bribe from Frank Siebenlist (thanks!).
  - Parser now handles escapes in URIs correctly.
  - Made embedded-URIs' reference text omittable.  Idea from Beni
    Cherniavsky.
  - Refactored explicit target processing code.
  - Added name attribute to references containing the reference name only
    through whitespace_normalize_name (no case changes).
  - parse_target no longer returns the refname after going through
    normalize_name. This is now handled in make_target.
  - Fixed bug relating to role-less interpreted text in non-English
    contexts.
  - Reorganized interpreted text processing; moved code into the new
    roles.py module.  Contributed by Edward Loper.
  - Refactored ``Body.parse_directive`` into ``run_directive`` and
    ``parse_directive_block``.

* docutils/parsers/rst/tableparser.py:

  - Reworked for ``StringList``, to support "include" directives in
    table cells.

* docutils/parsers/rst/directives/__init__.py:

  - Renamed ``unchanged()`` directive option conversion function to
    ``unchanged_required``, and added a new ``unchanged``.
  - Catch unicode value too high error; fixes bug 781766.
  - Beefed up directive error reporting.

* docutils/parsers/rst/directives/body.py:

  - Added basic "table" directive.

* docutils/parsers/rst/directives/images.py:

  - Added "target" option to "image" directive.
  - Added name attribute to references containing the reference name only
    through whitespace_normalize_name (no case changes).

* docutils/parsers/rst/directives/misc.py:

  - Isolated the import of the ``urllib2`` module; was causing
    problems on SourceForge (``libssl.so.2`` unavailable?).
  - Added the "role" directive for declaring custom interpreted text
    roles.

* docutils/parsers/rst/directives/parts.py:

  - The "contents" directive does more work up-front, creating the
    "topic" and "title", and leaving the "pending" node for the
    transform.  Allows earlier reference resolution; fixes subtle bug.

* docutils/parsers/rst/languages/af.py: Added to project; Afrikaans
  mappings by Jannie Hofmeyr.

* docutils/parsers/rst/languages/cs.py: Added to project; Czech
  mappings by Marek Blaha.

* docutils/parsers/rst/languages/eo.py: Added to project; Esperanto
  mappings by Marcelo Huerta San Martin.

* docutils/parsers/rst/languages/pt_br.py: Added to project; Brazilian
  Portuguese mappings by Lalo Martins.

* docutils/parsers/rst/languages/ru.py: Added to project; Russian
  mappings by Roman Suzi.

* docutils/transforms/parts.py:

  - The "contents" directive does more work up-front, creating the
    "topic" and "title", and leaving the "pending" node for the
    transform.  Allows earlier reference resolution; fixes subtle bug.
  - Added support for disabling of section numbering.

* docutils/transforms/references.py:

  - Verifying that external targets are truly targets and not indirect
    references. This is because we are now adding a "name" attribute to
    references in addition to targets. Note sure if this is correct!
  - Added code to hook into the unknown_reference_resolvers list for a
    transformer in resolve_indirect_target. This allows the
    unknown_reference_resolvers to keep around indirect targets which
    docutils doesn't know about.
  - Added specific error message for duplicate targets.

* docutils/transforms/universal.py:

  - Added FilterMessages transform (removes system messages below the
    verbosity threshold).
  - Added hook (via docutils.TransformSpec.unknown_reference_resolvers)
    to FinalCheckVisitor for application-specific handling of
    unresolvable references.
  - Added specific error message for duplicate targets.

* docutils/writers/__init__.py:

  - Added assemble_parts method to the Writer class to allow for
    access to a documents individual parts.
  - Documented & set default for ``Writer.output`` attribute.

* docutils/writers/html4css1.py:

  - Fixed unicode handling of attribute values (bug 760673).
  - Prevent duplication of "class" attribute values (bug report from
    Kirill Lapshin).
  - Improved table grid/border handling (prompted by report from Bob
    Marshall).
  - Added support for table titles.
  - Added "<title />" for untitled docs, for XHTML conformance; thanks
    to Darek Suchojad.
  - Added functionality to keep track of individual parts of a document
    and store them in a dictionary as the "parts" attribute of the writer.
    Contributed by Reggie Dugard at the Docutils sprint at PyCon DC 2004.
  - Added proper support for the "scale" attribute of the "image"
    element.  Contributed by Brent Cook.
  - Added ``--initial-header-level`` option.
  - Fixed bug: the body_pre_docinfo segment depended on there being a
    docinfo; if no docinfo, the document title was incorporated into
    the body segment.  Adversely affected the publish_parts interface.

* docutils/writers/latex2e.py:

  - Changed default stylesheet to "no stylesheet" to avoid latex complaining
    about a missing file.
  - Added options and support: ``--compound-enumerators``,
    ``--section-prefix-for-enumerators``, and
    ``--section-enumerator-separator``.  By John F Meinel Jr (SF patch
    934322).
  - Added option ``--use-verbatim-when-possible``, to avoid
    problematic characters (eg, '~' in italian) in literal blocks.
  - It's now possible to use four section levels in the `book` and
    `report` LaTeX classes.  The default `article` class still has
    three levels limit.

* docs/config.txt: "Docutils Configuration Files", added to project.
  Moved config file entry descriptions from tools.txt.

* docs/tools.txt:

  - Moved config file entry descriptions to config.txt.

* spec/notes.txt: Continual updates.  Added "Setting Up For Docutils
  Development".

* spec/howto/rst-roles.txt: "Creating reStructuredText Interpreted
  Text Roles", added to project.

* spec/rst/reStructuredText.txt:

  - Added description of support for <angle-bracketed> complex option
    arguments to option lists.
  - Added subsections for indented and quoted literal blocks.

* test: Continually adding & updating tests.

  - Added test/test_settings.py & test/data/config_*.txt support
    files.
  - Added test/test_writers/test_htmlfragment.py.

* test/DocutilsTestSupport.py:

  - Refactored LaTeX publisher test suite/case class names to make
    testing other writers easier.
  - Added HtmlWriterPublishTestCase and HtmlFragmentTestSuite classes
    to test the processing of HTML fragments which use the new
    publish_parts convenience function.

* tools/buildhtml.py:

  - Added support for the "--prune" option.
  - Removed dependency on pep2html.py; plaintext PEPs no longer
    supported.

* tools/docutils.conf:

  - Updated for configuration file reorganization.

* tools/rst2html.py:

  - copied from tools/html.py

* setup.py:

  - added a 'scripts' section to configuration
  - added 'tools/rst2html.py' to the scripts section

Release 0.3 (2003-06-24)
========================

General:

* Renamed "attribute" to "option" for directives/extensions.

* Renamed transform method "transform" to "apply".

* Renamed "options" to "settings" for runtime settings (as set by
  command-line options).  Sometimes "option" (singular) became
  "settings" (plural).  Some variations below:

  - document.options -> document.settings (stored in other objects as
    well)
  - option_spec -> settings_spec (not directives though)
  - OptionSpec -> SettingsSpec
  - cmdline_options -> settings_spec
  - relative_path_options -> relative_path_settings
  - option_default_overrides -> settings_default_overrides
  - Publisher.set_options -> Publisher.get_settings

Specific:

* COPYING.txt: Added "Public Domain Dedication".

* FAQ.txt: Frequently asked questions, added to project.

* setup.py:

  - Updated with PyPI Trove classifiers.
  - Conditional installation of third-party modules.

* docutils/__init__.py:

  - Bumped version to 0.2.1 to reflect changes to I/O classes.
  - Bumped version to 0.2.2 to reflect changes to stylesheet options.
  - Factored ``SettingsSpec`` out of ``Component``; separately useful.
  - Bumped version to 0.2.3 because of the new "--embed-stylesheet"
    option and its effect on the PEP template & writer.
  - Bumped version to 0.2.4 due to changes to the PEP template &
    stylesheet.
  - Bumped version to 0.2.5 to reflect changes to Reporter output.
  - Added ``TransformSpec`` class for new transform system.
  - Bumped version to 0.2.6 for API changes (renaming).
  - Bumped version to 0.2.7 for new ``docutils.core.publish_*``
    convenience functions.
  - Added ``Component.component_type`` attribute.
  - Bumped version to 0.2.8 because of the internal parser switch from
    plain lists to the docutils.statemachine.StringList objects.
  - Bumped version to 0.2.9 because of the frontend.py API changes.
  - Bumped version to 0.2.10 due to changes to the project layout
    (third-party modules removed from the "docutils" package), and
    signature changes in ``io.Input``/``io.Output``.
  - Changed version to 0.3.0 for release.

* docutils/core.py:

  - Made ``publish()`` a bit more convenient.
  - Generalized ``Publisher.set_io``.
  - Renamed ``publish()`` to ``publish_cmdline()``; rearranged its
    parameters; improved its docstring.
  - Added ``publish_file()`` and ``publish_string()``.
  - Factored ``Publisher.set_source()`` and ``.set_destination()``
    out of ``.set_io``.
  - Added support for "--dump-pseudo-xml", "--dump-settings", and
    "--dump-transforms" hidden options.
  - Added ``Publisher.apply_transforms()`` method.
  - Added ``Publisher.set_components()`` method; support for
    ``publish_*()`` conveninece functions.
  - Moved config file processing to docutils/frontend.py.
  - Added support for exit status ("exit_level" setting &
    ``enable_exit`` parameter for Publisher.publish() and convenience
    functions).

* docutils/frontend.py:

  - Check for & exit on identical source & destination paths.
  - Fixed bug with absolute paths & "--config".
  - Set non-command-line defaults in ``OptionParser.__init__()``:
    ``_source`` & ``_destination``.
  - Distributed ``relative_path_settings`` to components; updated
    ``OptionParser.populate_from_components()`` to combine it all.
  - Require list of keys in ``make_paths_absolute`` (was implicit in
    global ``relative_path_settings``).
  - Added "--expose-internal-attribute", "--dump-pseudo-xml",
    "--dump-settings", and "--dump-transforms" hidden options.
  - Removed nasty internals-fiddling ``ConfigParser.get_section``
    code, replaced with correct code.
  - Added validation functionality for config files.
  - Added "--error-encoding" option/setting, "_disable_config"
    internal setting.
  - Added encoding validation; updated "--input-encoding" and
    "--output-encoding"; added "--error-encoding-error-handler" and
    "--output-encoding-error-handler".
  - Moved config file processing from docutils/core.py.
  - Updated ``OptionParser.populate_from_components`` to handle new
    ``SettingsSpec.settings_defaults`` dict.
  - Added support for "-" => stdin/stdout.
  - Added "exit_level" setting ("--exit" option).

* docutils/io.py:

  - Split ``IO`` classes into subclasses of ``Input`` and ``Output``.
  - Added automatic closing to ``FileInput`` and ``FileOutput``.
  - Delayed opening of ``FileOutput`` file until ``write()`` called.
  - ``FileOutput.write()`` now returns the encoded output string.
  - Try to get path/stream name automatically in ``FileInput`` &
    ``FileOutput``.
  - Added defaults for source & destination paths.
  - Allow for Unicode I/O with an explicit "unicode" encoding.
  - Added ``Output.encode()``.
  - Removed dependency on runtime settings; pass encoding directly.
  - Recognize Unicode strings in ``Input.decode()``.
  - Added support for output encoding error handlers.

* docutils/nodes.py:

  - Added "Invisible" element category class.
  - Changed ``Node.walk()`` & ``.walkabout()`` to permit more tree
    modification during a traversal.
  - Added element classes: ``line_block``, ``generated``, ``address``,
    ``sidebar``, ``rubric``, ``attribution``, ``admonition``,
    ``superscript``, ``subscript``, ``inline``
  - Added support for lists of nodes to ``Element.insert()``.
  - Fixed parent linking in ``Element.replace()``.
  - Added new abstract superclass ``FixedTextElement``; adds
    "xml:space" attribute.
  - Added support for "line" attribute of ``system_message`` nodes.
  - Added support for the observer pattern from ``utils.Reporter``.
    Added ``parse_messages`` and ``transform_messages`` attributes to
    ``document``, removed ``messages``.  Added ``note_parse_message``
    and ``note_transform_message`` methods.
  - Added support for improved diagnostics:

    - Added "document", "source", and "line" internal attributes to
      ``Node``, set by ``Node.setup_child()``.
    - Converted variations on ``node.parent = self`` to
      ``self.setup_child(node)``.
    - Added ``document.current_source`` & ``.current_line``
      attributes, and ``.note_source`` observer method.
    - Changed "system_message" output to GNU-Tools format.

  - Added a "rawsource" attribute to the ``Text`` class, for text
    before backslash-escape resolution.
  - Support for new transform system.
  - Reworked ``pending`` element.
  - Fixed XML DOM bug (SF #660611).
  - Removed the ``interpeted`` element class and added
    ``title_reference``, ``abbreviation``, ``acronym``.
  - Made substitutions case-sensitive-but-forgiving; moved some code
    from the parser.
  - Fixed Unicode bug on element attributes (report: William Dode).

* docutils/optik.py: Removed from project; replaced with
  extras/optparse.py and extras/textwrap.py.  These will be installed
  only if they're not already present in the Python installation.

* docutils/roman.py: Moved to extras/roman.py; this will be installed
  only if it's not already present in the Python installation.

* docutils/statemachine.py:

  - Factored out ``State.add_initial_transitions()`` so it can be
    extended.
  - Converted whitespace-specific "blank" and "indent" transitions
    from special-case code to ordinary transitions: removed
    ``StateMachineWS.check_line()`` & ``.check_whitespace()``, added
    ``StateWS.add_initial_transitions()`` method, ``ws_patterns`` &
    ``ws_initial_transitions`` attributes.
  - Removed ``State.match_transition()`` after merging it into
    ``.check_line()``.
  - Added ``StateCorrection`` exception.
  - Added support for ``StateCorrection`` in ``StateMachine.run()``
    (moved ``TransitionCorrection`` support there too.)
  - Changed ``StateMachine.next_line()`` and ``.goto_line()`` to raise
    ``EOFError`` instead of ``IndexError``.
  - Added ``State.no_match`` method.
  - Added support for the Observer pattern, triggered by input line
    changes.
  - Added ``strip_top`` parameter to
    ``StateMachineWS.get_first_known_indented``.
  - Made ``context`` a parameter to ``StateMachine.run()``.
  - Added ``ViewList`` & ``StringList`` classes;
    ``extract_indented()`` becomes ``StringList.get_indented()``.
  - Added ``StateMachine.insert_input()``.
  - Fixed ViewList slice handling for Python 2.3.  Patch from (and
    thanks to) Fred Drake.

* docutils/utils.py:

  - Added a ``source`` attribute to Reporter instances and
    ``system_message`` elements.
  - Added an observer pattern to ``utils.Reporter`` to keep track of
    system messages.
  - Fixed bugs in ``relative_path()``.
  - Added support for improved diagnostics.
  - Moved ``normalize_name()`` to nodes.py (``fully_normalize_name``).
  - Added support for encoding Reporter stderr output, and encoding
    error handlers.
  - Reporter keeps track of the highest level system message yet
    generated.

* docutils/languages: Fixed bibliographic field language lookups.

* docutils/languages/es.py: Added to project; Spanish mappings by
  Marcelo Huerta San Martin.

* docutils/languages/fr.py: Added to project; French mappings by
  Stefane Fermigier.

* docutils/languages/it.py: Added to project; Italian mappings by
  Nicola Larosa.

* docutils/languages/sk.py: Added to project; Slovak mappings by
  Miroslav Vasko.

* docutils/parser/__init__.py:

  - Added ``Parser.finish_parse()`` method.

* docutils/parser/rst/__init__.py:

  - Added options: "--pep-references", "--rfc-references",
    "--tab-width", "--trim-footnote-reference-space".

* docutils/parsers/rst/states.py:

  - Changed "title under/overline too short" system messages from INFO
    to WARNING, and fixed its insertion location.
  - Fixed enumerated list item parsing to allow paragraphs & section
    titles to begin with enumerators.
  - Converted system messages to use the new "line" attribute.
  - Fixed a substitution reference edge case.
  - Added support for "--pep-references" and "--rfc-references"
    options; reworked ``Inliner`` code to make customization easier.
  - Removed field argument parsing.
  - Added support for short section title over/underlines.
  - Fixed "simple reference name" regexp to ignore text like
    "object.__method__"; not an anonymous reference.
  - Added support for improved diagnostics.
  - Reworked directive API, based on Dethe Elza's contribution.  Added
    ``Body.parse_directive()``, ``.parse_directive_options()``,
    ``.parse_directive_arguments()`` methods.
  - Added ``ExtensionOptions`` class, to parse directive options
    without parsing field bodies.  Factored
    ``Body.parse_field_body()`` out of ``Body.field()``, overridden in
    ``ExtensionOptions``.
  - Improved definition list term/classifier parsing.
  - Added warnings for unknown directives.
  - Renamed ``Stuff`` to ``Struct``.
  - Now flagged as errors: transitions at the beginning or end of
    sections, empty sections (except title), and empty documents.
  - Updated for ``statemachine.StringList``.
  - Enabled recognition of schemeless email addresses in targets.
  - Added support for embedded URIs in hyperlink references.
  - Added backslash-escapes to inline markup end-string suffix.
  - Added support for correct interpreted text processing.
  - Fixed nested title parsing (topic, sidebar directives).
  - Added special processing of backslash-escaped whitespace (idea
    from David Abrahams).
  - Made substitutions case-sensitive-but-forgiving; moved some code
    to ``docutils.nodes``.
  - Added support for block quote attributions.
  - Added a kludge to work-around a conflict between the bubble-up
    parser strategy and short titles (<= 3 char-long over- &
    underlines).  Fixes SF bug #738803 "infinite loop with multiple
    titles" submitted by Jason Diamond.
  - Added explicit interpreted text roles for standard inline markup:
    "emphasis", "strong", "literal".
  - Implemented "superscript" and "subscript" interpreted text roles.
  - Added initial support for "abbreviation" and "acronym" roles;
    incomplete.
  - Added support for "--trim-footnote-reference-space" option.
  - Optional space before colons in directives & hyperlink targets.

* docutils/parsers/rst/tableparser.py:

  - Fixed a bug that was producing unwanted empty rows in "simple"
    tables.
  - Detect bad column spans in "simple" tables.

* docutils/parsers/rst/directives: Updated all directive functions to
  new API.

* docutils/parsers/rst/directives/__init__.py:

  - Added ``flag()``, ``unchanged()``, ``path()``,
    ``nonnegative_int()``, ``choice()``, and ``class_option()``
    directive option helper functions.
  - Added warnings for unknown directives.
  - Return ``None`` for missing directives.
  - Added ``register_directive()``, thanks to William Dode and Paul
    Moore.

* docutils/parsers/rst/directives/admonitions.py:

  - Added "admonition" directive.

* docutils/parsers/rst/directives/body.py: Added to project.  Contains
  the "topic", "sidebar" (from Patrick O'Brien), "line-block",
  "parsed-literal", "rubric", "epigraph", "highlights" and
  "pull-quote" directives.

* docutils/parsers/rst/directives/images.py:

  - Added an "align" attribute to the "image" & "figure" directives
    (by Adam Chodorowski).
  - Added "class" option to "image", and "figclass" to "figure".

* docutils/parsers/rst/directives/misc.py:

  - Added "include", "raw", and "replace" directives, courtesy of
    Dethe Elza.
  - Added "unicode" and "class" directives.

* docutils/parsers/rst/directives/parts.py:

  - Added the "sectnum" directive; by Dmitry Jemerov.
  - Added "class" option to "contents" directive.

* docutils/parsers/rst/directives/references.py: Added to project.
  Contains the "target-notes" directive.

* docutils/parsers/rst/languages/__init__.py:

  - Return ``None`` from get_language() for missing language modules.

* docutils/parsers/rst/languages/de.py: Added to project; German
  mappings by Engelbert Gruber.

* docutils/parsers/rst/languages/en.py:

  - Added interpreted text roles mapping.

* docutils/parsers/rst/languages/es.py: Added to project; Spanish
  mappings by Marcelo Huerta San Martin.

* docutils/parsers/rst/languages/fr.py: Added to project; French
  mappings by William Dode.

* docutils/parsers/rst/languages/it.py: Added to project; Italian
  mappings by Nicola Larosa.

* docutils/parsers/rst/languages/sk.py: Added to project; Slovak
  mappings by Miroslav Vasko.

* docutils/readers/__init__.py:

  - Added support for the observer pattern from ``utils.Reporter``, in
    ``Reader.parse`` and ``Reader.transform``.
  - Removed ``Reader.transform()`` method.
  - Added default parameter values to ``Reader.__init__()`` to make
    instantiation easier.
  - Removed bogus aliases: "restructuredtext" is *not* a Reader.

* docutils/readers/pep.py:

  - Added the ``peps.TargetNotes`` transform to the Reader.
  - Removed PEP & RFC reference detection code; moved to
    parsers/rst/states.py as options (enabled here by default).
  - Added support for pre-acceptance PEPs (no PEP number yet).
  - Moved ``Inliner`` & made it a class attribute of ``Reader`` for
    easy subclassing.

* docutils/readers/python: Python Source Reader subpackage added to
  project, including preliminary versions of:

  - __init__.py
  - moduleparser.py: Parser for Python modules.

* docutils/transforms/__init__.py:

  - Added ``Transformer`` class and completed transform reform.
  - Added unknown_reference_resolvers list for each transformer. This list holds
    the list of functions provided by each component of the transformer that
    help resolve references.

* docutils/transforms/frontmatter.py:

  - Improved support for generic fields.
  - Fixed bibliographic field language lookups.

* docutils/transforms/misc.py: Added to project.  Miscellaneous
  transforms.

* docutils/transforms/parts.py:

  - Moved the "id" attribute from TOC list items to the references
    (``Contents.build_contents()``).
  - Added the ``SectNum`` transform; by Dmitry Jemerov.
  - Added "class" attribute support to ``Contents``.

* docutils/transforms/peps.py:

  - Added ``mask_email()`` function, updating to pep2html.py's
    functionality.
  - Linked "Content-Type: text/x-rst" to PEP 12.
  - Added the ``TargetNotes`` PEP-specific transform.
  - Added ``TargetNotes.cleanup_callback``.
  - Added title check to ``Headers``.

* docutils/transforms/references.py:

  - Added the ``TargetNotes`` generic transform.
  - Split ``Hyperlinks`` into multiple transforms.
  - Fixed bug with multiply-indirect references (report: Bruce Smith).
  - Added check for circular indirect references.
  - Made substitutions case-sensitive-but-forgiving.

* docutils/transforms/universal.py:

  - Added support for the "--expose-internal-attributes" option.
  - Removed ``Pending`` transform classes & data.

* docutils/writers/__init__.py:

  - Removed ``Writer.transform()`` method.

* docutils/writers/docutils-xml.py:

  - Added XML and doctype declarations.
  - Added "--no-doctype" and "--no-xml-declaration" options.

* docutils/writers/html4css1.py:

  - "name" attributes only on these tags: a, applet, form, frame,
    iframe, img, map.
  - Added "name" attribute to <a> in section titles for Netscape 4
    support (bug report: Pearu Peterson).
  - Fixed targets (names) on footnote, citation, topic title,
    problematic, and system_message nodes (for Netscape 4).
  - Changed field names from "<td>" to "<th>".
  - Added "@" to "&#64;" encoding to thwart address harvesters.
  - Improved the vertical whitespace optimization; ignore "invisible"
    nodes (targets, comments, etc.).
  - Improved inline literals with ``<span class="pre">`` around chunks
    of text and ``&nbsp;`` for runs of spaces.
  - Improved modularity of output; added ``self.body_pre_docinfo`` and
    ``self.docinfo`` segments.
  - Added support for "line_block", "address" elements.
  - Improved backlinks (footnotes & system_messages).
  - Improved system_message output.
  - Redefined "--stylesheet" as containing an invariant URL, used
    verbatim.  Added "--stylesheet-path", interpreted w.r.t. the
    working directory.
  - Added "--footnote-references" option (superscript or brackets).
  - Added "--compact-lists" and "--no-compact-lists" options.
  - Added "--embed-stylesheet" and "--link-stylesheet" options;
    factored out ``HTMLTranslator.get_stylesheet_reference()``.
  - Improved field list rendering.
  - Added Docutils version to "generator" meta tag.
  - Fixed a bug with images; they must be inline, so wrapped in <p>.
  - Improved layout of <pre> HTML source.
  - Fixed attribute typo on <colspec>.
  - Refined XML prologue.
  - Support for no stylesheet.
  - Removed "interpreted" element support.
  - Added support for "title_reference", "sidebar", "attribution",
    "rubric", and generic "admonition" elements.
  - Added "--attribution" option.
  - Added support for "inline", "subscript", "superscript" elements.
  - Added initial support for "abbreviation" and "acronym";
    incomplete.

* docutils/writers/latex2e.py: LaTeX Writer, added by Engelbert Gruber
  (from the sandbox).

  - Added french.
  - Double quotes in literal blocks (special treatment for de/ngerman).
  - Added '--hyperlink-color' option ('0' turns off coloring of links).
  - Added  "--attribution" option.
  - Right align attributions.

* docutils/writers/pep_html.py:

  - Parameterized output encoding in PEP template.
  - Reworked substitutions from ``locals()`` into ``subs`` dict.
  - Redefined "--pep-stylesheet" as containing an invariant URL, used
    verbatim.  Added "--pep-stylesheet-path", interpreted w.r.t. the
    working directory.
  - Added an override on the "--footnote-references" option.
  - Factored out ``HTMLTranslator.get_stylesheet_reference()``.
  - Added Docutils version to "generator" meta tag.
  - Added a "DO NOT EDIT THIS FILE" comment to generated HTML.

* docs/tools.txt:

  - Added a "silent" setting for ``buildhtml.py``.
  - Added a "Getting Help" section.
  - Rearranged the structure.
  - Kept up to date, with new settings, command-line options etc.
  - Added section for ``rst2latex.py`` (Engelbert Gruber).
  - Converted settings table into a definition list.

* docs/rst/quickstart.txt:

  - Added a table of contents.
  - Added feedback information.
  - Added mention of minimum section title underline lengths.
  - Removed the 4-character minimum for section title underlines.

* docs/rst/quickref.html:

  - Added a "Getting Help" section.
  - Added a style to make section title backlinks more subtle.
  - Added mention of minimum section title underline lengths.
  - Removed the 4-character minimum for section title underlines.

* extras: Directory added to project; contains third-party modules
  that Docutils depends on (optparse, textwrap, roman).  These are
  only installed if they're not already present.

* licenses: Directory added to project; contains copies of license
  files for non-public-domain files.

* spec/doctree.txt:

  - Changed the focus.  It's about DTD elements:  structural
    relationships, semantics, and external (public) attributes.  Not
    about the element class library.
  - Moved some implementation-specific stuff into ``docutils.nodes``
    docstrings.
  - Wrote descriptions of all common attributes and parameter
    entities.  Filled in introductory material.
  - Working through the element descriptions: 55 down, 37 to go.
  - Removed "Representation of Horizontal Rules" to
    spec/rst/alternatives.txt.

* spec/docutils.dtd:

  - Added "generated" inline element.
  - Added "line_block" body element.
  - Added "auto" attribute to "title".
  - Changed content models of "literal_block" and "doctest_block" to
    ``%text.model``.
  - Added ``%number;`` attribute type parameter entity.
  - Changed ``%structural.elements;`` to ``%section.elements``.
  - Updated attribute types; made more specific.
  - Added "address" bibliographic element.
  - Added "line" attribute to ``system_message`` element.
  - Removed "field_argument" element; "field_name" may contain
    multiple words and whitespace.
  - Changed public identifier to docutils.sf.net.
  - Removed "interpreted" element; added "title_reference",
    "abbreviation", "acronym".
  - Removed "refuri" attribute from "footnote_reference" and
    "citation_reference".
  - Added "sidebar", "rubric", "attribution", "admonition",
    "superscript", "subscript", and "inline" elements.

* spec/pep-0256.txt: Converted to reStructuredText & updated.

* spec/pep-0257.txt: Converted to reStructuredText & updated.

* spec/pep-0258.txt: Converted to reStructuredText & updated.

* spec/semantics.txt: Updated with text from a Doc-SIG response to
  Dallas Mahrt.

* spec/transforms.txt: Added to project.

* spec/howto: Added subdirectory, for developer how-to docs.

* spec/howto/rst-directives.txt: Added to project.  Original by Dethe
  Elza, edited & extended by David Goodger.

* spec/howto/i18n.txt: Docutils Internationalization.  Added to
  project.

* spec/rst/alternatives.txt:

  - Added "Doctree Representation of Transitions" from
    spec/doctree.txt.
  - Updated "Inline External Targets" & closed the debate.
  - Added ideas for interpreted text syntax extensions.
  - Added "Nested Inline Markup" section.

* spec/rst/directives.txt:

  - Added directives: "topic", "sectnum", "target-notes",
    "line-block", "parsed-literal", "include", "replace", "sidebar",
    "admonition", "rubric", "epigraph", "highlights", "unicode" and
    "class".
  - Formalized descriptions of directive details.
  - Added an "align" attribute to the "image" & "figure" directives
    (by Adam Chodorowski).
  - Added "class" options to "topic", "sidebar", "line-block",
    "parsed-literal", "contents", and "image"; and "figclass" to
    "figure".

* spec/rst/interpreted.txt: Added to project.  Descriptions of
  interpreted text roles.

* spec/rst/introduction.txt:

  - Added pointers to material for new users.

* spec/rst/reStructuredText.txt:

  - Disambiguated comments (just add a newline after the "::").
  - Updated enumerated list description; added a discussion of the
    second-line validity checking.
  - Updated directive description.
  - Added a note redirecting newbies to the user docs.
  - Expanded description of inline markup start-strings in non-markup
    contexts.
  - Removed field arguments and made field lists a generic construct.
  - Removed the 4-character minimum for section title underlines.
  - Clarified term/classifier delimiter & inline markup ambiguity
    (definition lists).
  - Added "Embedded URIs".
  - Updated "Interpreted Text" section.
  - Added "Character-Level Inline Markup" section.

* test: Continually adding & updating tests.

  - Moved test/test_rst/ to test/test_parsers/test_rst/.
  - Moved test/test_pep/ to test/test_readers/test_pep/.
  - Added test/test_readers/test_python/.
  - Added test/test_writers/ (Engelbert Gruber).

* tools:

  - Made the ``locale.setlocale()`` calls in front ends
    fault-tolerant.

* tools/buildhtml.py:

  - Added "--silent" option.
  - Fixed bug with absolute paths & "--config".
  - Updated for new I/O classes.
  - Added some exception handling.
  - Separated publishers' setting defaults; prevents interference.
  - Updated for new ``publish_file()`` convenience function.

* tools/pep-html-template:

  - Allow for "--embed-stylesheet".
  - Added Docutils version to "generator" meta tag.
  - Added a "DO NOT EDIT THIS FILE" comment to generated HTML.
  - Conform to XHTML spec.

* tools/pep2html.py:

  - Made ``argv`` a parameter to ``main()``.
  - Added support for "Content-Type:" header & arbitrary PEP formats.
  - Linked "Content-Type: text/plain" to PEP 9.
  - Files skipped (due to an error) are not pushed onto the server.
  - Updated for new I/O classes.
  - Added ``check_requirements()`` & ``pep_type_error()``.
  - Added some exception handling.
  - Updated for new ``publish_string()`` convenience function.
  - Added a "DO NOT EDIT THIS FILE" comment to generated HTML.

* tools/quicktest.py:

  - Added "-V"/"--version" option.

* tools/rst2latex.py: LaTeX front end, added by Engelbert Gruber.

* tools/unicode2rstsubs.py: Added to project.  Produces character
  entity files (reSructuredText substitutions) from the MathML master
  unicode.xml file.

* tools/editors: Support code for editors, added to project.  Contains
  ``emacs/restructuredtext.el``.

* tools/stylesheets/default.css: Moved into the stylesheets directory.

  - Added style for chunks of inline literals.
  - Removed margin for first child of table cells.
  - Right-aligned field list names.
  - Support for auto-numbered section titles in TOCs.
  - Increased the size of inline literals (<tt>) in titles.
  - Restored the light gray background for inline literals.
  - Added support for "line_block" elements.
  - Added style for "address" elements.
  - Removed "a.footnote-reference" style; doing it with ``<sup>`` now.
  - Improved field list rendering.
  - Vertical whitespace improvements.
  - Removed "a.target" style.

* tools/stylesheets/pep.css:

  - Fixed nested section margins.
  - Other changes parallel those of ``../default.css``.

Release 0.2 (2002-07-31)
========================

General:

- The word "component" was being used ambiguously.  From now on,
  "component" will be used to mean "Docutils component", as in Reader,
  Writer, Parser, or Transform.  Portions of documents (Table of
  Contents, sections, etc.)  will be called "document parts".
- Did a grand renaming: a lot of ``verylongnames`` became
  ``very_long_names``.
- Cleaned up imports: no more relative package imports or
  comma-separated lists of top-level modules.
- Added support for an option values object which carries default
  settings and overrides (from command-line options and library use).
- Added internal Unicode support, and support for both input and
  output encodings.
- Added support for the ``docutils.io.IO`` class & subclasses.

Specific:

* docutils/__init__.py:

  - Added ``ApplicationError`` and ``DataError``, for use throughout
    the package.
  - Added ``Component`` base class for Docutils components; implements
    the ``supports`` method.
  - Added ``__version__`` (thus, ``docutils.__version__``).

* docutils/core.py:

  - Removed many keyword parameters to ``Publisher.__init__()`` and
    ``publish()``; bundled into an option values object.  Added
    "argv", "usage", "description", and "option_spec" parameters for
    command-line support.
  - Added ``Publisher.process_command_line()`` and ``.set_options()``
    methods.
  - Reworked I/O model for ``docutils.io`` wrappers.
  - Updated ``Publisher.set_options()``; now returns option values
    object.
  - Added support for configuration files (/etc/docutils.conf,
    ./docutils.conf, ~/.docutils).
  - Added ``Publisher.setup_option_parser()``.
  - Added default usage message and description.

* docutils/frontend.py: Added to project; support for front-end
  (command-line) scripts.  Option specifications may be augmented by
  components.  Requires Optik (http://optik.sf.net/) for option
  processing (installed locally as docutils/optik.py).

* docutils/io.py: Added to project; uniform API for a variety of input
  output mechanisms.

* docutils/nodes.py:

  - Added ``TreeCopyVisitor`` class.
  - Added a ``copy`` method to ``Node`` and subclasses.
  - Added a ``SkipDeparture`` exception for visitors.
  - Renamed ``TreePruningException`` from ``VisitorException``.
  - Added docstrings to ``TreePruningException``, subclasses, and
    ``Nodes.walk()``.
  - Improved docstrings.
  - Added ``SparseNodeVisitor``, refined ``NodeVisitor``.
  - Moved ``utils.id()`` to ``nodes.make_id()`` to avoid circular
    imports.
  - Added ``decoration``, ``header``, and ``footer`` node classes, and
    ``PreDecorative`` mixin.
  - Reworked the name/id bookkeeping; to ``document``, removed
    ``explicit_targets`` and ``implicit_targets`` attributes, added
    ``nametypes`` attribute and ``set_name_id_map`` method.
  - Added ``NodeFound`` exception, for use with ``NodeVisitor``
    traversals.
  - Added ``document.has_name()`` method.
  - Fixed DOM generation for list-attributes.
  - Added category class ``Labeled`` (used by footnotes & citations).
  - Added ``Element.set_class()`` method (sets "class" attribute).

* docutils/optik.py: Added to project.  Combined from the Optik
  package, with added option groups and other modifications.  The use
  of this module is probably only temporary.

* docutils/statemachine.py:

  - Added ``runtime_init`` method to ``StateMachine`` and ``State``.
  - Added underscores to improve many awkward names.
  - In ``string2lines()``, changed whitespace normalizing translation
    table to regexp; restores Python 2.0 compatibility with Unicode.

* docutils/urischemes.py:

  - Filled in some descriptions.
  - Added "shttp" scheme.

* docutils/utils.py:

  - Added ``clean_rcs_keywords`` function (moved from
    docutils/transforms/frontmatter.py
    ``DocInfo.filter_rcs_keywords``).
  - Added underscores to improve many awkward names.
  - Changed names of Reporter's thresholds:
    warning_level -> report_level; error_level -> halt_level.
  - Moved ``utils.id()`` to ``nodes.make_id()``.
  - Added ``relative_path(source, target)``.

* docutils/languages/de.py: German mappings; added to project.  Thanks
  to Gunnar Schwant for the translations.

* docutils/languages/en.py: Added "Dedication" bibliographic field
  mappings.

* docutils/languages/sv.py: Swedish mappings; added to project by Adam
  Chodorowski.

* docutils/parsers/rst/states.py:

  - Added underscores to improve many awkward names.
  - Added RFC-2822 header support.
  - Extracted the inline parsing code from ``RSTState`` to a separate
    class, ``Inliner``, which will allow easy subclassing.
  - Made local bindings for ``memo`` container & often-used contents
    (reduces code complexity a lot).  See ``RSTState.runtime_init()``.
  - ``RSTState.parent`` replaces ``RSTState.statemachine.node``.
  - Added ``MarkupMismatch`` exception; for late corrections.
  - Added ``-/:`` characters to inline markup's start string prefix,
    ``/`` to end string suffix.
  - Fixed a footnote bug.
  - Fixed a bug with literal blocks.
  - Applied patch from Simon Budig: simplified regexps with symbolic
    names, removed ``Inliner.groups`` and ``Body.explicit.groups``.
  - Converted regexps from ``'%s' % var`` to ``'%(var)s' % locals()``.
  - Fixed a bug in ``Inliner.interpreted_or_phrase_ref()``.
  - Allowed non-ASCII in "simple names" (directive names, field names,
    references, etc.).
  - Converted ``Inliner.patterns.initial`` to be dynamically built
    from parts with ``build_regexp()`` function.
  - Changed ``Inliner.inline_target`` to ``.inline_internal_target``.
  - Updated docstrings.
  - Changed "table" to "grid_table"; added "simple_table" support.

* docutils/parsers/rst/tableparser.py:

  - Changed ``TableParser`` to ``GridTableParser``.
  - Added ``SimpleTableParser``.
  - Refactored naming.

* docutils/parsers/rst/directives/__init__.py: Added "en" (English) as
  a fallback language for directive names.

* docutils/parsers/rst/directives/html.py: Changed the ``meta``
  directive to use a ``pending`` element, used only by HTML writers.

* docutils/parsers/rst/directives/parts.py: Renamed from
  components.py.

  - Added "backlinks" attribute to "contents" directive.

* docutils/parsers/rst/languages/sv.py: Swedish mappings; added to
  project by Adam Chodorowski.

* docutils/readers/__init__.py: Gave Readers more control over
  choosing and instantiating Parsers.

* docutils/readers/pep.py: Added to project; for PEP processing.

* docutils/transforms/__init__.py: ``Transform.__init__()`` now
  requires a ``component`` parameter.

* docutils/transforms/components.py: Added to project; transforms
  related to Docutils components.

* docutils/transforms/frontmatter.py:

  - In ``DocInfo.extract_authors``, check for a single "author" in an
    "authors" group, and convert it to a single "author" element.
  - Added support for "Dedication" and generic bibliographic fields.

* docutils/transforms/peps.py: Added to project; PEP-specific.

* docutils/transforms/parts.py: Renamed from old components.py.

  - Added filter for `Contents`, to use alt-text for inline images,
    and to remove inline markup that doesn't make sense in the ToC.
  - Added "name" attribute to TOC topic depending on its title.
  - Added support for optional TOC backlinks.

* docutils/transforms/references.py: Fixed indirect target resolution
  in ``Hyperlinks`` transform.

* docutils/transforms/universal.py:

  - Changed ``Messages`` transform to properly filter out system
    messages below the warning threshold.
  - Added ``Decorations`` transform (support for ``--generator``,
    ``--date``, ``--time``, ``--source-link`` options).

* docutils/writers/__init__.py: Added "pdf" alias in anticipation of
  Engelbert Gruber's PDF writer.

* docutils/writers/html4css1.py:

  - Made XHTML-compatible (switched to lowercase element & attribute
    names; empty tag format).
  - Escape double-dashes in comment text.
  - Improved boilerplate & modularity of output.
  - Exposed modular output in Writer class.
  - Added a "generator" meta tag to <head>.
  - Added support for the ``--stylesheet`` option.
  - Added support for ``decoration``, ``header``, and ``footer``
    elements.
  - In ``HTMLTranslator.attval()``, changed whitespace normalizing
    translation table to regexp; restores Python 2.0 compatibility
    with Unicode.
  - Added the translator class as instance variable to the Writer, to
    make it easily subclassable.
  - Improved option list spacing (thanks to Richard Jones).
  - Modified field list output.
  - Added backlinks to footnotes & citations.
  - Added percentage widths to "<col>" tags (from colspec).
  - Option lists: "<code>" changed to "<kbd>", ``option_argument``
    "<span>" changed to "<var>".
  - Inline literals: "<code>" changed to "<tt>".
  - Many changes to optimize vertical space: compact simple lists etc.
  - Add a command-line options & directive attributes to control TOC
    and footnote/citation backlinks.
  - Added support for optional footnote/citation backlinks.
  - Added support for generic bibliographic fields.
  - Identify backrefs.
  - Relative URLs for stylesheet links.

* docutils/writers/pep_html.py: Added to project; HTML Writer for
  PEPs (subclass of ``html4css1.Writer``).

* docutils/writers/pseudoxml.py: Renamed from pprint.py.

* docutils/writers/docutils_xml.py: Added to project; trivial writer
  of the Docutils internal doctree in XML.

* docs/tools.txt: "Docutils Front-End Tools", added to project.

* spec/doctree.txt:

  - Changed the title to "The Docutils Document Tree".
  - Added "Hyperlink Bookkeeping" section.

* spec/docutils.dtd:

  - Added ``decoration``, ``header``, and ``footer`` elements.
  - Brought ``interpreted`` element in line with the parser: changed
    attribute "type" to "role", added "position".
  - Added support for generic bibliographic fields.

* spec/notes.txt: Continual updates.  Added "Project Policies".

* spec/pep-0256.txt:  Updated.  Added "Roadmap to the Doctring PEPs"
  section.

* spec/pep-0257.txt: Clarified prohibition of signature repetition.

* spec/pep-0258.txt: Updated.  Added text from pysource.txt and
  mailing list discussions.

* spec/pep-0287.txt:

  - Renamed to "reStructuredText Docstring Format".
  - Minor edits.
  - Reworked Q&A as an enumerated list.
  - Converted to reStructuredText format.

* spec/pysource.dtd:

  - Reworked structural elements, incorporating ideas from Tony Ibbs.

* spec/pysource.txt: Removed from project.  Moved much of its contents
  to pep-0258.txt.

* spec/rst/alternatives.txt:

  - Expanded auto-enumerated list idea; thanks to Fred Bremmer.
  - Added "Inline External Targets" section.

* spec/rst/directives.txt:

  - Added "backlinks" attribute to "contents" directive.

* spec/rst/problems.txt:

  - Updated the Enumerated List Markup discussion.
  - Added new alternative table markup syntaxes.

* spec/rst/reStructuredText.txt:

  - Clarified field list usage.
  - Updated enumerated list description.
  - Clarified purpose of directives.
  - Added ``-/:`` characters to inline markup's start string prefix,
    ``/`` to end string suffix.
  - Updated "Authors" bibliographic field behavior.
  - Changed "inline hyperlink targets" to "inline internal targets".
  - Added "simple table" syntax to supplement the existing but
    newly-renamed "grid tables".
  - Added cautions for anonymous hyperlink use.
  - Added "Dedication" and generic bibliographic fields.

* test: Made test modules standalone (subdirectories became packages).

* test/DocutilsTestSupport.py:

  - Added support for PEP extensions to reStructuredText.
  - Added support for simple tables.
  - Refactored naming.

* test/package_unittest.py: Renamed from UnitTestFolder.py.

  - Now supports true packages containing test modules
    (``__init__.py`` files required); fixes duplicate module name bug.

* test/test_pep/: Subpackage added to project; PEP testing.

* test/test_rst/test_SimpleTableParser.py: Added to project.

* tools:

  - Updated html.py and publish.py front-end tools to use the new
    command-line processing facilities of ``docutils.frontend``
    (exposed in ``docutils.core.Publisher``), reducing each to just a
    few lines of code.
  - Added ``locale.setlocale()`` calls to front-end tools.

* tools/buildhtml.py: Added to project; batch-generates .html from all
  the .txt files in directories and subdirectories.

* tools/default.css:

  - Added support for ``header`` and ``footer`` elements.
  - Added styles for "Dedication" topics (biblio fields).

* tools/docutils.conf: A configuration file; added to project.

* tools/docutils-xml.py: Added to project.

* tools/pep.py: Added to project; PEP to HTML front-end tool.

* tools/pep-html-template: Added to project.

* tools/pep2html.py: Added to project from Python (nondist/peps).
  Added support for Docutils (reStructuredText PEPs).

* tools/quicktest.py:

  - Added the ``--attributes`` option, hacked a bit.
  - Added a second command-line argument (output file); cleaned up.

* tools/stylesheets/: Subdirectory added to project.

* tools/stylesheets/pep.css: Added to project; stylesheet for PEPs.

Release 0.1 (2002-04-20)
========================

This is the first release of Docutils, merged from the now inactive
reStructuredText__ and `Docstring Processing System`__ projects.  For
the pre-Docutils history, see the `reStructuredText HISTORY`__ and the
`DPS HISTORY`__ files.

__ http://structuredtext.sourceforge.net/
__ http://docstring.sourceforge.net/
__ http://structuredtext.sourceforge.net/HISTORY.html
__ http://docstring.sourceforge.net/HISTORY.html

General changes: renamed 'dps' package to 'docutils'; renamed
'restructuredtext' subpackage to 'rst'; merged the codebases; merged
the test suites (reStructuredText's test/test_states renamed to
test/test_rst); and all modifications required to make it all work.

* docutils/parsers/rst/states.py:

  - Improved diagnostic system messages for missing blank lines.
  - Fixed substitution_reference bug.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


==================
 README: Docutils
==================

:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Date: $Date: 2004/06/10 02:16:29 $
:Web site: http://docutils.sourceforge.net/
:Copyright: This document has been placed in the public domain.

.. contents::

Thank you for downloading the Python Docutils project archive.  As
this is a work in progress, please check the project website for
updated working files (snapshots).  This project should be considered
highly experimental; APIs are subject to change at any time.

Quick-Start
===========

This is for those who want to get up & running quickly.  Read on for
complete details.

1. Get and install the latest release of Python, available from

       http://www.python.org/

   Python 2.2 or later [1]_ is required; Python 2.2.2 or later is
   recommended.

2. Use the latest Docutils code.  Get the code from CVS or from the
   snapshot:

       http://docutils.sf.net/docutils-snapshot.tgz

   See `Releases & Snapshots`_ below for details.

3. Unpack the tarball in a temporary directory (**not** directly in
   Python's ``site-packages``) and install with the standard ::

       python setup.py install

   See Installation_ below for details.

4. Use a front-end tool from the "tools" subdirectory of the same
   directory as in step 3.  For example::

       cd tools
       ./rst2html.py test.txt test.html

   See Usage_ below for details.

Purpose
=======

The purpose of the Docutils project is to create a set of tools for
processing plaintext documentation into useful formats, such as HTML,
XML, and TeX.  Support for the following sources has been implemented:

* Standalone files.

* `PEPs (Python Enhancement Proposals)`_.

Support for the following sources is planned:

* Inline documentation from Python modules and packages, extracted
  with namespace context.  **This is the focus of the current
  development effort.**

* Email (RFC-822 headers, quoted excerpts, signatures, MIME parts).

* Wikis, with global reference lookups of "wiki links".

* Compound documents, such as multiple chapter files merged into a
  book.

* And others as discovered.

.. _PEPs (Python Enhancement Proposals):
   http://www.python.org/peps/pep-0012.html

Releases & Snapshots
====================

Putting together an official "Release" of Docutils is a significant
effort, so it isn't done that often.  In the meantime, the CVS
snapshots always contain the latest code and documentation, usually
updated within an hour of changes being committed to the repository,
and usually bug-free:

* Snapshot of Docutils code, documentation, front-end tools, and
  tests: http://docutils.sf.net/docutils-snapshot.tgz

* Snapshot of the Sandbox (experimental, contributed code):
  http://docutils.sf.net/docutils-sandbox-snapshot.tgz

* Snapshot of web files (the files that generate the web site):
  http://docutils.sf.net/docutils-web-snapshot.tgz

To keep up to date on the latest developments, download fresh copies
of the snapshots regularly.  New functionality is being added weekly,
sometimes daily.  (There's also the CVS repository, and a mailing list
for CVS messages.  See the web site [address above] or
docs/dev/policies.txt for details.)

Requirements
============

To run the code, Python 2.2 or later [1]_ must already be installed.
The latest release is recommended.  Python is available from
http://www.python.org/.

The `Python Imaging Library`, or PIL, is used for some image
manipulation operations if it is installed.

Docutils uses Greg Ward's Optik_/optparse option processing package.
It is included in the Docutils distribution.  Python 2.3 and later
come with optparse in the standard library; in this case, the Docutils
copy is not installed.

.. [1] Python 2.1 may be used providing the compiler package is
   installed.  The compiler package can be found in the Tools/
   directory of Python 2.1's source distribution.

.. _Python Imaging Library: http://www.pythonware.com/products/pil/
.. _Optik: http://optik.sourceforge.net/

Project Files & Directories
===========================

* README.txt: You're reading it.

* COPYING.txt: Public Domain Dedication and copyright details for
  non-public-domain files (most are PD).

* FAQ.txt: Docutils Frequently Asked Questions.

* HISTORY.txt: Release notes for the current and previous project
  releases.

* setup.py: Installation script.  See "Installation" below.

* install.py: Quick & dirty installation script.  Just run it.

* docutils: The project source directory, installed as a Python
  package.

* extras: Directory for third-party modules that Docutils depends on.
  These are only installed if they're not already present.

* docs: The project documentation directory.  Read ``docs/index.txt``
  for an overview, which is especially interesting for developers.

* docs/user: The project user documentation directory.  Contains the
  following documents, among others:

  - docs/user/tools.txt: Docutils Front-End Tools
  - docs/user/latex.txt: Docutils LaTeX Writer
  - docs/user/rst/quickstart.txt: A ReStructuredText Primer
  - docs/user/rst/quickref.html: Quick reStructuredText (HTML only)

* docs/ref: The project reference directory.
  ``docs/ref/rst/restructuredtext.txt`` is the reStructuredText
  reference.

* licenses: Directory containing copies of license files for
  non-public-domain files.

* tools: Directory for Docutils front-end tools.  See
  ``docs/user/tools.txt`` for documentation.

* test: Unit tests.  Not required to use the software, but very useful
  if you're planning to modify it.  See `Running the Test Suite`_
  below.

Installation
============

The first step is to expand the ``.tar.gz`` or ``.tgz`` archive in a
temporary directory (**not** directly in Python's ``site-packages``).
It contains a distutils setup file "setup.py".  OS-specific
installation instructions follow.

GNU/Linux, BSDs, Unix, Mac OS X, etc.
-------------------------------------

1. Open a shell.

2. Go to the directory created by expanding the archive::

       cd <archive_directory_path>

3. Install the package::

       python setup.py install

   If the python executable isn't on your path, you'll have to specify
   the complete path, such as /usr/local/bin/python.  You may need
   root permissions to complete this step.

You can also just run install.py; it does the same thing.

Windows
-------

1. Open a DOS box (Command Shell, MSDOS Prompt, or whatever they're
   calling it these days).

2. Go to the directory created by expanding the archive::

       cd <archive_directory_path>

3. Install the package::

       <path_to_python.exe>\python setup.py install

If your system is set up to run Python when you double-click on .py
files, you can run install.py to do the same as the above.

Mac OS 8/9
----------

1. Open the folder containing the expanded archive.

2. Double-click on the file "setup.py", which should be a "Python
   module" file.

   If the file isn't a "Python module", the line endings are probably
   also wrong, and you will need to set up your system to recognize
   ".py" file extensions as Python files.  See
   http://gotools.sourceforge.net/mac/python.html for detailed
   instructions.  Once set up, it's easiest to start over by expanding
   the archive again.

3. The distutils options window will appear.  From the "Command" popup
   list choose "install", click "Add", then click "OK".

If install.py is a "Python module" (see step 2 above if it isn't), you
can run it (double-click) instead of the above.  The distutils options
window will not appear.

Usage
=====

After unpacking and installing the Docutils package, the following
shell commands will generate HTML for all included documentation::

    cd <archive_directory_path>/tools
    ./buildhtml.py ../

The final directory name of the ``<archive_directory_path>`` is
"docutils" for snapshots.  For official releases, the directory may be
called "docutils-X.Y", where "X.Y" is the release version.
Alternatively::

    cd <archive_directory_path>
    tools/buildhtml.py --config=tools/docutils.conf

Some files may generate system messages (warnings and errors).  The
``tools/test.txt`` file (under the archive directory) contains 5
intentional errors.  (They test the error reporting mechanism!)

There are many front-end tools in the unpacked "tools" subdirectory.
You may want to begin with the "rst2html.py" front-end tool.  Most
tools take up to two arguments, the source path and destination path,
with STDIN and STDOUT being the defaults.  Use the "--help" option to
the front-end tools for details on options and arguments.  See
Docutils Front-End Tools (``docs/user/tools.txt``) for full documentation.

The package modules are continually growing and evolving.  The
``docutils.statemachine`` module is usable independently.  It contains
extensive inline documentation (in reStructuredText format of course).

Contributions are welcome!

Running the Test Suite
======================

To run the entire test suite, after installation_ open a shell and use
the following commands::

    cd <archive_directory_path>/test
    ./alltests.py

You should see a long line of periods, one for each test, and then a
summary like this::

    Ran 518 tests in 24.653s

    OK
    Elapsed time: 26.189 seconds

The number of tests will grow over time, and the times reported will
depend on the computer running the tests.  The difference between the
two times represents the time required to set up the tests (import
modules, create data structures, etc.).

If any of the tests fail, please `open a bug report`_ or `send email`_
[2]_.  Please include all relevant output, information about your
operating system, Python version, and Docutils version.  To see the
Docutils version, use these commands::

    cd ../tools
    ./quicktest.py --version

.. _open a bug report:
   http://sourceforge.net/tracker/?group_id=38414&atid=422030
.. _send email: mailto:docutils-users@lists.sourceforge.net
   ?subject=Docutils%20test%20suite%20failure

Getting Help
============

If you have questions or need assistance with Docutils or
reStructuredText, please `post a message`_ to the `Docutils-Users
mailing list`_ [2]_.

.. [2] Due to overwhelming amounts of spam, the
   docutils-users@lists.sourceforge.net mailing list has been set up
   for subscriber posting only.  Non-subscribers who post to
   docutils-users will receive a message with "Subject: Your message
   to Docutils-users awaits moderator approval".  Legitimate messages
   are accepted and posted as soon as possible (a list administrator
   must verify the message manually).  If you'd like to subscribe to
   docutils-users, please visit
   <http://lists.sourceforge.net/lists/listinfo/docutils-users>.

.. _post a message: mailto:docutils-users@lists.sourceforge.net
.. _Docutils-Users mailing list:
   http://lists.sourceforge.net/lists/listinfo/docutils-users

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


Acknowledgements
================

:Author: David Goodger
:Contact: goodger@python.org
:Date: $Date: 2004/06/10 02:17:20 $
:Revision: $Revision: 1.2 $
:Copyright: This document has been placed in the public domain.

I would like to acknowledge the people who have made a direct impact
on the Docutils project, knowingly or not, in terms of encouragement,
suggestions, criticism, bug reports, code contributions, cash
donations, tasty treats, and related projects:

* Aahz
* David Abrahams
* David Ascher
* Heiko Baumann
* Eric Bellot
* Ian Bicking
* Marek Blaha
* Martin Blais
* Stephen Boulet
* Fred Bremmer
* Simon Budig
* Bill Bumgarner
* Brett Cannon
* Greg Chapman
* Nicolas Chauveau
* Beni Cherniavsky
* Adam Chodorowski
* Brent Cook
* Laura Creighton
* Artur de Sousa Rocha
* Stephan Deibel & `Wing IDE <http://wingide.com/>`__
* Jason Diamond
* William Dode
* Fred Drake
* Reggie Dugard
* Dethe Elza
* Marcus Ertl
* Benja Fallenstein
* fantasai
* Stefane Fermigier
* Jim Fulton
* Peter Funk
* Lele Gaifax
* Dinu C. Gherman
* Matt Gilbert
* Jorge Gonzalez
* Engelbert Gruber
* Jacob Hallen
* Simon Hefti
* Doug Hellmann
* Marc Herbert
* Juergen Hermann
* Jannie Hofmeyr
* Steve Holden
* Michael Hudson
* Marcelo Huerta San Martin
* Ludger Humbert
* Jeremy Hylton
* Tony Ibbs
* Alan Jaffray
* Dmitry Jemerov
* Richard Jones
* Andreas Jung
* Garth Kidd
* Axel Kollmorgen
* Jeff Kowalczyk
* Dave Kuhlman
* Lloyd Kvam
* Kirill Lapshin
* Nicola Larosa
* Daniel Larsson
* Marc-Andre Lemburg
* Julien Letessier
* Wolfgang Lipp
* Edward Loper
* Dallas Mahrt
* Ken Manheimer
* Bob Marshall
* Mark McEahern
* Vincent McIntyre
* John F Meinel Jr
* Vasko Miroslav
* Skip Montanaro
* Paul Moore
* Nigel W. Moriarty
* Mark Nodine
* Patrick K. O'Brien
* Michel Pelletier
* Sam Penrose
* Tim Peters
* Pearu Peterson
* Mark Pilgrim
* Brett g Porter
* David Priest
* Jens Quade
* Andy Robinson
* Tavis Rudd
* Tracy Ruggles
* Oliver Rutherfurd
* Kenichi Sato
* Ueli Schlaepfer
* Gunnar Schwant
* Bill Sconce
* Frank Siebenlist
* Bruce Smith
* Darek Suchojad
* Roman Suzi
* tav
* Kent Tenney
* Bob Tolbert
* Paul Tremblay
* Laurence Tratt
* Adrian van den Dries
* Guido van Rossum
* Paul Viren
* Martin von Loewis
* Greg Ward
* Barry Warsaw
* Edward Welbourne
* Felix Wiemann
* Ka-Ping Yee
* Moshe Zadka

Thank you!

Special thanks to `SourceForge <http://sourceforge.net>`__ and the
`Python Software Foundation <http://www.python.org/psf/>`__.

Hopefully I haven't forgotten anyone or misspelled any names;
apologies (and please let me know!) if I have.


==========================================
 Docutils_ Project Documentation Overview
==========================================

:Author: David Goodger
:Contact: goodger@python.org
:Date: $Date: 2004/06/28 01:07:50 $
:Revision: $Revision: 1.15 $
:Copyright: This document has been placed in the public domain.

The latest working documents may be accessed individually below, or
from the ``docs`` directory of the `development snapshots`__.  They
are also distributed with the `latest project release package`__, but
those may not be completely up to date.

__ http://docutils.sourceforge.net/#development-snapshots
__ http://docutils.sourceforge.net/#project-releases
.. _Docutils: http://docutils.sourceforge.net/

.. contents::

Docutils Stakeholders
=====================

Docutils stakeholders can be categorized in several groups:

1. End-users: users of reStructuredText and the Docutils tools.
   Although some are developers (e.g. Python developers utilizing
   reStructuredText for docstrings in their source), many are not.

2. Client-developers: developers using Docutils as a library,
   programmers developing *with* Docutils.

3. Component-developers: those who implement application-specific
   components, directives, and/or roles, separately from Docutils.

4. Core-developers: developers of the Docutils codebase and
   participants in the Docutils project community.

5. Re-implementers: developers of alternate implementations of
   Docutils.

There's a lot of overlap between these groups.  Most (perhaps all)
core-developers, component-developers, client-developers, and
re-implementers are also end-users.  Core-developers are also
client-developers, and may also be component-developers in other
projects.  Component-developers are also client-developers.

Project Fundamentals
====================

These files are for all Docutils stakeholders.  They are kept at the
top level of the Docutils project directory.

:README.txt_: Project overview: quick-start, requirements,
              installation, and usage.
:COPYING.txt_: Conditions for Docutils redistribution, with links to
               licenses.
:FAQ.txt_: Docutils Frequently Asked Questions.  If you have a
           question or issue, there's a good chance it's already
           answered here.
:BUGS.txt_: A list of known bugs, and how to report a bug.
:HISTORY.txt_: Change history log.
:THANKS.txt_: Acknowledgements.

.. _README.txt: ../README.html
.. _BUGS.txt: ../BUGS.html
.. _COPYING.txt: ../COPYING.html
.. _Docutils FAQ:
.. _FAQ.txt: ../FAQ.html
.. _HISTORY.txt: ../HISTORY.html
.. _THANKS.txt: ../THANKS.html

``user/``: Introductory & Tutorial Material for End-Users
=========================================================

Docutils-general:

* `Docutils Front-End Tools <user/tools.html>`__
* `Docutils Configuration Files <user/config.html>`__
* `Docutils LaTeX Writer <user/latex.html>`__

`reStructuredText <http://docutils.sourceforge.net/rst.html>`_:

* `A ReStructuredText Primer (HTML) <user/rst/quickstart.html>`__ (or
  `text source <user/rst/quickstart.txt>`__)
* `Quick reStructuredText <user/rst/quickref.html>`__ (user reference)
* `reStructuredText Cheat Sheet <user/rst/cheatsheet.txt>`__ (text
  only; 1 page for syntax, 1 page directive & role reference)

``ref/``: Reference Material for All Groups
===========================================

Many of these files began as developer specifications, but now that
they're mature and used by end-users and client-developers, they have
become reference material.  Successful specs evolve into refs.

Docutils-general:

* `The Docutils Document Tree <ref/doctree.html>`__ (incomplete)
* `Docutils Transforms <ref/transforms.html>`__
* `Docutils Generic DTD <ref/docutils.dtd>`__
* `OASIS XML Exchange Table Model Declaration Module
  <ref/soextblx.dtd>`__ (CALS tables DTD module)

Although not in the "ref" directory, `PEP 258`_ is a must-read
reference for any Docutils developer.

reStructuredText_:

* `An Introduction to reStructuredText <ref/rst/introduction.html>`__
  (includes the `Goals <ref/rst/introduction.html#goals>`__ and
  `History <ref/rst/introduction.html#history>`__ of reStructuredText)
* `reStructuredText Markup Specification <ref/rst/restructuredtext.html>`__
* `reStructuredText Directives <ref/rst/directives.html>`__
* `reStructuredText Interpreted Text Roles <ref/rst/roles.html>`__

Prehistoric:

* `Setext Documents Mirror
  <http://docutils.sourceforge.net/mirror/setext.html>`__

``peps/``: Python Enhancement Proposals
=======================================

* `PEP 256: Docstring Processing System Framework`__ is a high-level
  generic proposal.  [`PEP 256`__ in the `master repository`_]
* `PEP 257: Docstring Conventions`__ addresses docstring style and
  touches on content.  [`PEP 257`__ in the `master repository`_]
* `PEP 258: Docutils Design Specification`__ is an overview of the
  architecture of Docutils.  It documents design issues and
  implementation details.  [`PEP 258`__ in the `master repository`_]
* `PEP 287: reStructuredText Docstring Format`__ proposes a standard
  markup syntax.  [`PEP 287`__ in the `master repository`_]

Please note that PEPs in the `master repository`_ may not be current,
whereas the local versions are.

__ peps/pep-0287.html
__ http://www.python.org/peps/pep-0287.html
__ peps/pep-0257.html
__ http://www.python.org/peps/pep-0257.html
__ peps/pep-0256.html
__ http://www.python.org/peps/pep-0256.html
.. _PEP 258:
__ peps/pep-0258.html
__ http://www.python.org/peps/pep-0258.html
.. _master repository: http://www.python.org/peps/

``api/``: API Reference Material for Client-Developers
======================================================

* `The Docutils Publisher <api/publisher.html>`__
* `Inside A Docutils Command-Line Front-End Tool <api/cmdline-tool.html>`__
* `Docutils Runtime Settings <api/runtime-settings.html>`__
* (`Docutils Transforms <ref/transforms.html>`__ should be moved here)

`PEP 258`_ is an overview of the architecture of Docutils.

``howto/``: Instructions for Developers
=======================================

* `Docutils Internationalization <howto/i18n.html>`__
* `Creating reStructuredText Directives <howto/rst-directives.html>`__
* `Creating reStructuredText Interpreted Text Roles
  <howto/rst-roles.html>`__

``dev/``: Development Notes and Plans for Core-Developers
=========================================================

Docutils-general:

* `Docutils To Do List <dev/todo.html>`__
* `Docutils Project Policies <dev/policies.html>`__
* `Docutils Web Site <dev/website.html>`__
* `Docutils Release Procedure <dev/release.html>`__
* `Docutils Testing <dev/testing.html>`__
* `Docstring Semantics <dev/semantics.html>`__ (incomplete)
* `Python Source Reader <dev/pysource.html>`_ (incomplete)
* `Docutils Python DTD <dev/pysource.dtd>`_ (experimental)
* `Plan for Enthought API Documentation Tool <dev/enthought-plan.html>`_
* `Enthought API Documentation Tool RFP <dev/enthought-rfp.html>`_

reStructuredText_:

* `A Record of reStructuredText Syntax Alternatives
  <dev/rst/alternatives.html>`__
* `Problems With StructuredText <dev/rst/problems.html>`__

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


===============================================
 Inside A Docutils Command-Line Front-End Tool
===============================================

:Author: David Goodger
:Contact: goodger@python.org
:Date: $Date: 2004/06/27 03:47:39 $
:Revision: $Revision: 1.1 $
:Copyright: This document has been placed in the public domain.

`The Docutils Publisher`_ class was set up to make building
command-line tools easy.  All that's required is to choose components
and supply settings for variations.  Let's take a look at a typical
command-line front-end tool, ``tools/rst2html.py``, from top to
bottom.

On Unixish systems, it's best to make the file executable (``chmod +x
file``), and supply an interpreter on the first line, the "shebang" or
"hash-bang" line::

    #!/usr/bin/env python

Windows systems can be set up to associate the Python interpreter with
the ``.py`` extension.

Next are some comments providing metadata::

    # Author: David Goodger
    # Contact: goodger@python.org
    # Revision: $Revision: ...
    # Date: $Date: ...
    # Copyright: This module has been placed in the public domain.

The module docstring describes the purpose of the tool::

    """
    A minimal front end to the Docutils Publisher, producing HTML.
    """

This next block attempts to invoke locale support for
internationalization services, specifically text encoding.  It's not
supported on all platforms though, so it's forgiving::

    try:
        import locale
        locale.setlocale(locale.LC_ALL, '')
    except:
        pass

The real work will be done by the code that's imported here::

    from docutils.core import publish_cmdline, default_description

We construct a description of the tool, for command-line help::

    description = ('Generates (X)HTML documents from standalone '
                   'reStructuredText sources.  ' + default_description)

Now we call the Publisher convenience function, which takes over.
Most of it's defaults are used ("standalone" Reader,
"reStructuredText" Parser, etc.).  The HTML Writer is chosen by name,
and a description for command-line help is passed in::

    publish_cmdline(writer_name='html', description=description)

That's it!  `The Docutils Publisher`_ takes care of the rest.

.. _The Docutils Publisher: ./publisher.html


========================
 The Docutils Publisher
========================

:Author: David Goodger
:Contact: goodger@python.org
:Date: $Date: 2004/06/27 17:36:49 $
:Revision: $Revision: 1.2 $
:Copyright: This document has been placed in the public domain.

.. contents::

Publisher Convenience Functions
===============================

Each of these functions set up a ``docutils.core.Publisher`` object,
then call its ``publish`` method.  ``docutils.core.Publisher.publish``
handles everything else.  There are five convenience functions in the
``docutils.core`` module:

* ``publish_cmdline``: for command-line front-end tools, like
  ``rst2html.py``.  There are several examples in the ``tools/``
  directory.  A detailed analysis of one such tool is in `Inside A
  Docutils Command-Line Front-End Tool`_

* ``publish_file``: for programmatic use with file-like I/O.  In
  addition to writing the encoded output to a file, also returns the
  encoded output as a string.

* ``publish_string``: for programmatic use with string I/O.  Returns
  the encoded output as a string.

* ``publish_parts``: for programmatic use with string input; returns a
  dictionary of document parts.  Dictionary keys are the names of
  parts, and values are Unicode strings; encoding is up to the client.
  Useful when only portions of the processed document are desired.
  Currently only implemented for the HTML Writer.

  There are examples in the ``docutils/examples.py`` module.

* ``publish_programmatically``: for custom programmatic use.  This
  function implements common code and is used by ``publish_file``,
  ``publish_string``, and ``publish_parts``.  It returns a 2-tuple:
  the encoded string output and the Publisher object.

.. _Inside A Docutils Command-Line Front-End Tool: ./cmdline-tool.html

Configuration
-------------

To pass application-specific setting defaults to the Publisher
convenience functions, use the ``settings_overrides`` parameter.  Pass
a dictionary of setting names & values, like this::

    overrides = {'input_encoding': 'ascii',
                 'output_encoding': 'latin-1'}
    output = publish_string(..., settings_overrides=overrides)

Settings from command-line options override configuration file
settings, and they override application defaults.  For details, see
`Docutils Runtime Settings`_.  See `Docutils Configuration Files`_ for
details about individual settings.

.. _Docutils Runtime Settings: ./runtime-settings.html
.. _Docutils Configuration Files: ../user/tools.html

Encodings
---------

The default output encoding of Docutils is UTF-8.  If you have any
non-ASCII in your text, you may have to do a bit more setup.  Docutils
may introduce some non-ASCII text if you use symbol-footnotes or
section numbering.


===========================
 Docutils Runtime Settings
===========================

:Author: David Goodger
:Contact: goodger@python.org
:Date: $Date: 2004/06/27 21:18:22 $
:Revision: $Revision: 1.6 $
:Copyright: This document has been placed in the public domain.

.. contents::

Introduction
============

Docutils runtime settings are assembled from several sources:
component settings specifications, application settings
specifications, configuration files, and command-line options.
Docutils overlays default and explicitly specified values from these
sources such that settings behave the way we want and expect them to
behave.

To understand how Docutils deals with runtime settings, the attributes
and parameters involved must first be understood.  Begin with the the
docstrings of the attributes of the ``docutils.SettingsSpec`` base
class (in the ``docutils/__init__.py`` module):

* ``settings_spec``
* ``settings_defaults``
* ``settings_default_overrides``
* ``relative_path_settings``
* ``config_section``
* ``config_section_dependencies``

Next, several _`convenience function parameters` are also significant
(described in the ``docutils.core.publish_programmatically`` function
docstring):

* ``settings``, if present, is assumed to be complete and no further
  runtime settings processing is done.

* ``settings_spec`` is treated as a fourth component (after the
  Parser, Reader, and Writer).

* ``settings_overrides`` is a dictionary which will override the
  defaults of the components.

* ``config_section`` sets or overrides an application-specific
  configuration file section.

.. _command-line tools:

Runtime Settings Processing for Command-Line Tools
==================================================

Following along with the actual code is recommended.  The
``docutils/__init__.py``, ``docutils/core.py``, and
``docutils.frontend`` modules are described.

1. A command-line front-end tool imports and calls
   ``docutils.core.publish_cmdline``.  The relevant `convenience
   function parameters`_ are described above.

2. ``docutils.core.publish_cmdline`` initializes a
   ``docutils.core.Publisher`` object, then calls its ``publish``
   method.

3. The ``docutils.core.Publisher`` object's ``publish`` method checks
   its ``settings`` attribute to see if it's defined.  If it is, no
   further runtime settings processing is done.

   If ``settings`` is not defined, ``self.process_command_line`` is
   called with the following relevant arguments:

   * ``settings_spec``
   * ``config_section``
   * ``settings_overrides`` (in the form of excess keyword
     arguments, collected in the ``defaults`` parameter)

4. ``self.process_command_line`` calls ``self.setup_option_parser``,
   passing ``settings_spec``, ``config_section``, and ``defaults``.

5. ``self.setup_option_parser`` checks its ``config_section``
   parameter; if defined, it adds that config file section to
   ``settings_spec`` (or to a new, empty ``docutils.SettingsSpec``
   object), replacing anything defined earlier.  (See `Docutils
   Configuration Files`_ for details.)  Then it instantiates a new
   ``docutils.frontend.OptionParser`` object, passing the following
   relevant arguments:

   * ``components``: A tuple of ``docutils.SettingsSpec`` objects,
     ``(self.parser, self.reader, self.writer, settings_spec)``
   * ``defaults`` (originally from ``settings_overrides``)

6. The ``docutils.frontend.OptionParser`` object's ``__init__`` method
   calls ``self.populate_from_components`` with ``self.components``,
   which consists of ``self`` prepended to the ``components`` tuple it
   received.  ``self`` (``docutils.frontend.OptionParser``) defines
   general Docutils settings.

7. In ``self.populate_from_components``, for each component passed,
   ``component.settings_spec`` is processed and
   ``component.settings_defaults`` is applied.  Then, for each
   component, ``component.settings_default_overrides`` is applied.
   This two-loop process ensures that
   ``component.settings_default_overrides`` can override the default
   settings of any other component.

8. Back in ``docutils.frontend.OptionParser.__init__``, the
   ``defaults`` parameter (derived from the ``settings_overrides``
   parameter of ``docutils.core.Publisher.publish``) is overlaid over
   ``self.defaults``.  So ``settings_overrides`` has priority over all
   ``SettingsSpec`` data.

9. Next, ``docutils.frontend.OptionParser.__init__`` checks if
   configuration files are enabled (its ``read_config_files``
   parameter is true, and ``self.defaults['_disable_config']`` is
   false).  If they are enabled (and normally, they are),
   ``self.get_standard_config_settings`` is called.  This reads the
   `docutils configuration files`_, and returns a dictionary of
   settings.  This is then overlaid on ``self.defaults``.  So
   configuration file settings have priority over all software-defined
   defaults.

10. Back in the ``docutils.core.Publisher`` object,
    ``self.setup_option_parser`` returns the ``option_parser`` object
    to its caller, ``self.process_command_line``.

11. ``self.process_command_line`` calls ``option_parser.parse_args``,
    which parses all command line options and returns a
    ``docutils.frontend.Values`` object.  This is assigned to the
    ``docutils.core.Publisher`` object's ``self.settings``.  So
    command-line options have priority over configuration file
    settings.

    When ``option_parser.parse_args`` is called, the source and
    destination command-line arguments are also parsed, and assigned
    to the ``_source`` and ``_destination`` attributes of what becomes
    the ``docutils.core.Publisher`` object's ``self.settings``.

12. From ``docutils.core.Publisher.publish``, ``self.set_io`` is
    called with no arguments.  If either ``self.source`` or
    ``self.destination`` are not set, the corresponding
    ``self.set_source`` and ``self.set_destination`` are called,
    effectively with no arguments.

13. ``self.set_source`` checks for a ``source_path`` parameter, and if
    there is none (which is the case for command-line use), it is
    taken from ``self.settings._source``.  ``self.source`` is set by
    instantiating a ``self.source_class`` object.  For command-line
    front-end tools, the default ``self.source_class`` is used,
    ``docutils.io.FileInput``.

14. ``self.set_destination`` does the same job for the destination
    that ``self.set_source`` does for the source (the default
    ``self.destination_class`` is ``docutils.io.FileOutput``).

.. _Docutils Configuration Files: ../user/tools.html

Runtime Settings Processing From Applications
=============================================

Applications process runtime settings in a significantly different way
than `command-line tools`_ do.  Instead of calling
``publish_cmdline``, the application calls one of ``publish_file``,
``publish_string``, or ``publish_parts``.  These in turn call
``publish_programmatically``, which implements a generic programmatic
interface.  Although an application may also call
``publish_programmatically`` directly, it is not recommended (if it
does seem to be necessary, please write to the `Docutils-Develop
mailing list`_).

``publish_programmatically`` accepts the same `convenience function
parameters`_ as ``publish_cmdline``.  Where things differ is that
programmatic use does not

TO BE COMPLETED.

.. copy & modify the list from command-line tools?

.. _Docutils-Develop mailing list: docutils-develop@lists.sf.net


===========================================
 Plan for Enthought API Documentation Tool
===========================================

:Author: David Goodger
:Contact: goodger@python.org
:Date: $Date: 2004/06/10 16:25:03 $
:Revision: $Revision: 1.4 $
:Copyright: 2004 by `Enthought, Inc. <http://www.enthought.com>`_
:License: `Enthought License`_ (BSD-style)

.. _Enthought License: http://docutils.sf.net/licenses/enthought.txt

This document should be read in conjunction with the `Enthought API
Documentation Tool RFP`__ prepared by Janet Swisher.

__ enthought-rfp.html

.. contents::
.. sectnum::

Introduction
============

In March 2004 at I met Eric Jones, president and CTO of `Enthought,
Inc.`_, at `PyCon 2004`_ in Washington DC.  He told me that Enthought
was using reStructuredText_ for source code documentation, but they
had some issues.  He asked if I'd be interested in doing some work on
a customized API documentation tool.  Shortly after PyCon, Janet
Swisher, Enthought's senior technical writer, contacted me to work out
details.  Some email, a trip to Austin in May, and plenty of Texas
hospitality later, we had a project.  This document will record the
details, milestones, and evolution of the project.

In a nutshell, Enthought is sponsoring the implementation of an open
source API documentation tool that meets their needs.  Fortuitously,
their needs coincide well with the "Python Source Reader" description
in `PEP 258`_.  In other words, Enthought is funding some significant
improvements to Docutils, improvements that were planned but never
implemented due to time and other constraints.  The implementation
will take place gradually over several months, on a part-time basis.

This is an ideal example of cooperation between a corporation and an
open-source project.  The corporation, the project, I personally, and
the community all benefit.  Enthought, whose commitment to open source
is also evidenced by their sponsorship of SciPy_, benefits by
obtaining a useful piece of software, much more quickly than would
have been possible without their support.  Docutils benefits directly
from the implementation of one of its core subsystems.  I benefit from
the funding, which allows me to justify the long hours to my wife and
family.  All the corporations, projects, and individuals that make up
the community will benefit from the end result, which will be great.

All that's left now is to actually do the work!

.. _PyCon 2004: http://pycon.org/dc2004/
.. _reStructuredText: http://docutils.sf.net/rst.html
.. _SciPy: http://www.scipy.org/

Development Plan
================

1. Analyze prior art, most notably Epydoc_ and HappyDoc_, to see how
   they do what they do.  I have no desire to reinvent wheels
   unnecessarily.  I want to take the best ideas from each tool,
   combined with the outline in `PEP 258`_ (which will evolve), and
   build at least the foundation of the definitive Python
   auto-documentation tool.

   .. _Epydoc: http://epydoc.sourceforge.net/
   .. _HappyDoc: http://happydoc.sourceforge.net/
   .. _PEP 258:
      http://docutils.sf.net/spec/pep-0258.html#python-source-reader

2. Decide on a base platform.  The best way to achieve Enthought's
   goals in a reasonable time frame may be to extend Epydoc or
   HappyDoc.  Or it may be necessary to start fresh.

3. Extend the reStructuredText parser.  See `Proposed Changes to
   reStructuredText`_ below.

4. Depending on the base platform chosen, build or extend the
   docstring & doc comment extraction tool.  This may be the biggest
   part of the project, but I won't be able to break it down into
   details until more is known.

Repository
==========

If possible, all software and documentation files will be stored in
the CVS repository of Docutils and/or the base project, which are all
publicly-available via anonymous pserver access.

The Docutils project is very open about granting CVS write access; so
far, everyone who asked has been given access.  Any Enthought staff
member who would like CVS write access will get it.

If either Epydoc or HappyDoc is chosen as the base platform, I will
ask the project's administrator for CVS access for myself and any
Enthought staff member who wants it.  If sufficient access is not
granted -- although I doubt that there would be any problem -- we may
have to begin a fork, which could be hosted on SourceForge, on
Enthought's Subversion server, or anywhere else deemed appropriate.

Copyright & License
===================

Most existing Docutils files have been placed in the public domain, as
follows::

    :Copyright: This document has been placed in the public domain.

This is in conjunction with the "Public Domain Dedication" section of
COPYING.txt__.

__ http://docutils.sourceforge.net/COPYING.html

The code and documentation originating from Enthought funding will
have Enthought's copyright and license declaration.  While I will try
to keep Enthought-specific code and documentation separate from the
existing files, there will inevitably be cases where it makes the most
sense to extend existing files.

I propose the following:

1. New files related to this Enthought-funded work will be identified
   with the following field-list headers::

       :Copyright: 2004 by Enthought, Inc.
       :License: Enthought License (BSD Style)

   The license field text will be linked to the license file itself.

2. For significant or major changes to an existing file (more than 10%
   change), the headers shall change as follows (for example)::

       :Copyright: 2001-2004 by David Goodger
       :Copyright: 2004 by Enthought, Inc.
       :License: BSD-style

   If the Enthought-funded portion becomes greater than the previously
   existing portion, Enthought's copyright line will be shown first.

3. In cases of insignificant or minor changes to an existing file
   (less than 10% change), the public domain status shall remain
   unchanged.

A section describing all of this will be added to the Docutils
`COPYING`__ instructions file.

If another project is chosen as the base project, similar changes
would be made to their files, subject to negotiation.

__ http://docutils.sf.net/COPYING.html

Proposed Changes to reStructuredText
====================================

Doc Comment Syntax
------------------

The "traits" construct is implemented as dictionaries, where
standalone strings would be Python syntax errors.  Therefore traits
require documentation in comments.  We also need a way to
differentiate between ordinary "internal" comments and documentation
comments (doc comments).

Javadoc uses the following syntax for doc comments::

    /**
     * The first line of a multi-line doc comment begins with a slash
     * and *two* asterisks.  The doc comment ends normally.
     */

Python doesn't have multi-line comments; only single-line.  A similar
convention in Python might look like this::

    ##
    # The first line of a doc comment begins with *two* hash marks.
    # The doc comment ends with the first non-comment line.
    'data' : AnyValue,

    ## The double-hash-marks could occur on the first line of text,
    #  saving a line in the source.
    'data' : AnyValue,

How to indicate the end of the doc comment? ::

    ##
    # The first line of a doc comment begins with *two* hash marks.
    # The doc comment ends with the first non-comment line, or another
    # double-hash-mark.
    ##
    # This is an ordinary, internal, non-doc comment.
    'data' : AnyValue,

    ## First line of a doc comment, terse syntax.
    #  Second (and last) line.  Ends here: ##
    # This is an ordinary, internal, non-doc comment.
    'data' : AnyValue,

Or do we even need to worry about this case?  A simple blank line
could be used::

    ## First line of a doc comment, terse syntax.
    #  Second (and last) line.  Ends with a blank line.

    # This is an ordinary, internal, non-doc comment.
    'data' : AnyValue,

Other possibilities::

    #" Instead of double-hash-marks, we could use a hash mark and a
    #  quotation mark to begin the doc comment.
    'data' : AnyValue,

    ## We could require double-hash-marks on every line.  This has the
    ## added benefit of delimiting the *end* of the doc comment, as
    ## well as working well with line wrapping in Emacs
    ## ("fill-paragraph" command).
    # Ordinary non-doc comment.
    'data' : AnyValue,

    #" A hash mark and a quotation mark on each line looks funny, and
    #" it doesn't work well with line wrapping in Emacs.
    'data' : AnyValue,

These styles (repeated on each line) work well with line wrapping in
Emacs::

    ##  #>  #|  #-  #%  #!  #*

These styles do *not* work well with line wrapping in Emacs::

    #"  #'  #:  #)  #.  #/  #@  #$  #^  #=  #+  #_  #~

The style of doc comment indicator used could be a runtime, global
and/or per-module setting.  That may add more complexity than it's
worth though.

Recommendation
``````````````

I recommend adopting "#*" on every line::

    # This is an ordinary non-doc comment.

    #* This is a documentation comment, with an asterisk after the
    #* hash marks on every line.
    'data' : AnyValue,

I initially recommended adopting double-hash-marks::

    # This is an ordinary non-doc comment.

    ## This is a documentation comment, with double-hash-marks on
    ## every line.
    'data' : AnyValue,

But Janet Swisher rightly pointed out that this could collide with
ordinary comments that are then block-commented.  This applies to
double-hash-marks on the first line only as well.  So they're out.

Docstring Density & Whitespace Minimization
-------------------------------------------

One problem with extensively documented classes & functions, is that
there is a lot of screen space wasted on whitespace.  Here's some
current Enthought code (from lib/cp/fluids/gassmann.py)::

    def max_gas(temperature, pressure, api, specific_gravity=.56):
        """
        Computes the maximum dissolved gas in oil using Batzle and
        Wang (1992).

        Parameters
        ----------
        temperature : sequence
            Temperature in degrees Celsius
        pressure: sequence
            Pressure in MPa
        api : sequence
            Stock tank oil API
        specific_gravity : sequence
            Specific gravity of gas at STP, default is .56

        Returns
        -------
        max_gor : sequence
            Maximum dissolved gas in liters/liter

        Description
        -----------
        This estimate is based on equations given by Mavko, Mukerji,
        and Dvorkin, (1998, pp. 218-219, or 2003, p. 236) obtained
        originally from Batzle and Wang (1992).
        """
        code...

The docstring is 24 lines long.

Rather than using subsections, field lists (which exist now) can save
5 lines::

    def max_gas(temperature, pressure, api, specific_gravity=.56):
        """
        Computes the maximum dissolved gas in oil using Batzle and
        Wang (1992).

        :Parameters:
            temperature : sequence
                Temperature in degrees Celsius
            pressure: sequence
                Pressure in MPa
            api : sequence
                Stock tank oil API
            specific_gravity : sequence
                Specific gravity of gas at STP, default is .56
        :Returns:
            max_gor : sequence
                Maximum dissolved gas in liters/liter
        :Description:
            This estimate is based on equations given by Mavko,
            Mukerji, and Dvorkin, (1998, pp. 218-219, or 2003, p. 236)
            obtained originally from Batzle and Wang (1992).
        """
        code...

The output for field lists is typically a table structure.  For
example:

    :Parameters:
        temperature : sequence
            Temperature in degrees Celsius
        pressure: sequence
            Pressure in MPa
        api : sequence
            Stock tank oil API
        specific_gravity : sequence
            Specific gravity of gas at STP, default is .56
    :Returns:
        max_gor : sequence
            Maximum dissolved gas in liters/liter
    :Description:
        This estimate is based on equations given by Mavko,
        Mukerji, and Dvorkin, (1998, pp. 218-219, or 2003, p. 236)
        obtained originally from Batzle and Wang (1992).

But the definition lists describing the parameters and return values
are still wasteful of space.  There are a lot of half-filled lines.

Definition lists are currently defined as::

    term : classifier
        definition

Where the classifier part is optional.

* We could allow multiple classifiers::

      term : classifier one : two : three ...
          definition

* We could allow the definition on the same line as the term, but only
  in limited and well-known contexts::

      term -- definition

  This is the syntax used by StructuredText (one of reStructuredText's
  predecessors).  It was not adopted for reStructuredText because it
  is ambiguous -- people often use "--" in their text, as I just did.
  But given a constrained context, the ambiguity would be acceptable.
  That context would be: in docstrings, within a field list, perhaps
  only with certain well-defined field names (parameters, returns).

Recommendation
``````````````

Combining these ideas, the function definition becomes::

    def max_gas(temperature, pressure, api, specific_gravity=.56):
        """
        Computes the maximum dissolved gas in oil using Batzle and
        Wang (1992).

        :Parameters:
            temperature : sequence -- Temperature in degrees Celsius
            pressure: sequence -- Pressure in MPa
            api : sequence -- Stock tank oil API
            specific_gravity : sequence -- Specific gravity of gas at
                STP, default is .56
        :Returns:
            max_gor : sequence -- Maximum dissolved gas in liters/liter
        :Description:
            This estimate is based on equations given by Mavko,
            Mukerji, and Dvorkin, (1998, pp. 218-219, or 2003, p. 236)
            obtained originally from Batzle and Wang (1992).
        """
        code...

The docstring is reduced to 15 lines, from the original 24.  For
longer docstrings with many parameters and return values, the
difference would be more significant.


==================================
 Enthought API Documentation Tool
==================================
-----------------------
 Request for Proposals
-----------------------

:Author: Janet Swisher, Senior Technical Writer
:Organization: `Enthought, Inc. <http://www.enthought.com>`_
:Copyright: 2004 by Enthought, Inc.
:License: `Enthought License`_ (BSD Style)

.. _Enthought License: http://docutils.sf.net/licenses/enthought.txt

The following is excerpted from the full RFP, and is published here
with permission from `Enthought, Inc.`_  See the `Plan for Enthought
API Documentation Tool`__.

__ enthought-plan.html

.. contents::
.. sectnum::

Requirements
============

The documentation tool will address the following high-level goals:

Documentation Extraction
------------------------

1. Documentation will be generated directly from Python source code,
   drawing from the code structure, docstrings, and possibly other
   comments.

2. The tool will extract logical constructs as appropriate, minimizing
   the need for comments that are redundant with the code structure.
   The output should reflect both documented and undocumented
   elements.

Source Format
-------------

1. The docstrings will be formatted in as terse syntax as possible.
   Required tags, syntax, and white space should be minimized.

2. The tool must support the use of Traits.  Special comment syntax
   for Traits may be necessary.  Information about the Traits package
   is available at http://old.scipy.org/site_content/traits.  In the
   following example, each trait definition is prefaced by a plain
   comment::

       __traits__ = {

       # The current selection within the frame.
       'selection' : Trait([], TraitInstance(list)),

       # The frame has been activated or deactivated.
       'activated' : TraitEvent(),

       'closing' : TraitEvent(),

       # The frame is closed.
       'closed' : TraitEvent(),
       }

3. Support for ReStructuredText (ReST) format is desirable, because
   much of the existing docstrings uses ReST.  However, the complete
   ReST specification need not be supported, if a subset can achieve
   the project goals.  If the tool does not support ReST, the
   contractor should also provide a tool or path to convert existing
   docstrings.

Output Format
-------------

1. Documentation will be output as a navigable suite of HTML
   files.

2. The style of the HTML files will be customizable by a cascading
   style sheet and/or a customizable template.

3. Page elements such as headers and footer should be customizable, to
   support differing requirements from one documentation project to
   the next.

Output Structure and Navigation
-------------------------------

1. The navigation scheme for the HTML files should not rely on frames,
   and should harmonize with conversion to Microsoft HTML Help (.chm)
   format.

2. The output should be structured to make navigable the architecture
   of the Python code.  Packages, modules, classes, traits, and
   functions should be presented in clear, logical hierarchies.
   Diagrams or trees for inheritance, collaboration, sub-packaging,
   etc. are desirable but not required.

3. The output must include indexes that provide a comprehensive view
   of all packages, modules, and classes.  These indexes will provide
   readers with a clear and exhaustive view of the code base.  These
   indexes should be presented in a way that is easily accessible and
   allows easy navigation.

4. Cross-references to other documented elements will be used
   throughout the documentation, to enable the reader to move quickly
   relevant information.  For example, where type information for an
   element is available, the type definition should be
   cross-referenced.

5. The HTML suite should provide consistent navigation back to the
   home page, which will include the following information:

   * Bibliographic information

     - Author
     - Copyright
     - Release date
     - Version number

   * Abstract

   * References

     - Links to related internal docs (i.e., other docs for the same
       product)

     - Links to related external docs (e.g., supporting development
       docs, Python support docs, docs for included packages)

   It should be possible to specify similar information at the top
   level of each package, so that packages can be included as
   appropriate for a given application.

License
=======

Enthought intends to release the software under an open-source
("BSD-style") license.


===========================
 Docutils Project Policies
===========================

:Author: David Goodger; open to all Docutils developers
:Contact: goodger@python.org
:Date: $Date: 2004/06/25 12:56:27 $
:Revision: $Revision: 1.6 $
:Copyright: This document has been placed in the public domain.

.. contents::

A few quotes sum up the policies of the Docutils project.  The IETF's
classic credo (by MIT professor Dave Clark) is an ideal we can aspire
to:

    We reject: kings, presidents, and voting.  We believe in: rough
    consensus and running code.

As architect, chief cook and bottle-washer, I currently function as
BDFN (Benevolent Dictator For Now), but I would happily abdicate the
throne given a suitable candidate.  Any takers?

Eric S. Raymond, anthropologist of the hacker subculture, writes in
his essay `The Magic Cauldron`_:

    The number of contributors [to] projects is strongly and inversely
    correlated with the number of hoops each project makes a user go
    through to contribute.

    .. _The Magic Cauldron:
       http://www.tuxedo.org/~esr/writings/magic-cauldron/

Therefore, we will endeavour to keep the barrier to entry as low as
possible.  The policies below should not be thought of as barriers,
but merely as a codification of experience to date.  These are "best
practices", not absolutes; exceptions are expected, tolerated, and
used as a source of improvement.

As for control issues, Emmett Plant (CEO of the Xiph.org Foundation,
originators of Ogg Vorbis) put it well when he said:

    Open source dictates that you lose a certain amount of control
    over your codebase, and that's okay with us.

Python Coding Conventions
=========================

These are the conventions I use in my own code.  Contributed code will
not be refused merely because it does not strictly adhere to these
conditions; as long as it's internally consistent, clean, and correct,
it probably will be accepted.  But don't be surprised if the
"offending" code gets fiddled over time to conform to these
conventions.

The Docutils project shall follow the generic coding conventions as
specified in the `Style Guide for Python Code`_ and `Docstring
Conventions`_ PEPs, with the following clarifications (from most to
least important):

* 4 spaces per indentation level.  No tabs.

* Use only ASCII, no 8-bit strings.  See `Docutils
  Internationalization`_.

* No one-liner compound statements (i.e., no ``if x: return``: use two
  lines & indentation), except for degenerate class or method
  definitions (i.e., ``class X: pass`` is O.K.).

* Lines should be no more than 78 characters long.

* Use "StudlyCaps" for class names (except for element classes in
  docutils.nodes).

* Use "lowercase" or "lowercase_with_underscores" for function,
  method, and variable names.  For short names, maximum two words,
  joined lowercase may be used (e.g. "tagname").  For long names with
  three or more words, or where it's hard to parse the split between
  two words, use lowercase_with_underscores (e.g.,
  "note_explicit_target", "explicit_target").  If in doubt, use
  underscores.

* Use 'single quotes' for string literals, and """triple double
  quotes""" for docstrings.

.. _Style Guide for Python Code:
   http://www.python.org/peps/pep-0008.html
.. _Docstring Conventions: http://www.python.org/peps/pep-0257.html
.. _Docutils Internationalization: ../howto/i18n.html#python-code

Copyrights and Licensing
========================

The majority of the Docutils project code and documentation has been
placed in the public domain.  Unless clearly and explicitly indicated
otherwise, any patches (modifications to existing files) submitted to
the project for inclusion (via CVS, SourceForge trackers, mailing
lists, or private email) are assumed to be in the public domain as
well.

Any new files contributed to the project should clearly state their
intentions regarding copyright, in one of the following ways:

* Public domain (preferred): include the statement "This
  module/document has been placed in the public domain."

* Copyright & open source license: include a copyright notice, along
  with either an embedded license statement, a reference to an
  accompanying license file, or a license URL.

One of the goals of the Docutils project, once complete, is to be
incorporated into the Python standard library.  At that time copyright
of the Docutils code will be assumed by or transferred to the Python
Software Foundation (PSF), and will be released under Python's
license.  If the copyright/license option is chosen for new files, the
license should be compatible with Python's current license, and the
author(s) of the files should be willing to assign copyright to the
PSF.

CVS Check-ins
=============

Instructions for CVS access can be found at
http://sourceforge.net/cvs/?group_id=38414.  Anyone can access the CVS
repository anonymously.  Only project developers can make changes.
See `Setting Up For Docutils Development`_ below for more info.

Unless you really *really* know what you're doing, please limit your
CVS commands to ``cvs checkout``, ``cvs commit/checkin``, and ``cvs
add``.  Do **NOT** use ``cvs import`` unless you're absolutely sure
you know what you're doing.  Even then, grab a copy of the `nightly
CVS tarball`_, set it up on your own machine, and experiment *there*
first.

The `main source tree`_ ("docutils" CVS module) should always be kept
in a stable state (usable and as problem-free as possible).  The
Docutils project shall follow the `Python Check-in Policies`_ (as
applicable), with particular emphasis as follows:

* Before checking in any changes, run the entire Docutils test suite
  to be sure that you haven't broken anything.  From a shell::

      cd docutils/test
      alltests.py

* When adding new functionality (or fixing bugs), be sure to add test
  cases to the test suite.  Practise test-first programming; it's fun,
  it's addictive, and it works!

* The `sandbox CVS directory`_ is the place to put new, incomplete or
  experimental code.  See `Additions to Docutils`_ and `The Sandbox`_
  below.

* For bugs or omissions that have an obvious fix and can't possibly
  mess up anything else, go right ahead and check it in directly.

* For larger changes, use your best judgement.  If you're unsure of
  the impact, or feel that you require advice or approval, patches or
  `the sandbox`_ are the way to go.

Docutils will pursue an open and trusting policy for as long as
possible, and deal with any aberrations if (and hopefully not when)
they happen.  I'd rather see a torrent of loose contributions than
just a trickle of perfect-as-they-stand changes.  The occasional
mistake is easy to fix.  That's what CVS is for.

.. _main source tree:
   http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/docutils/docutils/
.. _Python Check-in Policies: http://www.python.org/dev/tools.html
.. _sandbox CVS directory:
   http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/docutils/sandbox/
.. _nightly CVS tarball:
   http://cvs.sourceforge.net/cvstarballs/docutils-cvsroot.tar.bz2

Additions to Docutils
---------------------

Additions to the project, such as new components, should be developed
in the `sandbox CVS directory`_ until they're in `good shape`_,
usable_, documented_, and `reasonably complete`_.  Adding to the `main
source tree`_ or to a `parallel project`_ implies a commitment to the
Docutils user community.

* Why the sandbox?

  Developers should be able to try out new components while they're
  being developed for addition to main source tree.  See `The
  Sandbox`_ below.

* _`Good shape` means that the component code is clean, readable, and
  free of junk code (unused legacy code; by analogy with "junk DNA").

* _`Usable` means that the code does what it claims to do.  An "XYZ
  Writer" should produce reasonable XYZ.

* _`Documented`: The more the better.  The modules/files must be at
  least minimally documented internally.  `Docutils Front-End Tools`_
  should have a new section for any front-end tool that is added.
  `Docutils Configuration Files`_ should be modified with any
  settings/options defined.

* _`Reasonably complete` means that the code must handle all input.
  Here "handle" means that no input can cause the code to fail (cause
  an exception, or silently and incorrectly produce nothing).
  "Reasonably complete" does not mean "finished" (no work left to be
  done).  For example, a writer must handle every standard element
  from the Docutils document model; for unimplemented elements, it
  must *at the very least* warn that "Output for element X is not yet
  implemented in writer Y".

If you really want to check code into the main source tree, you can,
but you'll have to be prepared to work on it intensively and complete
it quickly.  People will start to use it and they will expect it to
work!  If there are any issues with your code, or if you only have
time for gradual development, you should put it in the sandbox first.
It's easy to move code over to the main source tree once it's closer
to completion.

.. _Docutils Front-End Tools: ../user/tools.html
.. _Docutils Configuration Files: ../user/config.html

CVS Startup File
----------------

In your ~/.cvsrc file, make sure you have a line like this::

    update -dP

This ensures that new directories are created and empty directories
are pruned (removed) from your working copy.  In addition, these lines
are useful::

    cvs -q -z3
    diff -u

``cvs -q`` suppresses many non-essential messages.  ``-z3`` compresses
data between client and server.  ``diff -u`` creates "unified diffs";
specify ``diff -c`` if you prefer context diffs.

Setting Up For Docutils Development
-----------------------------------

When making changes to the code, good developers always test their
changes.  That means running the code to check that it produces the
expected results, and running the test suite too.  The modified
Docutils code has to be accessible to Python for the tests to have any
meaning.  There are two ways to keep the Docutils code accessible:

* Update your ``PYTHONPATH`` environment variable so that Python picks
  up your local working copy of the code.  This is the recommended
  method.

  For the bash shell and Docutils checked out from CVS in
  ``~/projects/docutils/``, add this to your ``~/.profile``::

      PYTHONPATH=$HOME/projects/docutils/docutils
      PYTHONPATH=$PYTHONPATH:$HOME/projects/docutils/docutils/extras
      export PYTHONPATH

  The first line points to the directory containing the ``docutils``
  package.  The second line adds the directory containing the
  third-party modules Docutils depends on.  The third line exports
  this environment variable.  You may also wish to add the ``tools``
  directory to your ``PATH``::

      PATH=$PATH:$HOME/projects/docutils/docutils/tools

* Before you run anything, every time you make a change, reinstall
  Docutils::

      python setup.py install

  .. CAUTION::

     This method is **not** recommended for day-to-day development;
     it's too easy to forget.  Confusion inevitably ensues.

     If you install Docutils this way, Python will always pick up the
     last-installed copy of the code.  If you ever forget to reinstall
     the "docutils" package, Python won't see your latest changes.

Mailing Lists
=============

Developers should subscribe to the mailing lists:

* The `Python Documentation Special Interest Group (Doc-SIG) mailing
  list`__ for high-level discussions on syntax, strategy, and design
  (email to Doc-SIG@python.org).
* Docutils-develop__, for implementation discussions
  (email to docutils-develop@lists.sourceforge.net).
* Docutils-checkins__, to monitor CVS checkin messages (automatically
  generated; normally read-only).
* Docutils-users__, to monitor and help out with usage issues and bug
  reports.

__ http://mail.python.org/mailman/listinfo/doc-sig
__ http://lists.sourceforge.net/lists/listinfo/docutils-develop
__ http://lists.sourceforge.net/lists/listinfo/docutils-checkins
__ http://lists.sourceforge.net/lists/listinfo/docutils-users

The Sandbox
===========

The `sandbox CVS directory`_ is a place to play around, to try out and
share ideas.  It's a part of the CVS repository but it isn't
distributed as part of Docutils releases.  Feel free to check in code
to the CVS sandbox; that way people can try it out but you won't have
to worry about it working 100% error-free, as is the goal of the `main
source tree`_.  Each developer who wants to play in the sandbox should
create either a project-specific subdirectory or personal subdirectory
(suggested name: SourceForge ID, nickname, or given name + family
initial).  It's OK to make a mess!  But please, play nice.

Please update the `sandbox README`_ file with links and a brief
description of your work.

In order to minimize the work necessary for others to install and try
out new, experimental components, the following sandbox directory
structure is recommended::

    sandbox/
        project_name/ # For a project where you invite contributions.
                      # Structure as in userid/component_name below.
        userid/       # For personal space.
            component_name/ # A verbose name is best.
                README.txt  # Please explain requirements,
                            # purpose/goals, and usage.
                docs/
                    ...
                component.py    # The component is a single module.
                            # *OR* (but *not* both)
                component/  # The component is a package.
                    __init__.py  # Contains the Reader/Writer class.
                    other1.py    # Other modules and data files used
                    data.txt     # by this component.
                    ...
                test/       # Test suite.
                    ...
                tools/      # For front ends etc.
                    ...
                setup.py    # Use Distutils to install the component
                            # code and tools/ files into the right
                            # places in Docutils.

Some sandbox projects are destined to become Docutils components once
completed.  Others, such as add-ons to Docutils or applications of
Docutils, graduate to become `parallel projects`_.

.. _sandbox README: http://docutils.sf.net/sandbox/README.html

.. _parallel project:

Parallel Projects
=================

Parallel projects contain useful code that is not central to the
functioning of Docutils.  Examples are specialized add-ons or
plug-ins, and applications of Docutils.  They use Docutils, but
Docutils does not require their presence to function.

An official parallel project will have its own CVS directory beside
(or parallel to) the main Docutils CVS directory.  It can have its own
web page in the docutils.sourceforge.net domain, its own file releases
and downloadable CVS snapshots, and even a mailing list if that proves
useful. However, an official parallel project has implications: it is
expected to be maintained and continue to work with changes to the
core Docutils.

A parallel project requires a project leader, who must commit to
coordinate and maintain the implementation:

* Answer questions from users and developers.
* Review suggestions, bug reports, and patches.
* Monitor changes and ensure the quality of the code and
  documentation.
* Coordinate with Docutils to ensure interoperability.
* Put together official project releases.

Of course, related projects may be created independently of Docutils.
The advantage of a parallel project is that the SourceForge
environment and the developer and user communities are already
established.  Core Docutils developers are available for consultation
and may contribute to the parallel project.  It's easier to keep the
projects in sync when there are changes made to the core Docutils
code.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


======================
 Python Source Reader
======================
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 1.7 $
:Date: $Date: 2004/06/05 19:19:08 $
:Copyright: This document has been placed in the public domain.

This document explores issues around extracting and processing
docstrings from Python modules.

For definitive element hierarchy details, see the "Python Plaintext
Document Interface DTD" XML document type definition, pysource.dtd_
(which modifies the generic docutils.dtd_).  Descriptions below list
'DTD elements' (XML 'generic identifiers' or tag names) corresponding
to syntax constructs.

.. contents::

Model
=====

The Python Source Reader ("PySource") model that's evolving in my mind
goes something like this:

1. Extract the docstring/namespace [#]_ tree from the module(s) and/or
   package(s).

   .. [#] See `Docstring Extractor`_ below.

2. Run the parser on each docstring in turn, producing a forest of
   doctrees (per nodes.py).

3. Join the docstring trees together into a single tree, running
   transforms:

   - merge hyperlinks
   - merge namespaces
   - create various sections like "Module Attributes", "Functions",
     "Classes", "Class Attributes", etc.; see pysource.dtd_
   - convert the above special sections to ordinary doctree nodes

4. Run transforms on the combined doctree.  Examples: resolving
   cross-references/hyperlinks (including interpreted text on Python
   identifiers); footnote auto-numbering; first field list ->
   bibliographic elements.

   (Or should step 4's transforms come before step 3?)

5. Pass the resulting unified tree to the writer/builder.

I've had trouble reconciling the roles of input parser and output
writer with the idea of modes ("readers" or "directors").  Does the
mode govern the tranformation of the input, the output, or both?
Perhaps the mode should be split into two.

For example, say the source of our input is a Python module.  Our
"input mode" should be the "Python Source Reader".  It discovers (from
``__docformat__``) that the input parser is "reStructuredText".  If we
want HTML, we'll specify the "HTML" output formatter.  But there's a
piece missing.  What *kind* or *style* of HTML output do we want?
PyDoc-style, LibRefMan style, etc.  (many people will want to specify
and control their own style).  Is the output style specific to a
particular output format (XML, HTML, etc.)?  Is the style specific to
the input mode?  Or can/should they be independent?

I envision interaction between the input parser, an "input mode" , and
the output formatter.  The same intermediate data format would be used
between each of these, being transformed as it progresses.

Docstring Extractor
===================

We need code that scans a parsed Python module, and returns an ordered
tree containing the names, docstrings (including attribute and
additional docstrings), and additional info (in parentheses below) of
all of the following objects:

- packages
- modules
- module attributes (+ values)
- classes (+ inheritance)
- class attributes (+ values)
- instance attributes (+ values)
- methods (+ formal parameters & defaults)
- functions (+ formal parameters & defaults)

(Extract comments too?  For example, comments at the start of a module
would be a good place for bibliographic field lists.)

In order to evaluate interpreted text cross-references, namespaces for
each of the above will also be required.

See python-dev/docstring-develop thread "AST mining", started on
2001-08-14.

Interpreted Text
================

DTD elements: package, module, class, method, function,
module_attribute, class_attribute, instance_attribute, variable,
parameter, type, exception_class, warning_class.

To classify identifiers explicitly, the role is given along with the
identifier in either prefix or suffix form::

    Use :method:`Keeper.storedata` to store the object's data in
    `Keeper.data`:instance_attribute:.

The role may be one of 'package', 'module', 'class', 'method',
'function', 'module_attribute', 'class_attribute',
'instance_attribute', 'variable', 'parameter', 'type',
'exception_class', 'exception', 'warning_class', or 'warning'.  Other
roles may be defined.

.. _pysource.dtd: pysource.dtd
.. _docutils.dtd: ../ref/docutils.dtd

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   fill-column: 70
   End:


=============================
 Docutils_ Release Procedure
=============================
:Author: David Goodger; open to all Docutils developers
:Contact: goodger@python.org
:Date: $Date: 2004/06/03 14:10:05 $
:Revision: $Revision: 1.1 $
:Copyright: This document has been placed in the public domain.

.. _Docutils: http://docutils.sourceforge.net/

1. Edit the version number in the following files:

   * docutils:

     - setup.py
     - HISTORY.txt
     - docutils/__init__.py

   * web: index.txt

2. Run the test suite: ``cd test ; alltests.py``.

3. Isolate from outside influence:

   (a) Remove the old installation from site-packages (including
       roman.py, and optparse.py, textwrap.py for pre-2.3
       installations).

   (b) Clear/unset the PYTHONPATH environment variable.

4. Create the release tarball:

   (a) Create a new empty directory and ``cd`` into it.

   (b) Get a clean snapshot of the CVS files::

           cvs -z3 -d:pserver:anonymous@cvs.sf.net:/cvsroot/docutils \
               export -rHEAD docutils

   (c) Use Distutils to create the release tarball::

           cd docutils
           python setup.py sdist

5. Expand and install the release tarball **in isolation** (as per
   step 3 above):

   (a) Expand the tarball in a new location, not over any existing
       files.

   (b) Install from expanded directory::

           cd docutils-X.Y
           python setup.py install

       The "install" command may require root permissions.

6. Run the test suite from the expanded archive directory: ``cd test ;
   alltests.py``.

7. Run ``cd tools ; buildhtml.py ..`` to confirm that there are no
   unexpected issues with the docs.

8. Upload the release tarball::

       $ ftp upload.sourceforge.net
       Connected to osdn.dl.sourceforge.net.
       ...
       Name (upload.sourceforge.net:david): anonymous
       331 Anonymous login ok, send your complete e-mail address as password.
       Password:
       ...
       230 Anonymous access granted, restrictions apply.
       ftp> bin
       200 Type set to I.
       ftp> cd /incoming
       250 CWD command successful.
       ftp> put filename

9. Log in to the SourceForge web interface.

10. Access the file release system on SourceForge (Admin interface).
    Fill in the fields:

        :Package ID:      docutils
        :Release Name:    <use release number only, e.g. 0.3>
        :Release Date:    <today's date>
        :Status:          Active
        :File Name:       <select the file just uploaded>
        :File Type:       Source .gz
        :Processor Type:  Platform-Independent
        :Release Notes:   <insert README.txt file here>
        :Change Log:      <insert summary from announcement>

        Also check the "Preserve my pre-formatted text" box.

11. Wait up to 30 minutes for the file to become available on
    SourceForge.

12. Download the release tarball and verify its integrity by walking
    through an installation, as outlined above (steps 5, 6, & 7).

13. Add a SourceForge News item, with title "Docutils 0.x released"
    and containing the release tarball's download URL.

14. Send announcement email to:

    * docutils-develop@lists.sourceforge.net
    * docutils-users@lists.sourceforge.net
    * doc-sig@python.org
    * python-list@python.org
    * python-announce@python.org

15. Register

    (a) with PyPI (Fill in details.  ``python setup.py register``?
        How to log in?)
    (b) with Vaults of Parnassus
    (c) with FreshMeat?

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


=====================
 Docstring Semantics
=====================
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 1.4 $
:Date: $Date: 2003/06/09 21:26:30 $
:Copyright: This document has been placed in the public domain.

These are notes for a possible future PEP providing the final piece of
the Python docstring puzzle: docstring semantics or documentation
methodology.  `PEP 257`_, Docstring Conventions, sketches out some
guidelines, but does not get into methodology details.

I haven't explored documentation methodology more because, in my
opinion, it is a completely separate issue from syntax, and it's even
more controversial than syntax.  Nobody wants to be told how to lay
out their documentation, a la JavaDoc_.  I think the JavaDoc way is
butt-ugly, but it *is* an established standard for the Java world.
Any standard documentation methodology has to be formal enough to be
useful but remain light enough to be usable.  If the methodology is
too strict, too heavy, or too ugly, many/most will not want to use it.

I think a standard methodology could benefit the Python community, but
it would be a hard sell.  A PEP would be the place to start.  For most
human-readable documentation needs, the free-form text approach is
adequate.  We'd only need a formal methodology if we want to extract
the parameters into a data dictionary, index, or summary of some kind.

PythonDoc
=========

(Not to be confused with Daniel Larsson's pythondoc_ project.)

A Python version of the JavaDoc_ semantics (not syntax).  A set of
conventions which are understood by the Docutils.  What JavaDoc has
done is to establish a syntax that enables a certain documentation
methodology, or standard *semantics*.  JavaDoc is not just syntax; it
prescribes a methodology.

- Use field lists or definition lists for "tagged blocks".  By this I
  mean that field lists can be used similarly to JavaDoc's ``@tag``
  syntax.  That's actually one of the motivators behind field lists.
  For example, we could have::

      """
      :Parameters:
          - `lines`: a list of one-line strings without newlines.
          - `until_blank`: Stop collecting at the first blank line if
            true (1).
          - `strip_indent`: Strip common leading indent if true (1,
            default).

      :Return:
          - a list of indented lines with mininum indent removed;
          - the amount of the indent;
          - whether or not the block finished with a blank line or at
            the end of `lines`.
      """

  This is taken straight out of docutils/statemachine.py, in which I
  experimented with a simple documentation methodology.  Another
  variation I've thought of exploits the Grouch_-compatible
  "classifier" element of definition lists.  For example::

      :Parameters:
          `lines` : [string]
              List of one-line strings without newlines.
          `until_blank` : boolean
              Stop collecting at the first blank line if true (1).
          `strip_indent` : boolean
              Strip common leading indent if true (1, default).

- Field lists could even be used in a one-to-one correspondence with
  JavaDoc ``@tags``, although I doubt if I'd recommend it.  Several
  ports of JavaDoc's ``@tag`` methodology exist in Python, most
  recently Ed Loper's "epydoc_".

Other Ideas
===========

- Can we extract comments from parsed modules?  Could be handy for
  documenting function/method parameters::

      def method(self,
                 source,        # path of input file
                 dest           # path of output file
                ):

  This would save having to repeat parameter names in the docstring.

  Idea from Mark Hammond's 1998-06-23 Doc-SIG post, "Re: [Doc-SIG]
  Documentation tool":

      it would be quite hard to add a new param to this method without
      realising you should document it

- Frederic Giacometti's `iPhrase Python documentation conventions`_ is
  an attachment to his Doc-SIG post of 2001-05-30.

.. _PEP 257: http://www.python.org/peps/pep-0257.html
.. _JavaDoc: http://java.sun.com/j2se/javadoc/
.. _pythondoc: http://starship.python.net/crew/danilo/pythondoc/
.. _Grouch: http://www.mems-exchange.org/software/grouch/
.. _epydoc: http://epydoc.sf.net/
.. _iPhrase Python documentation conventions:
   http://mail.python.org/pipermail/doc-sig/2001-May/001840.html

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


===================
 Docutils_ Testing
===================

:Author: Felix Wiemann
:Revision: $Revision: 1.2 $
:Date: $Date: 2004/06/27 14:37:24 $
:Copyright: This document has been placed in the public domain.

.. _Docutils: http://docutils.sourceforge.net/

.. contents::

This document describes how the tests are organized and how to add new
tests or modify existing tests.

Unit Tests
==========

Unit tests test single functions or modules (i.e. whitebox testing).

XXX Some details to be written.

Setting Up For Testing
----------------------

Recent versions of Python's ``unittest.py`` module have an annoying
new "feature", where newlines in test failure output are displayed as
"\n" (two characters) instead of as actual newlines.  This means that
test failure output is unreadable and unuseable.  Earlier versions
didn't have this feature, and the output could easily be copied and
pasted into test data modules.  Revision `1.7.2.1 of unittest.py`__
works well.  Download it, place it in the ``test/`` directory, and
enjoy much better output.

__ http://cvs.sf.net/viewcvs.py/*checkout*/python/python/dist/src/Lib/unittest.py?rev=1.7.2.1

.. _functional::

Functional Tests
================

The directory ``test/functional/`` contains data for functional tests.

Performing functional testing means testing the Docutils system as a
whole (i.e. blackbox testing).

Directory Structure
-------------------

+ ``functional/`` The main data directory.

  + ``input/`` The input files.

    - ``some_test.txt``, for example.

  + ``output/`` The actual output.

    - ``some_test.html``, for example.

  + ``expected/`` The expected output.

    - ``some_test.html``, for example.

  + ``tests/`` The config files for processing the input files.

    - ``some_test.py``, for example.

    - ``_default.py``, the `default configuration file`_.

The Testing Process
-------------------

When running ``test_functional.py``, all config files in
``functional/some_test.py`` are processed.

An example of what this means:

Provided ``functional/tests/some_test.py`` reads like this::

    # Source and destination file names.
    test_source = "some_test.txt"
    test_destination = "some_test.html"

    # Keyword parameters passed to publish_file.
    reader_name = "standalone"
    parser_name = "rst"
    writer_name = "html"
    settings_overrides['output-encoding'] = 'utf-8'

The two variables ``test_source`` and ``test_destination`` contain the
input file name (relative to ``functional/input/``) and the output
file name (relative to ``functional/output/`` and
``functional/expected/``).  Note that the file names can be chosen
arbitrarily.  However, the file names in ``functional/output/`` *must*
match the file names in ``functional/expected/``.

All other variables are passed as keyword arguments to
``docutils.core.publish_file``, so you can set reader, parser,
writer and anything else you want to configure.

Note that ``settings_overrides`` is already initialized as an empty
dictionary *before* the execution of the config file.  This is done in
order to allow subsequent assignments to ``settings_overrides`` in the
`default configuration file`_ and in the actual configuration file.

Creating New Tests
------------------

In order to create a new test, put the input test file into
``functional/input/``.  Then create a config file in
``functional/tests/`` which sets at least input and output file names,
reader, parser and writer.

Now run ``test_functional.py``.  The test will fail, of course,
because you do not have an expected output yet.

However, an output file will be generated in ``functional/output/``.
Check this output file for validity and correctness.  Then copy the
file to ``functional/expected/``.

If you run ``test_functional.py`` later and the actual output doesn't
match the expected output anymore, the test will fail.

If this is the case and you made an intentional change, check the
actual output for validity and correctness and copy it to
``functional/expected/``, overwriting the old expected output.

.. _default configuration file:

The Default Configuration File
------------------------------

The file ``functional/tests/_default.py`` contains default settings.
It is executed just before the actual configuration files, which has
the same effect as if the contents of ``_default.py`` were prepended
to every configuration file.


and empty component elements (

====================== Docutils_ To Do List ====================== :Author: David Goodger (with input from many); open to all Docutils developers :Contact: goodger@python.org :Date: $Date: 2004/06/28 19:42:03 $ :Revision: $Revision: 1.245 $ :Copyright: This document has been placed in the public domain. .. _Docutils: http://docutils.sourceforge.net/ .. contents:: Priority items are marked with "@" symbols. The more @s, the higher the priority. Items in question form (containing "?") are ideas which require more thought and debate; they are potential to-do's. Many of these items are awaiting champions. If you see something you'd like to tackle, please do! If there's something you'd like to see done but are unable to implement it yourself, `please consider contributing`__ in other ways. __ http://docutils.sf.net/#please-contribute Priorities ========== * Include substitution files for character entities, produced by the tools/unicode2rstsubs.py. As static data, these files could go inside the docutils package somewhere. * Rename front-end tools (perhaps to "rst2*" pattern) and have setup.py install them. * A Python Source Reader component (Python auto-documentation) will be added. See the document `"Plan for Enthought API Documentation Tool"`__ for details. If you'd like to help, let me know! __ enthought-plan.html General ======= * Refactor - Rename methods & variables according to the `Python coding conventions `_. - The name-to-id conversion and hyperlink resolution code needs to be checked for correctness and refactored. I'm afraid it's a bit of a spaghetti mess now. * Add validation? See http://pytrex.sourceforge.net, RELAX NG, pyRXP. * Ask Python-dev for opinions (GvR for a pronouncement) on special variables (__author__, __version__, etc.): convenience vs. namespace pollution. Ask opinions on whether or not Docutils should recognize & use them. * In ``docutils.readers.get_reader_class`` (& ``parsers`` & ``writers`` too), should we be importing "standalone" or "docutils.readers.standalone"? (This would avoid importing top-level modules if the module name is not in docutils/readers. Potential nastiness.) * @@@ Perhaps store a _`name-to-id mapping file`? This could be stored permanently, read by subsequent processing runs, and updated with new entries. ("Persistent ID mapping"?) * Need a Unicode to HTML entities codec for HTML writer? No, the "xmlcharrefreplace" output error handler is sufficient. Make it the default for HTML & XML writers? * Perhaps the ``Component.supports`` method should deal with individual features ("meta" etc.) instead of formats ("html" etc.)? * Add /usr/etc/docutils.conf to config file list? System-wide, whereas /etc/docutils.conf is machine-specific. /usr/local/etc/docutils.conf too? See the `Filesystem Hierarchy Standard`_. .. _Filesystem Hierarchy Standard: http://www.pathname.com/fhs * Add _`object numbering and object references` (tables & figures). These would be the equivalent of DocBook's "formal" elements. We may need _`persistent sequences`, such as chapter numbers. See `OpenOffice.org XML`_ "fields". Should the sequences be automatic or manual (user-specifyable)? We need to name the objects: - "name" option for the "figure" directive? :: .. figure:: image.png :name: image's name Same for the "table" directive:: .. table:: optional title here :name: table's name ===== ===== x not x ===== ===== True False False True ===== ===== This would also allow other options to be set, like border styles. The same technique could be used for other objects. A preliminary "table" directive has been implemented, supporting table titles. Perhaps the name should derive from the title. - The object could also be done this way:: .. _figure name: .. figure:: image.png This may be a more general solution, equally applicable to tables. However, explicit naming using an option seems simpler to users. We'll also need syntax for object references. See `OpenOffice.org XML`_ "reference fields": - Parameterized substitutions? For example:: See |figure (figure name)| on |page (figure name)|. .. |figure (name)| figure-ref:: (name) .. |page (name)| page-ref:: (name) The result would be:: See figure 3.11 on page 157. But this would require substitution directives to be processed at reference-time, not at definition-time as they are now. Or, perhaps the directives could just leave ``pending`` elements behind, and the transforms do the work? How to pass the data through? Too complicated. - An interpreted text approach is simpler and better:: See :figure:`figure name` on :page:`figure name`. The "figure" and "page" roles could generate appropriate boilerplate text. The position of the role (prefix or suffix) could also be utilized. See `Interpreted Text`_ below. .. _OpenOffice.org XML: http://xml.openoffice.org/ * Think about _`large documents` made up of multiple subdocument files. Issues: continuity (`persistent sequences`_ above), cross-references (`name-to-id mapping file`_ above and `targets in other documents`_ below). When writing a book, the author probably wants to split it up into files, perhaps one per chapter (but perhaps even more detailed). However, we'd like to be able to have references from one chapter to another, and have continuous numbering (pages and chapters, as applicable). Of course, none of this is implemented yet. There has been some thought put into some aspects; see `the "include" directive`__ and the `Reference Merging`_ transform below. When I was working with SGML in Japan, we had a system where there was a top-level coordinating file, book.sgml, which contained the top-level structure of a book: the element, containing the book and empty component elements (<preface>, <chapter>, <appendix>, etc.), each with filename attributes pointing to the actual source for the component. Something like this:: <book id="bk01"> <title>Title of the Book (The "inrefid" attribute stood for "insertion reference ID".) The processing system would process each component separately, but it would recognize and use the book file to coordinate chapter and page numbering, and keep a persistent ID to (title, page number) mapping database for cross-references. Docutils could use a similar system for large-scale, multipart documents. __ ../ref/rst/directives.html#including-an-external-document-fragment Aahz's idea: First the ToC:: .. ToC-list:: Introduction.txt Objects.txt Data.txt Control.txt Then a sample use:: .. include:: ToC.txt As I said earlier in chapter :chapter:`Objects.txt`, the reference count gets increased every time a binding is made. Which produces:: As I said earlier in chapter 2, the reference count gets increased every time a binding is made. The ToC in this form doesn't even need to be references to actual reST documents; I'm simply doing it that way for a minimum of future-proofing, in case I do want to add the ability to pick up references within external chapters. Perhaps, instead of ToC (which would overload the "contents" directive concept already in use), we could use "manifest". A "manifest" directive might associate local reference names with files:: .. manifest:: intro: Introduction.txt objects: Objects.txt data: Data.txt control: Control.txt Then the sample becomes:: .. include:: manifest.txt As I said earlier in chapter :chapter:`objects`, the reference count gets increased every time a binding is made. * Add testing for Docutils' front end tools? * Changes to sandbox/davidg/infrastructure/docutils-update? - Modify the script to only update the snapshots if files have actually changed in CVS (saving some SourceForge server cycles). - Make passing the test suite a prerequisite to snapshot update, but only if the process is completely automatic. - Rewrite in Python? * Publisher: "Ordinary setup" shouldn't requre specific ordering; at the very least, there ought to be error checking higher up in the call chain. [Aahz] ``Publisher.get_settings`` requires that all components be set up before it's called. Perhaps the I/O *objects* shouldn't be set, but I/O *classes*. Then options are set up (``.set_options``), and ``Publisher.set_io`` (or equivalent code) is called with source & destination paths, creating the I/O objects. Perhaps I/O objects shouldn't be instantiated until required. For split output, the Writer may be called multiple times, once for each doctree, and each doctree should have a separate Output object (with a different path). Is the "Builder" pattern applicable here? * Perhaps I/O objects should become full-fledged components (i.e. subclasses of ``docutils.Component``, as are Readers, Parsers, and Writers now), and thus have associated option/setting specs and transforms. * Multiple file I/O suggestion from Michael Hudson: use a file-like object or something you can iterate over to get file-like objects. * Language modules: in accented languages it may be useful to have both accented and unaccented entries in the ``bibliographic_fields`` mapping for versatility. * Add a "--strict-language" option & setting: no English fallback for language-dependent features. * Add an "--input-language" option & setting? Specify a different language module for input (bibliographic fields, directives) than for output. The "--language" option would set both input & output languages. * Auto-generate reference tables for language-dependent features? Could be generated from the source modules. A special command-line option could be added to Docutils front ends to do this. (Idea from Engelbert Gruber.) * Change the "class" attribute of elements (set with Element.set_class) to a list? * Enable feedback of some kind from internal decisions, such as reporting the successful input encoding. Modify runtime settings? System message? Simple stderr output? * Rationalize Writer settings (HTML/LaTeX/PEP) -- share settings. * The "docutils.conf" included with Docutils should become complete, with examples of every setting (many/most commented out). It's currently sparse, requiring doc lookups. * Merge docs/user/latex.txt info into tools.txt and config.txt. * Add internationalization to _`footer boilerplate text` (resulting from "--generator", "--source-link", and "--date" etc.), allowing translations. * Add an "--include file" command-line option (config setting too?), equivalent to ".. include:: file" as the first line of the doc text? Especially useful for character entity sets, text transform specs, boilerplate, etc. * Parameterize the Reporter object or class? See the `2004-02-18 "rest checking and source path"`_ thread. .. _2004-02-18 "rest checking and source path": http://thread.gmane.org/gmane.text.docutils.user/1112 * Add a "disable_transforms" setting? And a dummy Writer subclass that does nothing when its .write() method is called? Would allow for easy syntax checking. See the `2004-02-18 "rest checking and source path"`_ thread. * Add a generic meta-stylesheet mechanism? An external file could associate style names ("class" attributes) with specific elements. Could be generalized to arbitrary output attributes; useful for HTML & XMLs. Aahz implemented something like this in sandbox/aahz/Effective/EffMap.py. * William Dode suggested that table cells be assigned "class" attributes by columns, so that stylesheets can affect text alignment. Unfortunately, there doesn't seem to be a way (in HTML at least) to leverage the "colspec" elements (HTML "col" tags) by adding classes to them. The resulting HTML is very verbose:: 111 222 ... At the very least, it should be an option. People who don't use it shouldn't be penalized by increases in their HTML file sizes. Table rows could also be assigned classes (like odd/even). That would be easier to implement. How should it be implemented? * There could be writer options (column classes & row classes) with standard values. * The table directive could grow some options. Something like ":cell-classes: col1 col2 col3" (either must match the number of columns, or repeat to fill?) and ":row-classes: odd even" (repeat to fill; body rows only, or header rows too?). Probably per-table directive options are best. The "class" values could be used by any writer, and applying such classes to all tables in a document with writer options is too broad. * Make the csv-table directive work with Python 2.1/2.2. (See the discussion_ on the mailing list.) .. _discussion: http://thread.gmane.org/gmane.text.docutils.user/1319 Documentation ============= User Docs --------- What else is needed? Developer Docs -------------- * Complete `Docutils Runtime Settings <../api/runtime-settings.html>`_. * @@@ Improve the internal module documentation (docstrings in the code). Specific deficiencies listed below. - docutils.parsers.rst.states.State.build_table: data structure required (including StringList). - docutils.parsers.rst.states: more complete documentation of parser internals. * docs/ref/doctree.txt: DTD element structural relationships, semantics, and attributes. In progress; element descriptions to be completed. * Document the ``pending`` elements, how they're generated and what they do. * Document the transforms (perhaps in docstrings?): how they're used, what they do, dependencies & order considerations. * Document the HTML classes used by html4css1.py. * Write an overview of the Docutils architecture, as an introduction for developers. What connects to what, why, and how. Either update PEP 258 (see PEPs_ below) or as a separate doc. * Give information about unit tests. Maybe as a howto? How-Tos ------- * Creating Docutils Writers * Creating Docutils Readers * Creating Docutils Transforms * Creating Docutils Parsers * Using Docutils as a Library PEPs ---- * Complete PEP 258 Docutils Design Specification. - Fill in the blanks in API details. - Specify the nodes.py internal data structure implementation? [Tibs:] Eventually we need to have direct documentation in there on how it all hangs together - the DTD is not enough (indeed, is it still meant to be correct? [Yes, it is. --DG]). * Rework PEP 257, separating style from spec from tools, wrt Docutils? See Doc-SIG from 2001-06-19/20. Python Source Reader ==================== General: * Analyze Tony Ibbs' PySource code. * Analyze Doug Hellmann's HappyDoc project. * Investigate how POD handles literate programming. * Take the best ideas and integrate them into Docutils. Miscellaneous ideas: * If we can detect that a comment block begins with ``##``, a la JavaDoc, it might be useful to indicate interspersed section headers & explanatory text in a module. For example:: """Module docstring.""" ## # Constants # ========= a = 1 b = 2 ## # Exception Classes # ================= class MyException(Exception): pass # etc. * Should standalone strings also become (module/class) docstrings? Under what conditions? We want to prevent arbitrary strings from becomming docstrings of prior attribute assignments etc. Assume that there must be no blank lines between attributes and attribute docstrings? (Use lineno of NEWLINE token.) Triple-quotes are sometimes used for multi-line comments (such as commenting out blocks of code). How to reconcile? * HappyDoc's idea of using comment blocks when there's no docstring may be useful to get around the conflict between `additional docstrings`_ and ``from __future__ import`` for module docstrings. A module could begin like this:: #!/usr/bin/env python # :Author: Me # :Copyright: whatever """This is the public module docstring (``__doc__``).""" # More docs, in comments. # All comments at the beginning of a module could be # accumulated as docstrings. # We can't have another docstring here, because of the # ``__future__`` statement. from __future__ import division Using the JavaDoc convention of a doc-comment block beginning with ``##`` is useful though. It allows doc-comments and implementation comments. .. _additional docstrings: ../peps/pep-0258.html#additional-docstrings * HappyDoc uses an initial comment block to set "parser configuration values". Do the same thing for Docutils, to set runtime settings on a per-module basis? I.e.:: # Docutils:setting=value Could be used to turn on/off function parameter comment recognition & other marginal features. Could be used as a general mechanism to augment config files and command-line options (but which takes precedence?). * Multi-file output should be divisible at arbitrary level. * Support all forms of ``import`` statements: - ``import module``: listed as "module" - ``import module as alias``: "alias (module)" - ``from module import identifier``: "identifier (from module)" - ``from module import identifier as alias``: "alias (identifier from module)" - ``from module import *``: "all identifiers (``*``) from module" * Have links to colorized Python source files from API docs? And vice-versa: backlinks from the colorized source files to the API docs! * In summaries, use the first *sentence* of a docstring if the first line is not followed by a blank line. reStructuredText Parser ======================= Also see the `... Or Not To Do?`__ list. __ rst/alternatives.html#or-not-to-do * Clean up the code; refactor as required. * Add motivation sections for constructs in spec. * Document title should grow an implicit target. * Allow very long titles (on two or more lines)? * And for the sake of completeness, should definition list terms be allowed to be very long (two or more lines) also? * Support generic hyperlink references to _`targets in other documents`? Not in an HTML-centric way, though (it's trivial to say ``http://www.example.com/doc#name``, and useless in non-HTML contexts). XLink/XPointer? ``.. baseref::``? See Doc-SIG 2001-08-10. * .. _adaptable file extensions: In target URLs, it would be useful to not explicitly specify the file extension. If we're generating HTML, then ".html" is appropriate; if PDF, then ".pdf"; etc. How about using ".*" to indicate "choose the most appropriate filename extension? For example:: .. _Another Document: another.* Should the choice be from among existing files only? Documents only, or objects (images, etc.) also? (How to differentiate? Element context [within "image"]?) This may not be just a parser issue though; it may need framework support. Mailing list thread: `Images in both HTML and LaTeX`__ __ http://thread.gmane.org/gmane.text.docutils.user/1239 * Implement the header row separator modification to table.el. (Wrote to Takaaki Ota & the table.el mailing list on 2001-08-12, suggesting support for "=====" header rows. On 2001-08-17 he replied, saying he'd put it on his to-do list, but "don't hold your breath".) * Tony says inline markup rule 7 could do with a *little* more exposition in the spec, to make clear what is going on for people with head colds. * @@ Fix the parser's indentation handling to conform with the stricter definition in the spec. (Explicit markup blocks should be strict or forgiving?) * @@ Tighten up the spec for indentation of "constructs using complex markers": field lists and option lists? Bodies may begin on the same line as the marker or on a subsequent line (with blank lines optional). Require that for bodies beginning on the same line as the marker, all lines be in strict alignment. Currently, this is acceptable:: :Field-name-of-medium-length: Field body beginning on the same line as the field name. This proposal would make the above example illegal, instead requiring strict alignment. A field body may either begin on the same line:: :Field-name-of-medium-length: Field body beginning on the same line as the field name. Or it may begin on a subsequent line:: :Field-name-of-medium-length: Field body beginning on a line subsequent to that of the field name. This would be especially relevant in degenerate cases like this:: :Number-of-African-swallows-requried-to-carry-a-coconut: It would be very difficult to align the field body with the left edge of the first line if it began on the same line as the field name. * Allow for variant styles by interpreting _`indented lists` as if they weren't indented? For example, currently the list below will be parsed as a list within a block quote:: paragraph * list item 1 * list item 2 But a lot of people seem to write that way, and HTML browsers make it look as if that's the way it should be. The parser could check the contents of block quotes, and if they contain only a single list, remove the block quote wrapper. There would be two problems: 1. What if we actually *do* want a list inside a block quote? 2. What if such a list comes immediately after an indented construct, such as a literal block? Both could be solved using empty comments (problem 2 already exists for a block quote after a literal block). But that's a hack. Perhaps a runtime setting, allowing or disabling this convenience, would be appropriate. But that raises issues too: User A, who writes lists indented (and their config file is set up to allow it), sends a file to user B, who doesn't (and their config file disables indented lists). The result of processing by the two users will be different. It may seem minor, but it adds ambiguity to the parser, which is bad. See the `Doc-SIG discussion starting 2001-04-18`__ with Ed Loper's "Structuring: a summary; and an attempt at EBNF", item 4 (and follow-ups, here__ and here__). Also `docutils-users, 2003-02-17`__ and `beginning 2003-08-04`__. __ http://mail.python.org/pipermail/doc-sig/2001-April/001776.html __ http://mail.python.org/pipermail/doc-sig/2001-April/001789.html __ http://mail.python.org/pipermail/doc-sig/2001-April/001793.html __ http://sourceforge.net/mailarchive/message.php?msg_id=3838913 __ http://sf.net/mailarchive/forum.php?thread_id=2957175&forum_id=11444 * Make the parser modular. Allow syntax constructs to be added or disabled at run-time. Or is subclassing enough? * Continue to report (info, level 1) enumerated lists whose start value is not ordinal-1? * Generalize the "doctest block" construct (which is overly Python-centric) to other interactive sessions? "Doctest block" could be renamed to "I/O block" or "interactive block", and each of these could also be recognized as such by the parser: - Shell sessions:: $ cat example1.txt A block beginning with a "$ " prompt is interpreted as a shell session interactive block. As with Doctest blocks, the interactive block ends with the first blank line, and wouldn't have to be indented. - Root shell sessions:: # cat example2.txt A block beginning with a "# " prompt is interpreted as a root shell session (the user is or has to be logged in as root) interactive block. Again, the block ends with a blank line. Other standard (and unambiguous) interactive session prompts could easily be added (such as "> " for WinDOS). Tony Ibbs spoke out against this idea (2002-06-14 Doc-SIG thread "docutils feedback"). * Should the "doctest" element go away, and the construct simply be a front-end to generic literal blocks? * Add support for pragma (syntax-altering) directives. Some pragma directives could be local-scope unless explicitly specified as global/pragma using ":global:" options. * Remove leading numbers from section titles for implicit link names? A section titled "3. Conclusion" could then be referred to by "``Conclusion_``" (i.e., without the "3."). * `Syntax for line blocks`__ (currently a directive)? __ rst/alternatives.html#syntax-for-line-blocks * Implement auto-enumerated lists? See `Auto-Enumerated Lists`__. __ rst/alternatives.html#auto-enumerated-lists * Support whitespace in angle-bracketed standalone URLs according to Appendix E ("Recommendations for Delimiting URI in Context") of `RFC 2396`_. .. _RFC 2396: http://www.rfc-editor.org/rfc/rfc2396.txt * Use the vertical spacing of the source text to determine the corresponding vertical spacing of the output? * [From Mark Nodine] For cells in simple tables that comprise a single line, the justification can be inferred according to the following rules: 1. If the text begins at the leftmost column of the cell, then left justification, ELSE 2. If the text begins at the rightmost column of the cell, then right justification, ELSE 3. Center justification. The onus is on the author to make the text unambiguous by adding blank columns as necessary. There should be a parser setting to turn off justification-recognition (normally on would be fine). Decimal justification? * Make enumerated list parsing more strict, so that this would parse as a paragraph with an info message:: 1. line one 3. line two * Generalize the "target-notes" directive into a command-line option somehow? See docutils-develop 2003-02-13. * Include the _`character entity substitution definition files` `temporarily stored here `__, perhaps in a ``docutils/parsers/rst/includes/`` directory. See `misc.include`_ below. * Should ^L (or something else in reST) be defined to mean force/suggest page breaks in whatever output we have? A "break" or "page-break" directive would be easy to add. A new doctree element would be required though (perhaps "break"). The final behavior would be up to the Writer. The directive argument could be one of page/column/recto/verso for added flexibility. Currently ^L (Python's "\f") characters are treated as whitespace. They're converted to single spaces, actually, as are vertical tabs (^K, Python's "\v"). It would be possible to recognize form feeds as markup, but it requires some thought and discussion first. Are there any downsides? Many editing environments do not allow the insertion of control characters. Will it cause any harm? It would be useful as a shorthand for the directive. It's common practice to use ^L before Emacs "Local Variables" lists:: ^L .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: These are already present in many PEPs and Docutils project documents. From the Emacs manual (info): A "local variables list" goes near the end of the file, in the last page. (It is often best to put it on a page by itself.) It would be unfortunate if this construct caused a final blank page to be generated (for those Writers that recognize the page breaks). We'll have to add a transform that looks for a "break" plus zero or more comments at the end of a document, and removes them. * Could the "break" concept above be extended to inline forms? E.g. "^L" in the middle of a sentence could cause a line break. Only recognize it at the end of a line (i.e., "\f\n")? Or is formfeed inappropriate? Perhaps vertical tab ("\v"), but even that's a stretch. Can't use carriage returns, since they're commonly used for line endings. * Allow a "::"-only paragraph (first line, actually) to introduce a literal block without a blank line? (Idea from Paul Moore.) :: :: This is a literal block Is indentation enough to make the separation between a paragraph which contains just a ``::`` and the literal text unambiguous? There's one problem with this concession. What if one wants a definition list item which defines the term "::"? We'd have to escape it. Currenty, "\::" doesn't work (although it should; **bug**), and ":\:" is misinterpreted as a field name (name "\"; also a **bug**). Assuming these bugs are squashed, I suppose it's a useful special case. It would only be reasonable to apply it to "::"-only paragraphs though. I think the blank line is visually necessary if there's text before the "::":: The text in this paragraph needs separation from the literal block following:: This doesn't look right. Another idea. Would it be worthwhile to allow literal blocks to begin without a newline after the "::"? Example:: :: while True: print 'hello world' Perhaps. Perhaps not. * Add new syntax for _`nested inline markup`? Or extend the parser to parse nested inline markup somehow? See the `collected notes `__. * Drop the backticks from embedded URIs with omitted reference text? Should the angle brackets be kept in the output or not? :: _ Probably not worth the trouble. * Add ``^superscript^`` inline markup? The only common non-markup uses of "^" I can think of are as short hand for "superscript" itself and for describing control characters ("^C to cancel"). The former supports the proposed syntax, and it could be argued that the latter ought to be literal text anyhow (e.g. "``^C`` to cancel"). * Add _`math markup`. We should try for a general solution, that's applicable to any output format. Using a standard, such as MathML_, would be best. TeX (or itex_) would be acceptable as a *front-end* to MathML. See `the culmination of a relevant discussion `__. Both a directive and an interpreted text role will be necessary (for each markup). Directive example:: .. itex:: \alpha_t(i) = P(O_1, O_2, \dots O_t, q_t = S_i \lambda) The same thing inline:: The equation in question is :itex:`\alpha_t(i) = P(O_1, O_2, \dots O_t, q_t = S_i \lambda)`. .. _MathML: http://www.w3.org/TR/MathML2/ .. _itex: http://pear.math.pitt.edu/mathzilla/itex2mmlItex.html * How about a syntax for alternative hyperlink behavior, such as "open in a new window" (as in HTML's ````)? Double angle brackets might work for inline targets:: The `reference docs <>`__ may be handy. But what about explicit targets? The MoinMoin wiki uses a caret ("^") at the beginning of the URL ("^" is not a legal URI character). That could work for both inline and explicit targets:: The `reference docs <^url>`__ may be handy. .. _name: ^url * Allow _`transitions` between sections? A transition immediately preceding a section title could be interpreted as a sibling of that section. The context would obviate the need for a special directive. The document model would have to accommodate the change:: * Add an option to allow arbitrary URI schemes (not just those in urischemes.py)? This would make text like "signal:noise" into a URI. * Add an option to add URI schemes at runtime. * _`Segmented lists`:: : segment : segment : segment : segment : segment : very long segment : segment : segment : segment The initial colon (":") can be thought of as a type of bullet We could even have segment titles:: :: title : title : title : segment : segment : segment : segment : segment : segment This would correspond well to DocBook's SegmentedList. Output could be tabular or "name: value" pairs, as described in DocBook's docs. * Allow backslash-escaped colons in field names:: :Case Study\: Event Handling: This chapter will be dropped. * _`footnote spaces`: When using superscripted footnote references, the space looks ugly. We could automatically activate --trim-footnote-reference-space when --footnote-references=superscript (or --use-latex-footnotes for the LaTeX writer) is supplied. There should be also an option to deactivate the space-trimming, e.g. --leave-footnote-reference-space. The implementation logic is a bit tricky: --leave-footnote-reference-space can't just be "settings.trim_footnote_reference_space = False", because of the automatic activation, unless the order of options is significant. If the order is significant, the automatic activation has to take place in the option-handling code. But config files are handled after the fact, so order can't be relied upon there. So they have to be two independent settings. * Enable _`tables inside XML comments`, where "--" ends comments. I see three implementation possibilities: 1. Make the table syntax characters into "table" directive options. This is the most flexible but most difficult, and we probably don't need that much flexibility. 2. Substitute "~" for "-" with a specialized directive option (e.g. ":tildes:"). 3. Make the standard table syntax recognize "~" as well as "-", even without a directive option. Individual tables would have to be internally consistent. Directive options are preferable to configuration settings, because tables are document-specific. A pragma directive would be another approach, to set the syntax once for a whole document. Directives ---------- Directives below are often referred to as "module.directive", the directive function. The "module." is not part of the directive name when used in a document. * Allow directives to be added at run-time? * Use the language module for directive option names? * Add "substitution_only" and "substitution_ok" function attributes, and automate context checking? * Change directive functions to directive classes? Superclass' ``__init__()`` could handle all the bookkeeping. * Implement options on existing directives: - Add a "name" option to directives, to set an author-supplied identifier? - All directives that produce titled elements should grow implicit reference names based on the titles. - _`images.image`: "border"? _`Units of measure`? (See `docutils-users, 2003-03-02 `__, and `docutils-develop, 2004-04-29 `_.) - _`images.figure`: "title" and "number", to indicate a formal figure? - _`parts.sectnum`: "local"?, "start", "refnum" A "local" option could enable numbering for sections from a certain point down, and sections in the rest of the document are not numbered. For example, a reference section of a manual might be numbered, but not the rest. OTOH, an all-or-nothing approach would probably be enough. The "start" option will specify the sequence set to use at the same time as the starting value, for the first part of the section number (i.e., section, not subsection). For example:: .. sectnum: :start: 1 .. sectnum: :start: A .. sectnum: :start: 5 .. sectnum: :start: I The first one is the default: start at 1, numbered. The second one specifies letters, and start at "A". The third specifies numbers, start at 5. The last example could signal Roman numerals, although I don't know if they'd be applicable here. Enumerated lists already do all this; perhaps that code could be reused. Here comes the tricky part. The "sectnum" directive should be usable multiple times in a single document. For example, in a long document with "chapter" and "appendix" sections, there could be a second "sectnum" before the first appendix, changing the sequence used (from 1,2,3... to A,B,C...). This is where the "local" concept comes in. This part of the implementation can be left for later. A "refnum" option (better name?) would insert reference names (targets) consisting of the reference number. Then a URL could be of the form ``http://host/document.html#2.5`` (or "2-5"?). Allow internal references by number? Allow name-based *and* number-based ids at the same time, or only one or the other (which would the table of contents use)? Usage issue: altering the section structure of a document could render hyperlinks invalid. - _`parts.contents`: Add a "suppress" or "prune" option? It would suppress contents display for sections in a branch from that point down. Or a new directive, like "prune-contents"? Add an option to include topics in the TOC? Another for sidebars? See docutils-develop 2003-01-29. - _`misc.include`: - @@@ Allow whitespace in paths (just not at line boundaries). - "encoding" option? Take default from runtime settings. Use Input component to read it in? - Option to select a range of lines? - Option to label lines? - Default directory for "built-in includes", using the C syntax ``#include ``? Use C-preprocessor semantics for locating include files? E.g., ``.. include:: file.txt`` will read another file into the current one, relative to the current file's directory, and ``.. include:: `` will read a standard include file from ``docutils/include/``. (Should "quotes" be required around non-standard include files?) -- http://sf.net/mailarchive/message.php?msg_id=1938401 I now think that ``docutils/parsers/rst/include/`` is a better place for these files, since they're reStructuredText-specific. Keeping standard data files together with the package code makes sense to me. It seems much less complex to implement than a separate system data directory, such as ``/usr/share/docutils``. Any reason a system data directory should be used? How does Distutils handle data files? How about an environment variable, say RSTINCLUDEPATH or RSTPATH? This could be combined with a setting/option to allow user-defined include directories. For a specific application, see the discussion of `character entity substitution definition files`_ above. Instead of C-include "", perhaps a new "include-sys" directive? - _`misc.raw`: "encoding" option? - _`misc.class`: @@@ Allow multiple classes (whitespace-separated) to be declared. * Implement directives. Each of the list items below begins with an identifier of the form, "module_name.directive_function_name". The directive name itself could be the same as the directive_function_name, or it could differ. - _`html.imagemap` (Useful outside of HTML? If not, replace with image only in non-HTML writers?) - _`parts.endnotes` (or "footnotes"): See `Footnote & Citation Gathering`_. - _`parts.citations`: See `Footnote & Citation Gathering`_. - _`misc.exec`: Execute Python code & insert the results. Perhaps dangerous? Call it "python" to allow for other languages? - _`misc.system`?: Execute an ``os.system()`` call, and insert the results (possibly as a literal block). Definitely dangerous! How to make it safe? Perhaps such processing should be left outside of the document, in the user's production system (a makefile or a script or whatever). Or, the directive could be disabled by default and only enabled with an explicit command-line option or config file setting. Even then, an interactive prompt may be useful, such as: The file.txt document you are processing contains a "system" directive requesting that the ``sudo rm -rf /`` command be executed. Allow it to execute? (y/N) - _`misc.eval`: Evaluate an expression & insert the text. At parse time or at substitution time? Dangerous? Perhaps limit to canned macros; see text.date_ below. - _`misc.encoding`: Specify the character encoding of the input data. But there are problems: - When it sees the directive, the parser will already have read the input data, and encoding determination will already have been done. - If a file with an "encoding" directive is edited and saved with a different encoding, the directive may cause data corruption. - _`misc.language`: Specify the language of a document. There is a problem similar to the first problem listed for misc.encoding_, although to a lesser degree. - _`misc.settings`: Set any Docutils runtime setting from within a document? - _`misc.charents`: Equivalent to:: .. include:: {includepath}/charents.txt - .. _conditional directives: Docutils already has the ability to say "use this content for Writer X" (via the "raw" directive), but it doesn't have the ability to say "use this content for any Writer other than X". It wouldn't be difficult to add this ability though. My first idea would be to add a set of conditional directives. Let's call them "writer-is" and "writer-is-not" for discussion purposes (don't worry about implemention details). We might have:: .. writer-is:: text-only :: +----------+ | SNMP | +----------+ | UDP | +----------+ | IP | +----------+ | Ethernet | +----------+ .. writer-is:: pdf .. figure:: protocol_stack.eps .. writer-is-not:: text-only pdf .. figure:: protocol_stack.png This could be an interface to the Filter transform (docutils.transforms.components.Filter). The ideas in `adaptable file extensions`_ above may also be applicable here. Here's an example of a directive that could produce multiple outputs (*both* raw troff pass-through *and* a GIF, for example) and allow the Writer to select. :: .. eqn:: .EQ delim %% .EN %sum from i=o to inf c sup i~=~lim from {m -> inf} sum from i=0 to m sup i% .EQ delim off .EN - _`body.qa` (directive a.k.a. "faq", "questions"): Questions & Answers. Implement as a generic two-column marked list? As a standalone (non-directive) construct? (Is the markup ambiguous?) Add support to parts.contents. New elements would be required. Perhaps:: Originally I thought of implementing a Q&A list with special syntax:: Q: What am I? A: You are a question-and-answer list. Q: What are you? A: I am the omniscient "we". Where each "Q" and "A" could also be numbered (e.g., "Q1"). However, a simple enumerated or bulleted list will do just fine for syntax. A directive could treat the list specially; e.g. the first paragraph could be treated as a question, the remainder as the answer (multiple answers could be represented by nested lists). Without special syntax, this directive becomes low priority. - _`body.example`: Examples; suggested by Simon Hefti. Semantics as per Docbook's "example"; admonition-style, numbered, reference, with a caption/title. - _`body.index`: Index targets. Were I writing a book with an index, I guess I'd need two different kinds of index targets: inline/implicit and out-of-line/explicit. For example:: In this `paragraph`:index:, several words are being `marked`:index: inline as implicit `index`:index: entries. .. index:: markup .. index:: syntax The explicit index directives above would refer to this paragraph. It might also make sense to allow multiple entries in an ``index`` directive: .. index:: markup syntax The words "paragraph", "marked", and "index" would become index entries pointing at the words in the first paragraph. The index entry words appear verbatim in the text. (Don't worry about the ugly ":index:" part; if indexing is the only/main application of interpreted text in your documents, it can be implicit and omitted.) The two directives provide manual indexing, where the index entry words ("markup" and "syntax") do not appear in the main text. We could combine the two directives into one:: .. index:: markup; syntax Semicolons instead of commas because commas could *be* part of the index target, like:: .. index:: van Rossum, Guido Another reason for index directives is because other inline markup wouldn't be possible within inline index targets. Sometimes index entries have multiple levels. Given:: .. index:: statement syntax: expression statements In a hypothetical index, combined with other entries, it might look like this:: statement syntax expression statements ..... 56 assignment ................ 57 simple statements ......... 58 compound statements ....... 60 Inline multi-level index targets could be done too. Perhaps something like:: When dealing with `expression statements `, we must remember ... The opposite sense could also be possible:: When dealing with `index entries <:multi-level>`, there are many permutations to consider. Also "see / see also" index entries. Given:: Here's a paragraph. .. index:: paragraph (The "index" directive above actually targets the *preceding* object.) The directive should produce something like this XML:: Here's a paragraph. This kind of content model would also allow true inline index-entries:: Here's a `paragraph`:index:. If the "index" role were the default for the application, it could be dropped:: Here's a `paragraph`. Both of these would result in this XML:: Here's a paragraph. - _`body.literal`: Literal block, possibly "formal" (see `object numbering and object references`_ above). Possible options: - "highlight" a range of lines - "number" or "line-numbers" - "styled" could indicate that the directive should check for style comments at the end of lines to indicate styling or markup. Specific derivatives (i.e., a "python-interactive" directive) could interpret style based on cues, like the ">>> " prompt and "input()"/"raw_input()" calls. See docutils-users 2003-03-03. - _`body.sidebar`: Add to the already implemented directive. Allow internal section structure, with adornment styles independent of the main document. - _`body.list-table`: See `List-Driven Tables `_. - _`colorize.python`: Colorize Python code. Fine for HTML output, but what about other formats? Revert to a literal block? Do we need some kind of "alternate" mechanism? Perhaps use a "pending" transform, which could switch its output based on the "format" in use. Use a factory function "transformFF()" which returns either "HTMLTransform()" instance or "GenericTransform" instance? If we take a Python-to-HTML pretty-printer and make it output a Docutils internal doctree (as per nodes.py) instead of HTML, then each output format's stylesheet (or equivalent) mechanism could take care of the rest. The pretty-printer code could turn this doctree fragment:: print 'This is Python code.' for i in range(10): print i into something like this ("" is end-tag shorthand):: print 'This is Python code.' for i in range(10): print i But I'm leaning toward adding a single new general-purpose element, "phrase", equivalent to HTML's . Here's the example rewritten using the generic "phrase":: print 'This is Python code.' for i in range(10): print i It's more verbose but more easily extensible and more appropriate for the case at hand. It allows us to edit style sheets to add support for new formats, not the Docutils code itself. Perhaps a single directive with a format parameter would be better:: .. colorize:: python print 'This is Python code.' for i in range(10): print i But directives can have synonyms for convenience. "format:: python" was suggested, but "format" seems too generic. - _`text.date`: Datestamp. For substitutions. The directive could be followed by a formatting string, using strftime codes. Default is "%Y-%m-%d" (ISO 8601 date), but time fields can also be used. - Combined with the "include" directive, implement canned macros? E.g.:: .. include:: Today's date is |date|. Where "macros" contains ``.. |date| date::``, among others. - _`text.time`: Timestamp. For substitutions. Shortcut for ``.. date:: %H:%M``. Date fields can also be used. - _`pysource.usage`: Extract a usage message from the program, either by running it at the command line with a ``--help`` option or through an exposed API. [Suggestion for Optik.] Interpreted Text ---------------- Interpreted text is entirely a reStructuredText markup construct, a way to get around built-in limitations of the medium. Some roles are intended to introduce new doctree elements, such as "title-reference". Others are merely convenience features, like "RFC". All supported interpreted text roles must already be known to the Parser when they are encountered in a document. Whether pre-defined in core/client code, or in the document, doesn't matter; the roles just need to have already been declared. Adding a new role often involves adding a new element to the DTD and may require extensive support, therefore such additions should be well thought-out. There should be a limited number of roles. The only place where no limit is placed on variation is at the start, at the Reader/Parser interface. Transforms are inserted by the Reader into the Transformer's queue, where non-standard elements are converted. Once past the Transformer, no variation from the standard Docutils doctree is possible. An example is the Python Source Reader, which will use interpreted text extensively. The default role will be "Python identifier", which will be further interpreted by namespace context into , , , , etc. elements (see pysource.dtd), which will be transformed into standard hyperlink references, which will be processed by the various Writers. No Writer will need to have any knowledge of the Python-Reader origin of these elements. * Alan Jaffray suggested (and I agree) that it would be sensible to: - have a directive and/or command-line option to specify a default role for interpreted text - allow the reST processor to take an argument for the default role (this will be subsumed by the above via the runtime settings mechanism) - issue a warning when processing documents with no default role which contain interpreted text with no explicitly specified role (there will always be a default role, so this won't happen) * Add explicit interpreted text roles for the rest of the implicit inline markup constructs: named-reference, anonymous-reference, footnote-reference, citation-reference, substitution-reference, target, uri-reference (& synonyms). * Add directives for each role as well? This would allow indirect nested markup:: This text contains |nested inline markup|. .. |nested inline markup| emphasis:: nested ``inline`` markup * Implement roles: - "acronym" and "abbreviation": Associate the full text with a short form. Jason Diamond's description: I want to translate ```reST`:acronym:`` into ``reST``. The value of the title attribute has to be defined out-of-band since you can't parameterize interpreted text. Right now I have them in a separate file but I'm experimenting with creating a directive that will use some form of reST syntax to let you define them. Should Docutils complain about undefined acronyms or abbreviations? What to do if there are multiple definitions? How to differentiate between CSS (Content Scrambling System) and CSS (Cascading Style Sheets) in a single document? David Priest responds, The short answer is: you don't. Anyone who did such a thing would be writing very poor documentation indeed. (Though I note that `somewhere else in the docs`__, there's mention of allowing replacement text to be associated with the abbreviation. That takes care of the duplicate acronyms/abbreviations problem, though a writer would be foolish to ever need it.) __ `inline parameter syntax`_ How to define the full text? Possibilities: 1. With a directive and a definition list? :: .. acronyms:: reST reStructuredText DPS Docstring Processing System Would this list remain in the document as a glossary, or would it simply build an internal lookup table? A "glossary" directive could be used to make the intention clear. Acronyms/abbreviations and glossaries could work together. Then again, a glossary could be formed by gathering individual definitions from around the document. 2. Some kind of `inline parameter syntax`_? :: `reST `:acronym: is `WYSIWYG `:acronym: plaintext markup. .. _inline parameter syntax: rst/alternatives.html#parameterized-interpreted-text 3. A combination of 1 & 2? The multiple definitions issue could be handled by establishing rules of priority. For example, directive-based lookup tables have highest priority, followed by the first inline definition. Multiple definitions in directive-based lookup tables would trigger warnings, similar to the rules of `implicit hyperlink targets`__. __ ../ref/rst/restructuredtext.html#implicit-hyperlink-targets - "annotation": The equivalent of the HTML "title" attribute. This is secondary information that may "pop up" when the pointer hovers over the main text. A corresponding directive would be required to associate annotations with the original text (by name, or positionally as in anonymous targets?). - "figure", "table", "listing", "chapter", "page", etc: See `object numbering and object references`_ above. - "term"?: Unfamiliar or specialized terminology. - "glossary-term": This would establish a link to a glossary. It would require an associated "glossary-entry" directive, whose contents could be a definition list:: .. glossary-entry:: term1 definition1 term2 definition2 This would allow entries to be defined anywhere in the document, and collected (via a "glossary" directive perhaps) at one point. Unimplemented Transforms ======================== * _`Footnote & Citation Gathering` Collect and move footnotes & citations to the end of a document. (Separate transforms.) * _`Hyperlink Target Gathering` It probably comes in two phases, because in a Python context we need to *resolve* them on a per-docstring basis [do we? --DG], but if the user is trying to do the callout form of presentation, they would then want to group them all at the end of the document. * _`Reference Merging` When merging two or more subdocuments (such as docstrings), conflicting references may need to be resolved. There may be: * duplicate reference and/or substitution names that need to be made unique; and/or * duplicate footnote numbers that need to be renumbered. Should this be done before or after reference-resolving transforms are applied? What about references from within one subdocument to inside another? * _`Document Splitting` If the processed document is written to multiple files (possibly in a directory tree), it will need to be split up. Internal references will have to be adjusted. (HTML only? Initially, yes. Eventually, anything should be splittable.) Idea: insert a "destination" attribute into the root element of each split-out document, containing the path/filename. The Output object or Writer will recognize this attribute and split out the files accordingly. Must allow for common headers & footers, prev/next, breadcrumbs, etc. * _`Navigation` If a document is split up, each segment will need navigation links: parent, children (small TOC), previous (preorder), next (preorder). Part of `Document Splitting`_? * _`List of System Messages` The ``system_message`` elements are inserted into the document tree, adjacent to the problems themselves where possible. Some (those generated post-parse) are kept until later, in ``document.messages``, and added as a special final section, "Docutils System Messages". Docutils could be made to generate hyperlinks to all known system_messages and add them to the document, perhaps to the end of the "Docutils System Messages" section. Fred L. Drake, Jr. wrote: I'd like to propose that both parse- and transformation-time messages are included in the "Docutils System Messages" section. If there are no objections, I can make the change. The advantage of the current way of doing things is that parse-time system messages don't require a transform; they're already in the document. This is valuable for testing (unit tests, tools/quicktest.py). So if we do decide to make a change, I think the insertion of parse-time system messages ought to remain as-is and the Messages transform ought to move all parse-time system messages (remove from their originally inserted positions, insert in System Messages section). * _`Index Generation` HTML Writer =========== * Construct a _`templating system`, as in ht2html/yaptu, using directives and substitutions for dynamic stuff? Or a specialized writer to generate .ht & links.h files for ht2html? This is an overly generic, ambitious, and unnecessary idea. The two systems, ht2html & yaptu, represent different approaches to templating. Ht2html stiches bits of HTML together to make unified web pages. Yaptu is for generic programmable text replacement. Templating should simply be left to other systems. Users can choose a templating system to apply to their reStructuredText documents as best serves their interests. * Add more support for elements, especially for navigation bars. * Make the admonitions more distinctive and varied. * Make the "class" attributes optional? Implies no stylesheet? * Base list compaction on the spacing of source list? Would require parser support. (Idea: fantasai, 16 Dec 2002, doc-sig.) * Add a tool tip ("title" attribute?) to footnote back-links identifying them as such. Text in Docutils language module. * Add an option to restrict the document title to only, and not include it in the document body. Subtitle? * Insert a comment at the top of HTML files that describes how to deal with the broken servers w.r.t. encodings? Perhaps something like this: <!-- If your browser is showing gibberish, the server may be broken. Try manually setting the character coding to "UTF-8". In Mozilla/Firefox, do ... In Internet Explorer, do ... For details, see <URL>. --> Front-End Tools =============== * What about if we don't know which Reader and/or Writer we are going to use? If the Reader/Writer is specified on the command-line? (Will this ever happen?) Perhaps have different types of front ends: a) _`Fully qualified`: Reader and Writer are hard-coded into the front end (e.g. ``pep2html [options]``, ``pysource2pdf [options]``). b) _`Partially qualified`: Reader is hard-coded, and the Writer is specified a sub-command (e.g. ``pep2 html [options]``, ``pysource2 pdf [options]``). The Writer is known before option processing happens, allowing the OptionParser to be built dynamically. Alternatively, the Writer could be hard-coded and the Reader specified as a sub-command (e.g. ``htmlfrom pep [options]``). c) _`Unqualified`: Reader and Writer are specified as subcommands (e.g. ``publish pep html [options]``, ``publish pysource pdf [options]``). A single front end would be sufficient, but probably only useful for testing purposes. d) _`Dynamic`: Reader and/or Writer are specified by options, with defaults if unspecified (e.g. ``publish --writer pdf [options]``). Is this possible? The option parser would have to be told about new options it needs to handle, on the fly. Component-specific options would have to be specified *after* the component-specifying option. Allow common options before subcommands, as in CVS? Or group all options together? In the case of the `fully qualified`_ front ends, all the options will have to be grouped together anyway, so there's no advantage (we can't use it to avoid conflicts) to splitting common and component-specific options apart. * Parameterize help text & defaults somehow? Perhaps a callback? Or initialize ``settings_spec`` in ``__init__`` or ``init_options``? * Disable common options that don't apply? * Implement the "sectnum" directive as a command-line option also? * @@@ Come up with better names for the most-used tools, and install them as scripts. * Create a single dynamic_ or unqualified_ front end that can be installed? .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:


===================
 Docutils Web Site
===================
:Author: David Goodger; open to all Docutils developers
:Contact: goodger@python.org
:Date: $Date: 2004/06/03 14:10:05 $
:Revision: $Revision: 1.1 $
:Copyright: This document has been placed in the public domain.

The Docutils web site, <http://docutils.sourceforge.net/>, is
maintained automatically by the docutils-update script, run as an
hourly cron job (by user "goodger").  The script will process any .txt
file which is newer than the corresponding .html file in the project's
web directory on SourceForge (``/home/groups/d/do/docutils/htdocs/``).
For a new .txt file, just SSH to SourceForge and ::

    touch filename.html
    chmod g+w filename.html
    sleep 1
    touch filename.txt

The script will take care of the rest within an hour.  Thereafter
whenever the .txt file is modified (checked in to CVS), the .html will
be regenerated automatically.

After adding directories to CVS, allow the script to run once to
create the directories in the filesystem before preparing for HTML
processing as described above.

The docutils-update__ script is in CVS as
``sandbox/davidg/infrastructure/docutils-update``.

__ http://docutils.sf.net/sandbox/davidg/infrastructure/docutils-update

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


Document

================================================== A Record of reStructuredText Syntax Alternatives ================================================== :Author: David Goodger :Contact: goodger@users.sourceforge.net :Revision: $Revision: 1.33 $ :Date: $Date: 2004/06/14 19:48:38 $ :Copyright: This document has been placed in the public domain. The following are ideas, alternatives, and justifications that were considered for reStructuredText syntax, which did not originate with Setext_ or StructuredText_. For an analysis of constructs which *did* originate with StructuredText or Setext, please see `Problems With StructuredText`_. See the `reStructuredText Markup Specification`_ for full details of the established syntax. .. _Setext: http://docutils.sourceforge.net/mirror/setext.html .. _StructuredText: http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage .. _Problems with StructuredText: problems.html .. _reStructuredText Markup Specification: ../../ref/rst/restructuredtext.html .. contents:: ------------- Implemented ------------- Field Lists =========== Prior to the syntax for field lists being finalized, several alternatives were proposed. 1. Unadorned RFC822_ everywhere:: Author: Me Version: 1 Advantages: clean, precedent (RFC822-compliant). Disadvantage: ambiguous (these paragraphs are a prime example). Conclusion: rejected. 2. Special case: use unadorned RFC822_ for the very first or very last text block of a document:: """ Author: Me Version: 1 The rest of the document... """ Advantages: clean, precedent (RFC822-compliant). Disadvantages: special case, flat (unnested) field lists only, still ambiguous:: """ Usage: cmdname [options] arg1 arg2 ... We obviously *don't* want the like above to be interpreted as a field list item. Or do we? """ Conclusion: rejected for the general case, accepted for specific contexts (PEPs, email). 3. Use a directive:: .. fields:: Author: Me Version: 1 Advantages: explicit and unambiguous, RFC822-compliant. Disadvantage: cumbersome. Conclusion: rejected for the general case (but such a directive could certainly be written). 4. Use Javadoc-style:: @Author: Me @Version: 1 @param a: integer Advantages: unambiguous, precedent, flexible. Disadvantages: non-intuitive, ugly, not RFC822-compliant. Conclusion: rejected. 5. Use leading colons:: :Author: Me :Version: 1 Advantages: unambiguous, obvious (*almost* RFC822-compliant), flexible, perhaps even elegant. Disadvantages: no precedent, not quite RFC822-compliant. Conclusion: accepted! 6. Use double colons:: Author:: Me Version:: 1 Advantages: unambiguous, obvious? (*almost* RFC822-compliant), flexible, similar to syntax already used for literal blocks and directives. Disadvantages: no precedent, not quite RFC822-compliant, similar to syntax already used for literal blocks and directives. Conclusion: rejected because of the syntax similarity & conflicts. Why is RFC822 compliance important? It's a universal Internet standard, and super obvious. Also, I'd like to support the PEP format (ulterior motive: get PEPs to use reStructuredText as their standard). But it *would* be easy to get used to an alternative (easy even to convert PEPs; probably harder to convert python-deviants ;-). Unfortunately, without well-defined context (such as in email headers: RFC822 only applies before any blank lines), the RFC822 format is ambiguous. It is very common in ordinary text. To implement field lists unambiguously, we need explicit syntax. The following question was posed in a footnote: Should "bibliographic field lists" be defined at the parser level, or at the DPS transformation level? In other words, are they reStructuredText-specific, or would they also be applicable to another (many/every other?) syntax? The answer is that bibliographic fields are a reStructuredText-specific markup convention. Other syntaxes may implement the bibliographic elements explicitly. For example, there would be no need for such a transformation for an XML-based markup syntax. .. _RFC822: http://www.rfc-editor.org/rfc/rfc822.txt Interpreted Text "Roles" ======================== The original purpose of interpreted text was as a mechanism for descriptive markup, to describe the nature or role of a word or phrase. For example, in XML we could say "len" to mark up "len" as a function. It is envisaged that within Python docstrings (inline documentation in Python module source files, the primary market for reStructuredText) the role of a piece of interpreted text can be inferred implicitly from the context of the docstring within the program source. For other applications, however, the role may have to be indicated explicitly. Interpreted text is enclosed in single backquotes (`). 1. Initially, it was proposed that an explicit role could be indicated as a word or phrase within the enclosing backquotes: - As a prefix, separated by a colon and whitespace:: `role: interpreted text` - As a suffix, separated by whitespace and a colon:: `interpreted text :role` There are problems with the initial approach: - There could be ambiguity with interpreted text containing colons. For example, an index entry of "Mission: Impossible" would require a backslash-escaped colon. - The explicit role is descriptive markup, not content, and will not be visible in the processed output. Putting it inside the backquotes doesn't feel right; the *role* isn't being quoted. 2. Tony Ibbs suggested that the role be placed outside the backquotes:: role:`prefix` or `suffix`:role This removes the embedded-colons ambiguity, but limits the role identifier to be a single word (whitespace would be illegal). Since roles are not meant to be visible after processing, the lack of whitespace support is not important. The suggested syntax remains ambiguous with respect to ratios and some writing styles. For example, suppose there is a "signal" identifier, and we write:: ...calculate the `signal`:noise ratio. "noise" looks like a role. 3. As an improvement on #2, we can bracket the role with colons:: :role:`prefix` or `suffix`:role: This syntax is similar to that of field lists, which is fine since both are doing similar things: describing. This is the syntax chosen for reStructuredText. 4. Another alternative is two colons instead of one:: role::`prefix` or `suffix`::role But this is used for analogies ("A:B::C:D": "A is to B as C is to D"). Both alternative #2 and #4 lack delimiters on both sides of the role, making it difficult to parse (by the reader). 5. Some kind of bracketing could be used: - Parentheses:: (role)`prefix` or `suffix`(role) - Braces:: {role}`prefix` or `suffix`{role} - Square brackets:: [role]`prefix` or `suffix`[role] - Angle brackets:: `prefix` or `suffix` (The overlap of \*ML tags with angle brackets would be too confusing and precludes their use.) Syntax #3 was chosen for reStructuredText. Comments ======== A problem with comments (actually, with all indented constructs) is that they cannot be followed by an indented block -- a block quote -- without swallowing it up. I thought that perhaps comments should be one-liners only. But would this mean that footnotes, hyperlink targets, and directives must then also be one-liners? Not a good solution. Tony Ibbs suggested a "comment" directive. I added that we could limit a comment to a single text block, and that a "multi-block comment" could use "comment-start" and "comment-end" directives. This would remove the indentation incompatibility. A "comment" directive automatically suggests "footnote" and (hyperlink) "target" directives as well. This could go on forever! Bad choice. Garth Kidd suggested that an "empty comment", a ".." explicit markup start with nothing on the first line (except possibly whitespace) and a blank line immediately following, could serve as an "unindent". An empty comment does **not** swallow up indented blocks following it, so block quotes are safe. "A tiny but practical wart." Accepted. Anonymous Hyperlinks ==================== Alan Jaffray came up with this idea, along with the following syntax:: Search the `Python DOC-SIG mailing list archives`{}_. .. _: http://mail.python.org/pipermail/doc-sig/ The idea is sound and useful. I suggested a "double underscore" syntax:: Search the `Python DOC-SIG mailing list archives`__. .. __: http://mail.python.org/pipermail/doc-sig/ But perhaps single underscores are okay? The syntax looks better, but the hyperlink itself doesn't explicitly say "anonymous":: Search the `Python DOC-SIG mailing list archives`_. .. _: http://mail.python.org/pipermail/doc-sig/ Mixing anonymous and named hyperlinks becomes confusing. The order of targets is not significant for named hyperlinks, but it is for anonymous hyperlinks:: Hyperlinks: anonymous_, named_, and another anonymous_. .. _named: named .. _: anonymous1 .. _: anonymous2 Without the extra syntax of double underscores, determining which hyperlink references are anonymous may be difficult. We'd have to check which references don't have corresponding targets, and match those up with anonymous targets. Keeping to a simple consistent ordering (as with auto-numbered footnotes) seems simplest. reStructuredText will use the explicit double-underscore syntax for anonymous hyperlinks. An alternative (see `Reworking Explicit Markup (Round 1)`_ below) for the somewhat awkward ".. __:" syntax is "__":: An anonymous__ reference. __ http://anonymous Reworking Explicit Markup (Round 1) =================================== Alan Jaffray came up with the idea of `anonymous hyperlinks`_, added to reStructuredText. Subsequently it was asserted that hyperlinks (especially anonymous hyperlinks) would play an increasingly important role in reStructuredText documents, and therefore they require a simpler and more concise syntax. This prompted a review of the current and proposed explicit markup syntaxes with regards to improving usability. 1. Original syntax:: .. _blah: internal hyperlink target .. _blah: http://somewhere external hyperlink target .. _blah: blahblah_ indirect hyperlink target .. __: anonymous internal target .. __: http://somewhere anonymous external target .. __: blahblah_ anonymous indirect target .. [blah] http://somewhere footnote .. blah:: http://somewhere directive .. blah: http://somewhere comment .. Note:: The comment text was intentionally made to look like a hyperlink target. Origins: * Except for the colon (a delimiter necessary to allow for phrase-links), hyperlink target ``.. _blah:`` comes from Setext. * Comment syntax from Setext. * Footnote syntax from StructuredText ("named links"). * Directives and anonymous hyperlinks original to reStructuredText. Advantages: + Consistent explicit markup indicator: "..". + Consistent hyperlink syntax: ".. _" & ":". Disadvantages: - Anonymous target markup is awkward: ".. __:". - The explicit markup indicator ("..") is excessively overloaded? - Comment text is limited (can't look like a footnote, hyperlink, or directive). But this is probably not important. 2. Alan Jaffray's proposed syntax #1:: __ _blah internal hyperlink target __ blah: http://somewhere external hyperlink target __ blah: blahblah_ indirect hyperlink target __ anonymous internal target __ http://somewhere anonymous external target __ blahblah_ anonymous indirect target __ [blah] http://somewhere footnote .. blah:: http://somewhere directive .. blah: http://somewhere comment The hyperlink-connoted underscores have become first-level syntax. Advantages: + Anonymous targets are simpler. + All hyperlink targets are one character shorter. Disadvantages: - Inconsistent internal hyperlink targets. Unlike all other named hyperlink targets, there's no colon. There's an extra leading underscore, but we can't drop it because without it, "blah" looks like a relative URI. Unless we restore the colon:: __ blah: internal hyperlink target - Obtrusive markup? 3. Alan Jaffray's proposed syntax #2:: .. _blah internal hyperlink target .. blah: http://somewhere external hyperlink target .. blah: blahblah_ indirect hyperlink target .. anonymous internal target .. http://somewhere anonymous external target .. blahblah_ anonymous indirect target .. [blah] http://somewhere footnote !! blah: http://somewhere directive ## blah: http://somewhere comment Leading underscores have been (almost) replaced by "..", while comments and directives have gained their own syntax. Advantages: + Anonymous hyperlinks are simpler. + Unique syntax for comments. Connotation of "comment" from some programming languages (including our favorite). + Unique syntax for directives. Connotation of "action!". Disadvantages: - Inconsistent internal hyperlink targets. Again, unlike all other named hyperlink targets, there's no colon. There's a leading underscore, matching the trailing underscores of references, which no other hyperlink targets have. We can't drop that one leading underscore though: without it, "blah" looks like a relative URI. Again, unless we restore the colon:: .. blah: internal hyperlink target - All (except for internal) hyperlink targets lack their leading underscores, losing the "hyperlink" connotation. - Obtrusive syntax for comments. Alternatives:: ;; blah: http://somewhere (also comment syntax in Lisp & others) ,, blah: http://somewhere ("comma comma": sounds like "comment"!) - Iffy syntax for directives. Alternatives? 4. Tony Ibbs' proposed syntax:: .. _blah: internal hyperlink target .. _blah: http://somewhere external hyperlink target .. _blah: blahblah_ indirect hyperlink target .. anonymous internal target .. http://somewhere anonymous external target .. blahblah_ anonymous indirect target .. [blah] http://somewhere footnote .. blah:: http://somewhere directive .. blah: http://somewhere comment This is the same as the current syntax, except for anonymous targets which drop their "__: ". Advantage: + Anonymous targets are simpler. Disadvantages: - Anonymous targets lack their leading underscores, losing the "hyperlink" connotation. - Anonymous targets are almost indistinguishable from comments. (Better to know "up front".) 5. David Goodger's proposed syntax: Perhaps going back to one of Alan's earlier suggestions might be the best solution. How about simply adding "__ " as a synonym for ".. __: " in the original syntax? These would become equivalent:: .. __: anonymous internal target .. __: http://somewhere anonymous external target .. __: blahblah_ anonymous indirect target __ anonymous internal target __ http://somewhere anonymous external target __ blahblah_ anonymous indirect target Alternative 5 has been adopted. Backquotes in Phrase-Links ========================== [From a 2001-06-05 Doc-SIG post in reply to questions from Doug Hellmann.] The first draft of the spec, posted to the Doc-SIG in November 2000, used square brackets for phrase-links. I changed my mind because: 1. In the first draft, I had already decided on single-backquotes for inline literal text. 2. However, I wanted to minimize the necessity for backslash escapes, for example when quoting Python repr-equivalent syntax that uses backquotes. 3. The processing of identifiers (function/method/attribute/module etc. names) into hyperlinks is a useful feature. PyDoc recognizes identifiers heuristically, but it doesn't take much imagination to come up with counter-examples where PyDoc's heuristics would result in embarassing failure. I wanted to do it deterministically, and that called for syntax. I called this construct "interpreted text". 4. Leveraging off the ``*emphasis*/**strong**`` syntax, lead to the idea of using double-backquotes as syntax. 5. I worked out some rules for inline markup recognition. 6. In combination with #5, double backquotes lent themselves to inline literals, neatly satisfying #2, minimizing backslash escapes. In fact, the spec says that no interpretation of any kind is done within double-backquote inline literal text; backslashes do *no* escaping within literal text. 7. Single backquotes are then freed up for interpreted text. 8. I already had square brackets required for footnote references. 9. Since interpreted text will typically turn into hyperlinks, it was a natural fit to use backquotes as the phrase-quoting syntax for trailing-underscore hyperlinks. The original inspiration for the trailing underscore hyperlink syntax was Setext. But for phrases Setext used a very cumbersome ``underscores_between_words_like_this_`` syntax. The underscores can be viewed as if they were right-pointing arrows: ``-->``. So ``hyperlink_`` points away from the reference, and ``.. _hyperlink:`` points toward the target. Substitution Mechanism ====================== Substitutions arose out of a Doc-SIG thread begun on 2001-10-28 by Alan Jaffray, "reStructuredText inline markup". It reminded me of a missing piece of the reStructuredText puzzle, first referred to in my contribution to "Documentation markup & processing / PEPs" (Doc-SIG 2001-06-21). Substitutions allow the power and flexibility of directives to be shared by inline text. They are a way to allow arbitrarily complex inline objects, while keeping the details out of the flow of text. They are the equivalent of SGML/XML's named entities. For example, an inline image (using reference syntax alternative 4d (vertical bars) and definition alternative 3, the alternatives chosen for inclusion in the spec):: The |biohazard| symbol must be used on containers used to dispose of medical waste. .. |biohazard| image:: biohazard.png [height=20 width=20] The ``|biohazard|`` substitution reference will be replaced in-line by whatever the ``.. |biohazard|`` substitution definition generates (in this case, an image). A substitution definition contains the substitution text bracketed with vertical bars, followed by a an embedded inline-compatible directive, such as "image". A transform is required to complete the substitution. Syntax alternatives for the reference: 1. Use the existing interpreted text syntax, with a predefined role such as "sub":: The `biohazard`:sub: symbol... Advantages: existing syntax, explicit. Disadvantages: verbose, obtrusive. 2. Use a variant of the interpreted text syntax, with a new suffix akin to the underscore in phrase-link references:: (a) `name`@ (b) `name`# (c) `name`& (d) `name`/ (e) `name`< (f) `name`:: (g) `name`: Due to incompatibility with other constructs and ordinary text usage, (f) and (g) are not possible. 3. Use interpreted text syntax with a fixed internal format:: (a) `:name:` (b) `name:` (c) `name::` (d) `::name::` (e) `%name%` (f) `#name#` (g) `/name/` (h) `&name&` (i) `|name|` (j) `[name]` (k) `` (l) `&name;` (m) `'name'` To avoid ML confusion (k) and (l) are definitely out. Square brackets (j) won't work in the target (the substitution definition would be indistinguishable from a footnote). The ```/name/``` syntax (g) is reminiscent of "s/find/sub" substitution syntax in ed-like languages. However, it may have a misleading association with regexps, and looks like an absolute POSIX path. (i) is visually equivalent and lacking the connotations. A disadvantage of all of these is that they limit interpreted text, albeit only slightly. 4. Use specialized syntax, something new:: (a) #name# (b) @name@ (c) /name/ (d) |name| (e) <> (f) //name// (g) ||name|| (h) ^name^ (i) [[name]] (j) ~name~ (k) !name! (l) =name= (m) ?name? (n) >name< "#" (a) and "@" (b) are obtrusive. "/" (c) without backquotes looks just like a POSIX path; it is likely for such usage to appear in text. "|" (d) and "^" (h) are feasible. 5. Redefine the trailing underscore syntax. See definition syntax alternative 4, below. Syntax alternatives for the definition: 1. Use the existing directive syntax, with a predefined directive such as "sub". It contains a further embedded directive resolving to an inline-compatible object:: .. sub:: biohazard .. image:: biohazard.png [height=20 width=20] .. sub:: parrot That bird wouldn't *voom* if you put 10,000,000 volts through it! The advantages and disadvantages are the same as in inline alternative 1. 2. Use syntax as in #1, but with an embedded directivecompressed:: .. sub:: biohazard image:: biohazard.png [height=20 width=20] This is a bit better than alternative 1, but still too much. 3. Use a variant of directive syntax, incorporating the substitution text, obviating the need for a special "sub" directive name. If we assume reference alternative 4d (vertical bars), the matching definition would look like this:: .. |biohazard| image:: biohazard.png [height=20 width=20] 4. (Suggested by Alan Jaffray on Doc-SIG from 2001-11-06.) Instead of adding new syntax, redefine the trailing underscore syntax to mean "substitution reference" instead of "hyperlink reference". Alan's example:: I had lunch with Jonathan_ today. We talked about Zope_. .. _Jonathan: lj [user=jhl] .. _Zope: http://www.zope.org/ A problem with the proposed syntax is that URIs which look like simple reference names (alphanum plus ".", "-", "_") would be indistinguishable from substitution directive names. A more consistent syntax would be:: I had lunch with Jonathan_ today. We talked about Zope_. .. _Jonathan: lj:: user=jhl .. _Zope: http://www.zope.org/ (``::`` after ``.. _Jonathan: lj``.) The "Zope" target is a simple external hyperlink, but the "Jonathan" target contains a directive. Alan proposed is that the reference text be replaced by whatever the referenced directive (the "directive target") produces. A directive reference becomes a hyperlink reference if the contents of the directive target resolve to a hyperlink. If the directive target resolves to an icon, the reference is replaced by an inline icon. If the directive target resolves to a hyperlink, the directive reference becomes a hyperlink reference. This seems too indirect and complicated for easy comprehension. The reference in the text will sometimes become a link, sometimes not. Sometimes the reference text will remain, sometimes not. We don't know *at the reference*:: This is a `hyperlink reference`_; its text will remain. This is an `inline icon`_; its text will disappear. That's a problem. The syntax that has been incorporated into the spec and parser is reference alternative 4d with definition alternative 3:: The |biohazard| symbol... .. |biohazard| image:: biohazard.png [height=20 width=20] We can also combine substitution references with hyperlink references, by appending a "_" (named hyperlink reference) or "__" (anonymous hyperlink reference) suffix to the substitution reference. This allows us to click on an image-link:: The |biohazard|_ symbol... .. |biohazard| image:: biohazard.png [height=20 width=20] .. _biohazard: http://www.cdc.gov/ There have been several suggestions for the naming of these constructs, originally called "substitution references" and "substitutions". 1. Candidate names for the reference construct: (a) substitution reference (b) tagging reference (c) inline directive reference (d) directive reference (e) indirect inline directive reference (f) inline directive placeholder (g) inline directive insertion reference (h) directive insertion reference (i) insertion reference (j) directive macro reference (k) macro reference (l) substitution directive reference 2. Candidate names for the definition construct: (a) substitution (b) substitution directive (c) tag (d) tagged directive (e) directive target (f) inline directive (g) inline directive definition (h) referenced directive (i) indirect directive (j) indirect directive definition (k) directive definition (l) indirect inline directive (m) named directive definition (n) inline directive insertion definition (o) directive insertion definition (p) insertion definition (q) insertion directive (r) substitution definition (s) directive macro definition (t) macro definition (u) substitution directive definition (v) substitution definition "Inline directive reference" (1c) seems to be an appropriate term at first, but the term "inline" is redundant in the case of the reference. Its counterpart "inline directive definition" (2g) is awkward, because the directive definition itself is not inline. "Directive reference" (1d) and "directive definition" (2k) are too vague. "Directive definition" could be used to refer to any directive, not just those used for inline substitutions. One meaning of the term "macro" (1k, 2s, 2t) is too programming-language-specific. Also, macros are typically simple text substitution mechanisms: the text is substituted first and evaluated later. reStructuredText substitution definitions are evaluated in place at parse time and substituted afterwards. "Insertion" (1h, 1i, 2n-2q) is almost right, but it implies that something new is getting added rather than one construct being replaced by another. Which brings us back to "substitution". The overall best names are "substitution reference" (1a) and "substitution definition" (2v). A long way to go to add one word! Inline External Targets ======================= Currently reStructuredText has two hyperlink syntax variations: * Named hyperlinks:: This is a named reference_ of one word ("reference"). Here is a `phrase reference`_. Phrase references may even cross `line boundaries`_. .. _reference: http://www.example.org/reference/ .. _phrase reference: http://www.example.org/phrase_reference/ .. _line boundaries: http://www.example.org/line_boundaries/ + Advantages: - The plaintext is readable. - Each target may be reused multiple times (e.g., just write ``"reference_"`` again). - No syncronized ordering of references and targets is necessary. + Disadvantages: - The reference text must be repeated as target names; could lead to mistakes. - The target URLs may be located far from the references, and hard to find in the plaintext. * Anonymous hyperlinks (in current reStructuredText):: This is an anonymous reference__. Here is an anonymous `phrase reference`__. Phrase references may even cross `line boundaries`__. __ http://www.example.org/reference/ __ http://www.example.org/phrase_reference/ __ http://www.example.org/line_boundaries/ + Advantages: - The plaintext is readable. - The reference text does not have to be repeated. + Disadvantages: - References and targets must be kept in sync. - Targets cannot be reused. - The target URLs may be located far from the references. For comparison and historical background, StructuredText also has two syntaxes for hyperlinks: * First, ``"reference text":URL``:: This is a "reference":http://www.example.org/reference/ of one word ("reference"). Here is a "phrase reference":http://www.example.org/phrase_reference/. * Second, ``"reference text", http://example.com/absolute_URL``:: This is a "reference", http://www.example.org/reference/ of one word ("reference"). Here is a "phrase reference", http://www.example.org/phrase_reference/. Both syntaxes share advantages and disadvantages: + Advantages: - The target is specified immediately adjacent to the reference. + Disadvantages: - Poor plaintext readability. - Targets cannot be reused. - Both syntaxes use double quotes, common in ordinary text. - In the first syntax, the URL and the last word are stuck together, exacerbating the line wrap problem. - The second syntax is too magical; text could easily be written that way by accident (although only absolute URLs are recognized here, perhaps because of the potential for ambiguity). A new type of "inline external hyperlink" has been proposed. 1. On 2002-06-28, Simon Budig proposed__ a new syntax for reStructuredText hyperlinks:: This is a reference_(http://www.example.org/reference/) of one word ("reference"). Here is a `phrase reference`_(http://www.example.org/phrase_reference/). Are these examples, (single-underscore), named? If so, `anonymous references`__(http://www.example.org/anonymous/) using two underscores would probably be preferable. __ http://mail.python.org/pipermail/doc-sig/2002-June/002648.html The syntax, advantages, and disadvantages are similar to those of StructuredText. + Advantages: - The target is specified immediately adjacent to the reference. + Disadvantages: - Poor plaintext readability. - Targets cannot be reused (unless named, but the semantics are unclear). + Problems: - The ``"`ref`_(URL)"`` syntax forces the last word of the reference text to be joined to the URL, making a potentially very long word that can't be wrapped (URLs can be very long). The reference and the URL should be separate. This is a symptom of the following point: - The syntax produces a single compound construct made up of two equally important parts, *with syntax in the middle*, *between* the reference and the target. This is unprecedented in reStructuredText. - The "inline hyperlink" text is *not* a named reference (there's no lookup by name), so it shouldn't look like one. - According to the IETF standards RFC 2396 and RFC 2732, parentheses are legal URI characters and curly braces are legal email characters, making their use prohibitively difficult. - The named/anonymous semantics are unclear. 2. After an analysis__ of the syntax of (1) above, we came up with the following compromise syntax:: This is an anonymous reference__ __ of one word ("reference"). Here is a `phrase reference`__ __. `Named references`_ _ use single underscores. __ http://mail.python.org/pipermail/doc-sig/2002-July/002670.html The syntax builds on that of the existing "inline internal targets": ``an _`inline internal target`.`` + Advantages: - The target is specified immediately adjacent to the reference, improving maintainability: - References and targets are easily kept in sync. - The reference text does not have to be repeated. - The construct is executed in two parts: references identical to existing references, and targets that are new but not too big a stretch from current syntax. - There's overwhelming precedent for quoting URLs with angle brackets [#]_. + Disadvantages: - Poor plaintext readability. - Lots of "line noise". - Targets cannot be reused (unless named; see below). To alleviate the readability issue slightly, we could allow the target to appear later, such as after the end of the sentence:: This is a named reference__ of one word ("reference"). __ Here is a `phrase reference`__. __ Problem: this could only work for one reference at a time (reference/target pairs must be proximate [refA trgA refB trgB], not interleaved [refA refB trgA trgB] or nested [refA refB trgB trgA]). This variation is too problematic; references and inline external targets will have to be kept imediately adjacent (see (3) below). The ``"reference__ __"`` syntax is actually for "anonymous inline external targets", emphasized by the double underscores. It follows that single trailing and leading underscores would lead to *implicitly named* inline external targets. This would allow the reuse of targets by name. So after ``"reference_ _"``, another ``"reference_"`` would point to the same target. .. [#] From RFC 2396 (URI syntax): The angle-bracket "<" and ">" and double-quote (") characters are excluded [from URIs] because they are often used as the delimiters around URI in text documents and protocol fields. Using <> angle brackets around each URI is especially recommended as a delimiting style for URI that contain whitespace. From RFC 822 (email headers): Angle brackets ("<" and ">") are generally used to indicate the presence of a one machine-usable reference (e.g., delimiting mailboxes), possibly including source-routing to the machine. 3. If it is best for references and inline external targets to be immediately adjacent, then they might as well be integrated. Here's an alternative syntax embedding the target URL in the reference:: This is an anonymous `reference `__ of one word ("reference"). Here is a `phrase reference `__. Advantages and disadvantages are similar to those in (2). Readability is still an issue, but the syntax is a bit less heavyweight (reduced line noise). Backquotes are required, even for one-word references; the target URL is included within the reference text, forcing a phrase context. We'll call this variant "embedded URIs". Problem: how to refer to a title like "HTML Anchors: " (which ends with an HTML/SGML/XML tag)? We could either require more syntax on the target (like ``"`reference text __`__"``), or require the odd conflicting title to be escaped (like ``"`HTML Anchors: \`__"``). The latter seems preferable, and not too onerous. Similarly to (2) above, a single trailing underscore would convert the reference & inline external target from anonymous to implicitly named, allowing reuse of targets by name. I think this is the least objectionable of the syntax alternatives. Other syntax variations have been proposed (by Brett Cannon and Benja Fallenstein):: `phrase reference`->http://www.example.com `phrase reference`@http://www.example.com `phrase reference`__ ->http://www.example.com `phrase reference` [-> http://www.example.com] `phrase reference`__ [-> http://www.example.com] `phrase reference` _ None of these variations are clearly superior to #3 above. Some have problems that exclude their use. With any kind of inline external target syntax it comes down to the conflict between maintainability and plaintext readability. I don't see a major problem with reStructuredText's maintainability, and I don't want to sacrifice plaintext readability to "improve" it. The proponents of inline external targets want them for easily maintainable web pages. The arguments go something like this: - Named hyperlinks are difficult to maintain because the reference text is duplicated as the target name. To which I said, "So use anonymous hyperlinks." - Anonymous hyperlinks are difficult to maintain becuase the references and targets have to be kept in sync. "So keep the targets close to the references, grouped after each paragraph. Maintenance is trivial." - But targets grouped after paragraphs break the flow of text. "Surely less than URLs embedded in the text! And if the intent is to produce web pages, not readable plaintext, then who cares about the flow of text?" Many participants have voiced their objections to the proposed syntax: Garth Kidd: "I strongly prefer the current way of doing it. Inline is spectactularly messy, IMHO." Tony Ibbs: "I vehemently agree... that the inline alternatives being suggested look messy - there are/were good reasons they've been taken out... I don't believe I would gain from the new syntaxes." Paul Moore: "I agree as well. The proposed syntax is far too punctuation-heavy, and any of the alternatives discussed are ambiguous or too subtle." Others have voiced their support: fantasai: "I agree with Simon. In many cases, though certainly not in all, I find parenthesizing the url in plain text flows better than relegating it to a footnote." Ken Manheimer: "I'd like to weigh in requesting some kind of easy, direct inline reference link." (Interesting that those *against* the proposal have been using reStructuredText for a while, and those *for* the proposal are either new to the list ["fantasai", background unknown] or longtime StructuredText users [Ken Manheimer].) I was initially ambivalent/against the proposed "inline external targets". I value reStructuredText's readability very highly, and although the proposed syntax offers convenience, I don't know if the convenience is worth the cost in ugliness. Does the proposed syntax compromise readability too much, or should the choice be left up to the author? Perhaps if the syntax is *allowed* but its use strongly *discouraged*, for aesthetic/readability reasons? After a great deal of thought and much input from users, I've decided that there are reasonable use cases for this construct. The documentation should strongly caution against its use in most situations, recommending independent block-level targets instead. Syntax #3 above ("embedded URIs") will be used. Doctree Representation of Transitions ===================================== (Although not reStructuredText-specific, this section fits best in this document.) Having added the "horizontal rule" construct to the `reStructuredText Markup Specification`_, a decision had to be made as to how to reflect the construct in the implementation of the document tree. Given this source:: Document ======== Paragraph 1 -------- Paragraph 2 The horizontal rule indicates a "transition" (in prose terms) or the start of a new "division". Before implementation, the parsed document tree would be::
Document <paragraph> Paragraph 1 -------- <--- error here <paragraph> Paragraph 2 There are several possibilities for the implementation: 1. Implement horizontal rules as "divisions" or segments. A "division" is a title-less, non-hierarchical section. The first try at an implementation looked like this:: <document> <section name="document"> <title> Document <paragraph> Paragraph 1 <division> <paragraph> Paragraph 2 But the two paragraphs are really at the same level; they shouldn't appear to be at different levels. There's really an invisible "first division". The horizontal rule splits the document body into two segments, which should be treated uniformly. 2. Treating "divisions" uniformly brings us to the second possibility:: <document> <section name="document"> <title> Document <division> <paragraph> Paragraph 1 <division> <paragraph> Paragraph 2 With this change, documents and sections will directly contain divisions and sections, but not body elements. Only divisions will directly contain body elements. Even without a horizontal rule anywhere, the body elements of a document or section would be contained within a division element. This makes the document tree deeper. This is similar to the way HTML_ treats document contents: grouped within a ``<body>`` element. 3. Implement them as "transitions", empty elements:: <document> <section name="document"> <title> Document <paragraph> Paragraph 1 <transition> <paragraph> Paragraph 2 A transition would be a "point element", not containing anything, only identifying a point within the document structure. This keeps the document tree flatter, but the idea of a "point element" like "transition" smells bad. A transition isn't a thing itself, it's the space between two divisions. However, transitions are a practical solution. Solution 3 was chosen for incorporation into the document tree model. .. _HTML: http://www.w3.org/MarkUp/ ----------------- Not Implemented ----------------- Reworking Footnotes =================== As a further wrinkle (see `Reworking Explicit Markup (Round 1)`_ above), in the wee hours of 2002-02-28 I posted several ideas for changes to footnote syntax: - Change footnote syntax from ``.. [1]`` to ``_[1]``? ... - Differentiate (with new DTD elements) author-date "citations" (``[GVR2002]``) from numbered footnotes? ... - Render footnote references as superscripts without "[]"? ... These ideas are all related, and suggest changes in the reStructuredText syntax as well as the docutils tree model. The footnote has been used for both true footnotes (asides expanding on points or defining terms) and for citations (references to external works). Rather than dealing with one amalgam construct, we could separate the current footnote concept into strict footnotes and citations. Citations could be interpreted and treated differently from footnotes. Footnotes would be limited to numerical labels: manual ("1") and auto-numbered (anonymous "#", named "#label"). The footnote is the only explicit markup construct (starts with ".. ") that directly translates to a visible body element. I've always been a little bit uncomfortable with the ".. " marker for footnotes because of this; ".. " has a connotation of "special", but footnotes aren't especially "special". Printed texts often put footnotes at the bottom of the page where the reference occurs (thus "foot note"). Some HTML designs would leave footnotes to be rendered the same positions where they're defined. Other online and printed designs will gather footnotes into a section near the end of the document, converting them to "endnotes" (perhaps using a directive in our case); but this "special processing" is not an intrinsic property of the footnote itself, but a decision made by the document author or processing system. Citations are almost invariably collected in a section at the end of a document or section. Citations "disappear" from where they are defined and are magically reinserted at some well-defined point. There's more of a connection to the "special" connotation of the ".. " syntax. The point at which the list of citations is inserted could be defined manually by a directive (e.g., ".. citations::"), and/or have default behavior (e.g., a section automatically inserted at the end of the document) that might be influenced by options to the Writer. Syntax proposals: + Footnotes: - Current syntax:: .. [1] Footnote 1 .. [#] Auto-numbered footnote. .. [#label] Auto-labeled footnote. - The syntax proposed in the original 2002-02-28 Doc-SIG post: remove the ".. ", prefix a "_":: _[1] Footnote 1 _[#] Auto-numbered footnote. _[#label] Auto-labeled footnote. The leading underscore syntax (earlier dropped because ``.. _[1]:`` was too verbose) is a useful reminder that footnotes are hyperlink targets. - Minimal syntax: remove the ".. [" and "]", prefix a "_", and suffix a ".":: _1. Footnote 1. _#. Auto-numbered footnote. _#label. Auto-labeled footnote. ``_1.``, ``_#.``, and ``_#label.`` are markers, like list markers. Footnotes could be rendered something like this in HTML | 1. This is a footnote. The brackets could be dropped | from the label, and a vertical bar could set them | off from the rest of the document in the HTML. Two-way hyperlinks on the footnote marker ("1." above) would also help to differentiate footnotes from enumerated lists. If converted to endnotes (by a directive/transform), a horizontal half-line might be used instead. Page-oriented output formats would typically use the horizontal line for true footnotes. + Footnote references: - Current syntax:: [1]_, [#]_, [#label]_ - Minimal syntax to match the minimal footnote syntax above:: 1_, #_, #label_ As a consequence, pure-numeric hyperlink references would not be possible; they'd be interpreted as footnote references. + Citation references: no change is proposed from the current footnote reference syntax:: [GVR2001]_ + Citations: - Current syntax (footnote syntax):: .. [GVR2001] Python Documentation; van Rossum, Drake, et al.; http://www.python.org/doc/ - Possible new syntax:: _[GVR2001] Python Documentation; van Rossum, Drake, et al.; http://www.python.org/doc/ _[DJG2002] Docutils: Python Documentation Utilities project; Goodger et al.; http://docutils.sourceforge.net/ Without the ".. " marker, subsequent lines would either have to align as in one of the above, or we'd have to allow loose alignment (I'd rather not):: _[GVR2001] Python Documentation; van Rossum, Drake, et al.; http://www.python.org/doc/ I proposed adopting the "minimal" syntax for footnotes and footnote references, and adding citations and citation references to reStructuredText's repertoire. The current footnote syntax for citations is better than the alternatives given. From a reply by Tony Ibbs on 2002-03-01: However, I think easier with examples, so let's create one:: Fans of Terry Pratchett are perhaps more likely to use footnotes [1]_ in their own writings than other people [2]_. Of course, in *general*, one only sees footnotes in academic or technical writing - it's use in fiction and letter writing is not normally considered good style [4]_, particularly in emails (not a medium that lends itself to footnotes). .. [1] That is, little bits of referenced text at the bottom of the page. .. [2] Because Terry himself does, of course [3]_. .. [3] Although he has the distinction of being *funny* when he does it, and his fans don't always achieve that aim. .. [4] Presumably because it detracts from linear reading of the text - this is, of course, the point. and look at it with the second syntax proposal:: Fans of Terry Pratchett are perhaps more likely to use footnotes [1]_ in their own writings than other people [2]_. Of course, in *general*, one only sees footnotes in academic or technical writing - it's use in fiction and letter writing is not normally considered good style [4]_, particularly in emails (not a medium that lends itself to footnotes). _[1] That is, little bits of referenced text at the bottom of the page. _[2] Because Terry himself does, of course [3]_. _[3] Although he has the distinction of being *funny* when he does it, and his fans don't always achieve that aim. _[4] Presumably because it detracts from linear reading of the text - this is, of course, the point. (I note here that if I have gotten the indentation of the footnotes themselves correct, this is clearly not as nice. And if the indentation should be to the left margin instead, I like that even less). and the third (new) proposal:: Fans of Terry Pratchett are perhaps more likely to use footnotes 1_ in their own writings than other people 2_. Of course, in *general*, one only sees footnotes in academic or technical writing - it's use in fiction and letter writing is not normally considered good style 4_, particularly in emails (not a medium that lends itself to footnotes). _1. That is, little bits of referenced text at the bottom of the page. _2. Because Terry himself does, of course 3_. _3. Although he has the distinction of being *funny* when he does it, and his fans don't always achieve that aim. _4. Presumably because it detracts from linear reading of the text - this is, of course, the point. I think I don't, in practice, mind the targets too much (the use of a dot after the number helps a lot here), but I do have a problem with the body text, in that I don't naturally separate out the footnotes as different than the rest of the text - instead I keep wondering why there are numbers interspered in the text. The use of brackets around the numbers ([ and ]) made me somehow parse the footnote references as "odd" - i.e., not part of the body text - and thus both easier to skip, and also (paradoxically) easier to pick out so that I could follow them. Thus, for the moment (and as always susceptable to argument), I'd say -1 on the new form of footnote reference (i.e., I much prefer the existing ``[1]_`` over the proposed ``1_``), and ambivalent over the proposed target change. That leaves David's problem of wanting to distinguish footnotes and citations - and the only thing I can propose there is that footnotes are numeric or # and citations are not (which, as a human being, I can probably cope with!). From a reply by Paul Moore on 2002-03-01: I think the current footnote syntax ``[1]_`` is *exactly* the right balance of distinctness vs unobtrusiveness. I very definitely don't think this should change. On the target change, it doesn't matter much to me. From a further reply by Tony Ibbs on 2002-03-01, referring to the "[1]" form and actual usage in email: Clearly this is a form people are used to, and thus we should consider it strongly (in the same way that the usage of ``*..*`` to mean emphasis was taken partly from email practise). Equally clearly, there is something "magical" for people in the use of a similar form (i.e., ``[1]``) for both footnote reference and footnote target - it seems natural to keep them similar. ... I think that this established plaintext usage leads me to strongly believe we should retain square brackets at both ends of a footnote. The markup of the reference end (a single trailing underscore) seems about as minimal as we can get away with. The markup of the target end depends on how one envisages the thing - if ".." means "I am a target" (as I tend to see it), then that's good, but one can also argue that the "_[1]" syntax has a neat symmetry with the footnote reference itself, if one wishes (in which case ".." presumably means "hidden/special" as David seems to think, which is why one needs a ".." *and* a leading underline for hyperlink targets. Given the persuading arguments voiced, we'll leave footnote & footnote reference syntax alone. Except that these discussions gave rise to the "auto-symbol footnote" concept, which has been added. Citations and citation references have also been added. -------- Tabled -------- Reworking Explicit Markup (Round 2) =================================== See `Reworking Explicit Markup (Round 1)`_ for an earlier discussion. In April 2004, a new thread becan on docutils-develop: `Inconsistency in RST markup`__. Several arguments were made; the first argument begat later arguments. Below, the arguments are paraphrased "in quotes", with responses. __ http://thread.gmane.org/gmane.text.docutils.devel/1386 1. References and targets take this form:: targetname_ .. _targetname: stuff But footnotes, "which generate links just like targets do", are written as:: [1]_ .. [1] stuff "Footnotes should be written as":: [1]_ .. _[1]: stuff But they're not the same type of animal. That's not a "footnote target", it's a *footnote*. Being a target is not a footnote's primary purpose (an arguable point). It just happens to grow a target automatically, for convenience. Just as a section title:: Title ===== isn't a "title target", it's a *title*, which happens to grow a target automatically. The consistency is there, it's just deeper than at first glance. Also, ".. [1]" was chosen for footnote syntax because it closely resembles one form of actual footnote rendering. ".. _[1]:" is too verbose; excessive punctuation is required to get the job done. For more of the reasoning behind the syntax, see `Problems With StructuredText (Hyperlinks) <problems.html#hyperlinks>`__ and `Reworking Footnotes`_. 2. "I expect directives to also look like ``.. this:`` [one colon] because that also closely parallels the link and footnote target markup." There are good reasons for the two-colon syntax: Two colons are used after the directive type for these reasons: - Two colons are distinctive, and unlikely to be used in common text. - Two colons avoids clashes with common comment text like:: .. Danger: modify at your own risk! - If an implementation of reStructuredText does not recognize a directive (i.e., the directive-handler is not installed), a level-3 (error) system message is generated, and the entire directive block (including the directive itself) will be included as a literal block. Thus "::" is a natural choice. -- `restructuredtext.html#directives <../../ref/rst/restructuredtext.html#directives>`__ The last reason is not particularly compelling; it's more of a convenient coincidence or mnemonic. 3. "Comments always seemed too easy. I almost never write comments. I'd have no problem writing '.. comment:' in front of my comments. In fact, it would probably be more readable, as comments *should* be set off strongly, because they are very different from normal text." Many people do use comments though, and some applications of reStructuredText require it. For example, all reStructuredText PEPs (and this document!) have an Emacs stanza at the bottom, in a comment. Having to write ".. comment::" would be very obtrusive. Comments *should* be dirt-easy to do. It should be easy to "comment out" a block of text. Comments in programming languages and other markup languages are invariably easy. Any author is welcome to preface their comments with "Comment:" or "Do Not Print" or "Note to Editor" or anything they like. A "comment" directive could easily be implemented. It might be confused with admonition directives, like "note" and "caution" though. In unrelated (and unpublished and unfinished) work, adding a "comment" directive as a true document element was considered:: If structure is necessary, we could use a "comment" directive (to avoid nonsensical DTD changes, the "comment" directive could produce an untitled topic element). 4. "One of the goals of reStructuredText is to be *readable* by people who don't know it. This construction violates that: it is not at all obvious to the uninitiated that text marked by '..' is a comment. On the other hand, '.. comment:' would be totally transparent." Totally transparent, perhaps, but also very obtrusive. Another of `reStructuredText's goals`_ is to be unobtrusive, and ".. comment::" would violate that. The goals of reStructuredText are many, and they conflict. Determining the right set of goals and finding solutions that best fit is done on a case-by-case basis. Even readability is has two aspects. Being readable without any prior knowledge is one. Being as easily read in raw form as in processed form is the other. ".." may not contribute to the former aspect, but ".. comment::" would certainly detract from the latter. .. _author's note: .. _reStructuredText's goals: ../../ref/rst/introduction.html#goals 5. "Recently I sent someone an rst document, and they got confused; I had to explain to them that '..' marks comments, *unless* it's a directive, etc..." The explanation of directives *is* roundabout, defining comments in terms of not being other things. That's definitely a wart. 6. "Under the current system, a mistyped directive (with ':' instead of '::') will be silently ignored. This is an error that could easily go unnoticed." A parser option/setting like "--comments-on-stderr" would help. 7. "I'd prefer to see double-dot-space / command / double-colon as the standard Docutils markup-marker. It's unusual enough to avoid being accidently used. Everything that starts with a double-dot should end with a double-colon." That would increase the punctuation verbosity of some constructs considerably. 8. Edward Loper proposed the following plan for backwards compatibility: 1. ".. foo" will generate a deprecation warning to stderr, and nothing in the output (no system messages). 2. ".. foo: bar" will be treated as a directive foo. If there is no foo directive, then do the normal error output. 3. ".. foo:: bar" will generate a deprecation warning to stderr, and be treated as a directive. Or leave it valid? So some existing documents might start printing deprecation warnings, but the only existing documents that would *break* would be ones that say something like:: .. warning: this should be a comment instead of:: .. warning:: this should be a comment Here, we're trading fairly common a silent error (directive falsely treated as a comment) for a fairly uncommon explicitly flagged error (comment falsely treated as directive). To make things even easier, we could add a sentence to the unknown-directive error. Something like "If you intended to create a comment, please use '.. comment:' instead". On one hand, I understand and sympathize with the points raised. On the other hand, I think the current syntax strikes the right balance (but I acknowledge a possible lack of objectivity). On the gripping hand, the comment and directive syntax has become well established, so even if it's a wart, it may be a wart we have to live with. Making any of these changes would cause a lot of breakage or at least deprecation warnings. I'm not sure the benefit is worth the cost. For now, we'll treat this as an unresolved legacy issue. ------- To Do ------- Auto-Enumerated Lists ===================== The advantage of auto-numbered enumerated lists would be similar to that of auto-numbered footnotes: lists could be written and rearranged without having to manually renumber them. The disadvantages are also the same: input and output wouldn't match exactly; the markup may be ugly or confusing (depending on which alternative is chosen). 1. Use the "#" symbol. Example:: #. Item 1. #. Item 2. #. Item 3. Advantages: simple, explicit. Disadvantage: enumeration sequence cannot be specified (limited to arabic numerals); ugly. 2. As a variation on #1, first initialize the enumeration sequence? For example:: a) Item a. #) Item b. #) Item c. Advantages: simple, explicit, any enumeration sequence possible. Disadvantages: ugly; perhaps confusing with mixed concrete/abstract enumerators. 3. Alternative suggested by Fred Bremmer, from experience with MoinMoin:: 1. Item 1. 1. Item 2. 1. Item 3. Advantages: enumeration sequence is explicit (could be multiple "a." or "(I)" tokens). Disadvantages: perhaps confusing; otherwise erroneous input (e.g., a duplicate item "1.") would pass silently, either causing a problem later in the list (if no blank lines between items) or creating two lists (with blanks). Take this input for example:: 1. Item 1. 1. Unintentional duplicate of item 1. 2. Item 2. Currently the parser will produce two list, "1" and "1,2" (no warnings, because of the presence of blank lines). Using Fred's notation, the current behavior is "1,1,2 -> 1 1,2" (without blank lines between items, it would be "1,1,2 -> 1 [WARNING] 1,2"). What should the behavior be with auto-numbering? Fred has produced a patch__, whose initial behavior is as follows:: 1,1,1 -> 1,2,3 1,2,2 -> 1,2,3 3,3,3 -> 3,4,5 1,2,2,3 -> 1,2,3 [WARNING] 3 1,1,2 -> 1,2 [WARNING] 2 (After the "[WARNING]", the "3" would begin a new list.) I have mixed feelings about adding this functionality to the spec & parser. It would certainly be useful to some users (myself included; I often have to renumber lists). Perhaps it's too clever, asking the parser to guess too much. What if you *do* want three one-item lists in a row, each beginning with "1."? You'd have to use empty comments to force breaks. Also, I question whether "1,2,2 -> 1,2,3" is optimal behavior. In response, Fred came up with "a stricter and more explicit rule [which] would be to only auto-number silently if *all* the enumerators of a list were identical". In that case:: 1,1,1 -> 1,2,3 1,2,2 -> 1,2 [WARNING] 2 3,3,3 -> 3,4,5 1,2,2,3 -> 1,2 [WARNING] 2,3 1,1,2 -> 1,2 [WARNING] 2 Should any start-value be allowed ("3,3,3"), or should auto-numbered lists be limited to begin with ordinal-1 ("1", "A", "a", "I", or "i")? __ http://sourceforge.net/tracker/index.php?func=detail&aid=548802 &group_id=38414&atid=422032 4. Alternative proposed by Tony Ibbs:: #1. First item. #3. Aha - I edited this in later. #2. Second item. The initial proposal required unique enumerators within a list, but this limits the convenience of a feature of already limited applicability and convenience. Not a useful requirement; dropped. Instead, simply prepend a "#" to a standard list enumerator to indicate auto-enumeration. The numbers (or letters) of the enumerators themselves are not significant, except: - as a sequence indicator (arabic, roman, alphabetic; upper/lower), - and perhaps as a start value (first list item). Advantages: explicit, any enumeration sequence possible. Disadvantages: a bit ugly. Nested Inline Markup ==================== These are collected notes on a long-discussed issue. The original mailing list messages should be referred to for details. * In a 2001-10-31 discussion I wrote: Try, for example, `Ed Loper's 2001-03-21 post`_, which details some rules for nested inline markup. I think the complexity is prohibitive for the marginal benefit. (And if you can understand that tree without going mad, you're a better man than I. ;-) Inline markup is already fragile. Allowing nested inline markup would only be asking for trouble IMHO. If it proves absolutely necessary, it can be added later. The rules for what can appear inside what must be well thought out first though. .. _Ed Loper's 2001-03-21 post: http://mail.python.org/pipermail/doc-sig/2001-March/001487.html -- http://mail.python.org/pipermail/doc-sig/2001-October/002354.html * In a 2001-11-09 Doc-SIG post, I wrote: The problem is that in the what-you-see-is-more-or-less-what-you-get markup language that is reStructuredText, the symbols used for inline markup ("*", "**", "`", "``", etc.) may preclude nesting. I've rethought this position. Nested markup is not precluded, just tricky. People and software parse "double and 'single' quotes" all the time. Continuing, I've thought over how we might implement nested inline markup. The first algorithm ("first identify the outer inline markup as we do now, then recursively scan for nested inline markup") won't work; counterexamples were given in my `last post <http://mail.python.org/pipermail/doc-sig/2001-November/002363.html>`__. The second algorithm makes my head hurt:: while 1: scan for start-string if found: push on stack scan for start or end string if new start string found: recurse elif matching end string found: pop stack elif non-matching end string found: if its a markup error: generate warning elif the initial start-string was misinterpreted: # e.g. in this case: ***strong** in emphasis* restart with the other interpretation # but it might be several layers back ... ... This is similar to how the parser does section title recognition, but sections are much more regular and deterministic. Bottom line is, I don't think the benefits are worth the effort, even if it is possible. I'm not going to try to write the code, at least not now. If somebody codes up a consistent, working, general solution, I'll be happy to consider it. -- http://mail.python.org/pipermail/doc-sig/2001-November/002388.html * In a `2003-05-06 Docutils-Users post`__ Paul Tremblay proposed a new syntax to allow for easier nesting. It eventually evolved into this:: :role:[inline text] The duplication with the existing interpreted text syntax is problematic though. __ http://article.gmane.org/gmane.text.docutils.user/317 * Could the parser be extended to parse nested interpreted text? :: :emphasis:`Some emphasized text with :strong:`some more emphasized text` in it and **perhaps** :reference:`a link`` * In a `2003-06-18 Docutils-Develop post`__, Mark Nodine reported on his implementation of a form of nested inline markup in his Perl-based parser (unpublished). He brought up some interesting ideas. The implementation was flawed, however, by the change in semantics required for backslash escapes. __ http://article.gmane.org/gmane.text.docutils.devel/795 * Docutils-develop threads between David Abrahams, David Goodger, and Mark Nodine (beginning 2004-01-16__ and 2004-01-19__) hashed out many of the details of a potentially successful implementation, as described below. David Abrahams checked in code to the "nesting" branch of CVS, awaiting thorough review. __ http://thread.gmane.org/gmane.text.docutils.devel/1102 __ http://thread.gmane.org/gmane.text.docutils.devel/1125 It may be possible to accomplish nested inline markup in general with a more powerful inline markup parser. There may be some issues, but I'm not averse to the idea of nested inline markup in general. I just don't have the time or inclination to write a new parser now. Of course, a good patch would be welcome! I envisage something like this. Explicit-role interpreted text must be nestable. Prefix-based is probably preferred, since suffix-based will look like inline literals:: ``text`:role1:`:role2: But it can be disambiguated, so it ought to be left up to the author:: `\ `text`:role1:`:role2: In addition, other forms of inline markup may be nested if unambiguous:: *emphasized ``literal`` and |substitution ref| and link_* IOW, the parser ought to be as permissive as possible. Syntax for Line Blocks ====================== * An early idea: How about a literal-block-like prefix, perhaps "``;;``"? (It is, after all, a *semi-literal* literal block, no?) Example:: Take it away, Eric the Orchestra Leader! ;; A one, two, a one two three four Half a bee, philosophically, must, *ipso facto*, half not be. But half the bee has got to be, *vis a vis* its entity. D'you see? But can a bee be said to be or not to be an entire bee, when half the bee is not a bee, due to some ancient injury? Singing... Kinda lame. * Another idea: in an ordinary paragraph, if the first line ends with a backslash (escaping the newline), interpret the entire paragraph as a verse block? For example:: Add just one backslash\ And this paragraph becomes An awful haiku (Awful, and arguably invalid, since in Japanese the word "haiku" contains three syllables not two.) This idea was superceded by the rules for escaped whitespace, useful for `character-level inline markup`_. * In a `2004-02-22 docutils-develop message`__, Jarno Elonen proposed a "plain list" syntax (and also provided a patch):: | John Doe | President, SuperDuper Corp. | jdoe@example.org __ http://thread.gmane.org/gmane.text.docutils.devel/1187 This syntax is very natural. However, these "plain lists" seem very similar to line blocks, and I see so little intrinsic "list-ness" that I'm loathe to add a new object. I used the term "blurbs" to remove the "list" connotation from the originally proposed name. Perhaps line blocks could be refined to add the two properties they currently lack: A) long lines wrap nicely B) HTML output doesn't look like program code in non-CSS web browsers (A) is an issue of all 3 aspects of Docutils: syntax (construct behaviour), internal representation, and output. (B) is partly an issue of internal representation but mostly of output. ReStructuredText will redefine line blocks with the "|"-quoting syntax. The following is my current thinking. Syntax ------ Perhaps line block syntax like this would do:: | M6: James Bond | MIB: Mr. J. | IMF: not decided yet, but probably one of the following: | Ethan Hunt | Jim Phelps | Claire Phelps | CIA: Felix Leiter Note that the "nested" list does not have nested syntax (the "|" are not further indented); the leading whitespace would still be significant somehow (more below). As for long lines in the input, this could suffice:: | John Doe | Founder, President, Chief Executive Officer, Cook, Bottle Washer, and All-Round Great Guy | SuperDuper Corp. | jdoe@example.org The lack of "|" on the third line indicates that it's a continuation of the second line, wrapped. I don't see much point in allowing arbitrary nested content. Multiple paragraphs or bullet lists inside a "blurb" doesn't make sense to me. Simple nested line blocks should suffice. Internal Representation ----------------------- Line blocks are currently represented as text blobs as follows:: <!ELEMENT line_block %text.model;> <!ATTLIST line_block %basic.atts; %fixedspace.att;> Instead, we could represent each line by a separate element:: <!ELEMENT line_block (line+)> <!ATTLIST line_block %basic.atts;> <!ELEMENT line %text.model;> <!ATTLIST line %basic.atts;> We'd keep the significance of the leading whitespace of each line either by converting it to non-breaking spaces at output, or with a per-line margin. Non-breaking spaces are simpler (for HTML, anyway) but kludgey, and wouldn't support indented long lines that wrap. But should inter-word whitespace (i.e., not leading whitespace) be preserved? Currently it is preserved in line blocks. Representing a more complex line block may be tricky:: | But can a bee be said to be | or not to be an entire bee, | when half the bee is not a bee, | due to some ancient injury? Perhaps the representation could allow for nested line blocks:: <!ELEMENT line_block (line | line_block)+> With this model, leading whitespace would no longer be significant. Instead, left margins are implied by the nesting. The example above could be represented as follows:: <line_block> <line> But can a bee be said to be <line_block> <line> or not to be an entire bee, <line_block> <line> when half the bee is not a bee, <line_block> <line> due to some ancient injury? I wasn't sure what to do about even more complex line blocks:: | Indented | Not indented | Indented a bit | A bit more | Only one space How should that be parsed and nested? Should the first line have the same nesting level (== indentation in the output) as the fourth line, or the same as the last line? Mark Nodine suggested that such line blocks be parsed similarly to complexly-nested block quotes, which seems reasonable. In the example above, this would result in the nesting of first line matching the last line's nesting. In other words, the nesting would be relative to neighboring lines only. Output ------ In HTML, line blocks are currently output as "<pre>" blocks, which gives us significant whitespace and line breaks, but doesn't allow long lines to wrap and causes monospaced output without stylesheets. Instead, we could output "<div>" elements parallelling the representation above, where each nested <div class="line_block"> would have an increased left margin (specified in the stylesheet). Jarno suggested the following HTML output:: <div class="line_block"> <span class="line">First, top level line</span><br class="hidden"/> <div class="line_block"><span class="hidden"> </span> <span class="line">Second, once nested</span><br class="hidden"/> <span class="line">Third, once nested</span><br class="hidden"/> ... </div> ... </div> The ``<br class="hidden" />`` and ``<span class="hidden"> </span>`` are meant to support non-CSS and non-graphical browsers. I understand the case for "br", but I'm not so sure about hidden " ". I question how much effort should be put toward supporting non-graphical and especially non-CSS browsers, at least for html4css1.py output. Should the lines themselves be ``<span>`` or ``<div>``? I don't like mixing inline and block-level elements. Implementation Plan ------------------- We'll leave the old implementation in place (via the "line-block" directive only) until all Writers have been updated to support the new syntax & implementation. The "line-block" directive can then be updated to use the new internal representation, and its documentation will be updated to recommend the new syntax. List-Driven Tables ================== The original idea came from Dylan Jay: ... to use a two level bulleted list with something to indicate it should be rendered as a table ... It's an interesting idea. It could be implemented in as a directive which transforms a uniform two-level list into a table. Using a directive would allow the author to explicitly set the table's orientation (by column or by row), the presence of row headers, etc. Alternatives: 1. Bullet-list-tables might look like this:: .. list-table:: * - Treat - Quantity - Description * - Albatross! - 299 - On a stick! * - Crunchy Frog! - 1499 - If we took the bones out, it wouldn't be crunchy, now would it? * - Gannet Ripple! - 199 - On a stick! This list must be written in two levels. This wouldn't work:: .. list-table:: * Treat * Albatross! * Gannet! * Crunchy Frog! * Quantity * 299 * 199 * 1499 * Description * On a stick! * On a stick! * If we took the bones out... The above is a single list of 12 items. The blank lines are not significant to the markup. We'd have to explicitly specify how many columns or rows to use, which isn't a good idea. 2. Beni Cherniavsky suggested a field list alternative. It could look like this:: .. field-list-table:: :headrows: 1 - :treat: Treat :quantity: Quantity :descr: Description - :treat: Albatross! :quantity: 299 :descr: On a stick! - :treat: Crunchy Frog! :quantity: 1499 :descr: If we took the bones out, it wouldn't be crunchy, now would it? Column order is determined from the order of fields in the first row. Field order in all other rows is ignored. As a side-effect, this allows trivial re-arrangement of columns. By using named fields, it becomes possible to omit fields in some rows without losing track of things, which is important for spans. 3. An alternative to two-level bullet lists would be to use enumerated lists for the table cells:: .. list-table:: * 1. Treat 2. Quantity 3. Description * 1. Albatross! 2. 299 3. On a stick! * 1. Crunchy Frog! 2. 1499 3. If we took the bones out, it wouldn't be crunchy, now would it? That provides better correspondence between cells in the same column than does bullet-list syntax, but not as good as field list syntax. I think that were only field-list-tables available, a lot of users would use the equivalent degenerate case:: .. field-list-table:: - :1: Treat :2: Quantity :3: Description ... 4. Another natural variant is to allow a description list with field lists as descriptions:: .. list-table:: :headrows: 1 Treat :quantity: Quantity :descr: Description Albatross! :quantity: 299 :descr: On a stick! Crunchy Frog! :quantity: 1499 :descr: If we took the bones out, it wouldn't be crunchy, now would it? This would make the whole first column a header column ("stub"). It's limited to a single column and a single paragraph fitting on one source line. Also it wouldn't allow for empty cells or row spans in the first column. But these are limitations that we could live with, like those of simple tables. The List-driven table feature could be done in many ways. Each user will have their preferred usage. Perhaps a single "list-table" directive could handle them all, depending on which options and content are present. Issues: * How to indicate that there's 1 header row? Perhaps two lists? :: .. list-table:: + - Treat - Quantity - Description * - Albatross! - 299 - On a stick! This is probably too subtle though. Better would be a directive option, like ``:headrows: 1``. An early suggestion for the header row(s) was to use a directive option:: .. field-list-table:: :header: - :treat: Treat :quantity: Quantity :descr: Description - :treat: Albatross! :quantity: 299 :descr: On a stick! But the table data is at two levels and looks inconsistent. In general, we cannot extract the header row from field lists' field names because field names cannot contain everything one might put in a table cell. A separate header row also allows shorter field names and doesn't force one to rewrite the whole table when the header text changes. But for simpler cases, we can offer a ":header: fields" option, which does extract header cells from field names:: .. field-list-table:: :header: fields - :Treat: Albatross! :Quantity: 299 :Description: On a stick! * How to indicate the column widths? A directive option? :: .. list-table:: :widths: 15 10 35 Automatic defaults from the text used? * How to handle row and/or column spans? In a field list, column-spans can be indicated by specifying the first and last fields, separated by space-dash-space or ellipsis:: - :foo - baz: quuux - :foo ... baz: quuux Commas were proposed for column spans:: - :foo, bar: quux But non-adjacent columns become problematic. Should we report an error, or duplicate the value into each span of adjacent columns (as was suggested)? The latter suggestion is appealing but may be too clever. Best perhaps to simply specify the two ends. It was suggested that comma syntax should be allowed, too, in order to allow the user to avoid trouble when changing the column order. But changing the column order of a table with spans is not trivial; we shouldn't make it easier to mess up. One possible syntax for row-spans is to simply treat any row where a field is missing as a row-span from the last row where it appeared. Leaving a field empty would still be possible by writing a field with empty content. But this is too implicit. Another way would be to require an explicit continuation marker (``...``/``-"-``/``"``?) in all but the first row of a spanned field. Empty comments could work (".."). If implemented, the same marker could also be supported in simple tables, which lack row-spanning abilities. Explicit markup like ":rowspan:" and ":colspan:" was also suggested. Sometimes in a table, the first header row contains spans. It may be necessary to provide a way to specify the column field names independently of data rows. A directive option would do it. * We could specify "column-wise" or "row-wise" ordering, with the same markup structure. For example, with definition data:: .. list-table:: :column-wise: Treat - Albatross! - Crunchy Frog! Quantity - 299 - 1499 Description - On a stick! - If we took the bones out, it wouldn't be crunchy, now would it? * A syntax for stubs in ASCII-art tables is easy to imagine:: +------------------------++------------+----------+ | Header row, column 1 || Header 2 | Header 3 | +========================++============+==========+ | body row 1, column 1 || column 2 | column 3 | +------------------------++------------+----------+ ------------------- ... Or Not To Do? ------------------- This is the realm of the possible but questionably probable. These ideas are kept here as a record of what has been proposed, for posterity and in case any of them prove to be useful. Compound Enumerated Lists ========================= Allow for compound enumerators, such as "1.1." or "1.a." or "1(a)", to allow for nested enumerated lists without indentation? Sloppy Indentation of List Items ================================ Perhaps the indentation shouldn't be so strict. Currently, this is required:: 1. First line, second line. Anything wrong with this? :: 1. First line, second line. Problem? :: 1. First para. Block quote. (no good: requires some indent relative to first para) Second Para. 2. Have to carefully define where the literal block ends:: Literal block Literal block? Hmm... Non-strict indentation isn't such a good idea. Lazy Indentation of List Items ============================== Another approach: Going back to the first draft of reStructuredText (2000-11-27 post to Doc-SIG):: - This is the fourth item of the main list (no blank line above). The second line of this item is not indented relative to the bullet, which precludes it from having a second paragraph. Change that to *require* a blank line above and below, to reduce ambiguity. This "loosening" may be added later, once the parser's been nailed down. However, a serious drawback of this approach is to limit the content of each list item to a single paragraph. David's Idea for Lazy Indentation --------------------------------- Consider a paragraph in a word processor. It is a single logical line of text which ends with a newline, soft-wrapped arbitrarily at the right edge of the page or screen. We can think of a plaintext paragraph in the same way, as a single logical line of text, ending with two newlines (a blank line) instead of one, and which may contain arbitrary line breaks (newlines) where it was accidentally hard-wrapped by an application. We can compensate for the accidental hard-wrapping by "unwrapping" every unindented second and subsequent line. The indentation of the first line of a paragraph or list item would determine the indentation for the entire element. Blank lines would be required between list items when using lazy indentation. The following example shows the lazy indentation of multiple body elements:: - This is the first paragraph of the first list item. Here is the second paragraph of the first list item. - This is the first paragraph of the second list item. Here is the second paragraph of the second list item. A more complex example shows the limitations of lazy indentation:: - This is the first paragraph of the first list item. Next is a definition list item: Term Definition. The indentation of the term is required, as is the indentation of the definition's first line. When the definition extends to more than one line, lazy indentation may occur. (This is the second paragraph of the definition.) - This is the first paragraph of the second list item. - Here is the first paragraph of the first item of a nested list. So this paragraph would be outside of the nested list, but inside the second list item of the outer list. But this paragraph is not part of the list at all. And the ambiguity remains:: - Look at the hyphen at the beginning of the next line - is it a second list item marker, or a dash in the text? Similarly, we may want to refer to numbers inside enumerated lists: 1. How many socks in a pair? There are 2. How many pants in a pair? Exactly 1. Go figure. Literal blocks and block quotes would still require consistent indentation for all their lines. For block quotes, we might be able to get away with only requiring that the first line of each contained element be indented. For example:: Here's a paragraph. This is a paragraph inside a block quote. Second and subsequent lines need not be indented at all. - A bullet list inside the block quote. Second paragraph of the bullet list inside the block quote. Although feasible, this form of lazy indentation has problems. The document structure and hierarchy is not obvious from the indentation, making the source plaintext difficult to read. This will also make keeping track of the indentation while writing difficult and error-prone. However, these problems may be acceptable for Wikis and email mode, where we may be able to rely on less complex structure (few nested lists, for example). Multiple Roles in Interpreted Text ================================== In reStructuredText, inline markup cannot be nested (yet; `see above`__). This also applies to interpreted text. In order to simultaneously combine multiple roles for a single piece of text, a syntax extension would be necessary. Ideas: 1. Initial idea:: `interpreted text`:role1,role2: 2. Suggested by Jason Diamond:: `interpreted text`:role1:role2: If a document is so complex as to require nested inline markup, perhaps another markup system should be considered. By design, reStructuredText does not have the flexibility of XML. __ `Nested Inline Markup`_ Parameterized Interpreted Text ============================== In some cases it may be expedient to pass parameters to interpreted text, analogous to function calls. Ideas: 1. Parameterize the interpreted text role itself (suggested by Jason Diamond):: `interpreted text`:role1(foo=bar): Positional parameters could also be supported:: `CSS`:acronym(Cascading Style Sheets): is used for HTML, and `CSS`:acronym(Content Scrambling System): is used for DVDs. Technical problem: current interpreted text syntax does not recognize roles containing whitespace. Design problem: this smells like programming language syntax, but reStructuredText is not a programming language. 2. Put the parameters inside the interpreted text:: `CSS (Cascading Style Sheets)`:acronym: is used for HTML, and `CSS (Content Scrambling System)`:acronym: is used for DVDs. Although this could be defined on an individual basis (per role), we ought to have a standard. Hyperlinks with embedded URIs already use angle brackets; perhaps they could be used here too:: `CSS <Cascading Style Sheets>`:acronym: is used for HTML, and `CSS <Content Scrambling System>`:acronym: is used for DVDs. Do angle brackets connote URLs too much for this to be acceptable? How about the "tag" connotation -- does it save them or doom them? Does this push inline markup too far? Readability becomes a serious issue. Substitutions may provide a better alternative (at the expense of verbosity and duplication) by pulling the details out of the text flow:: |CSS| is used for HTML, and |CSS-DVD| is used for DVDs. .. |CSS| acronym:: Cascading Style Sheets .. |CSS-DVD| acronym:: Content Scrambling System :text: CSS ---------------------------------------------------------------------- This whole idea may be going beyond the scope of reStructuredText. Documents requiring this functionality may be better off using XML or another markup system. This argument comes up regularly when pushing the envelope of reStructuredText syntax. I think it's a useful argument in that it provides a check on creeping featurism. In many cases, the resulting verbosity produces such unreadable plaintext that there's a natural desire *not* to use it unless absolutely necessary. It's a matter of finding the right balance. Syntax for Interpreted Text Role Bindings ========================================= The following syntax (idea from Jeffrey C. Jacobs) could be used to associate directives with roles:: .. :rewrite: class:: rewrite `She wore ribbons in her hair and it lay with streaks of grey`:rewrite: The syntax is similar to that of substitution declarations, and the directive/role association may resolve implementation issues. The semantics, ramifications, and implementation details would need to be worked out. The example above would implement the "rewrite" role as adding a ``class="rewrite"`` attribute to the interpreted text ("inline" element). The stylesheet would then pick up on the "class" attribute to do the actual formatting. The advantage of the new syntax would be flexibility. Uses other than "class" may present themselves. The disadvantage is complexity: having to implement new syntax for a relatively specialized operation, and having new semantics in existing directives ("class::" would do something different). The `"role" directive`__ has been implemented. __ ../../ref/rst/directives.html#role Character Processing ==================== Several people have suggested adding some form of character processing to reStructuredText: * Some sort of automated replacement of ASCII sequences: - ``--`` to em-dash (or ``--`` to en-dash, and ``---`` to em-dash). - Convert quotes to curly quote entities. (Essentially impossible for HTML? Unnecessary for TeX.) - Various forms of ``:-)`` to smiley icons. - ``"\ "`` to . Problem with line-wrapping though: it could end up escaping the newline. - Escaped newlines to <BR>. - Escaped period or quote or dash as a disappearing catalyst to allow character-level inline markup? * XML-style character entities, such as "©" for the copyright symbol. Docutils has no need of a character entity subsystem. Supporting Unicode and text encodings, character entities should be directly represented in the text: a copyright symbol should be represented by the copyright symbol character. If this is not possible in an authoring environment, a pre-processing stage can be added, or a table of substitution definitions can be devised. A "unicode" directive has been implemented to allow direct specification of esoteric characters. In combination with the substitution construct, "include" files defining common sets of character entities can be defined and used. `A set of character entity set definition files have been defined`__ (`tarball`__). There's also `a description and instructions for use`__. __ http://docutils.sf.net/tmp/charents/ __ http://docutils.sf.net/tmp/charents.tgz __ http://docutils.sf.net/tmp/charents/README.html To allow for `character-level inline markup`_, a limited form of character processing has been added to the spec and parser: escaped whitespace characters are removed from the processed document. Any further character processing will be of this functional type, rather than of the character-encoding type. .. _character-level inline markup: ../../ref/rst/restructuredtext.html#character-level-inline-markup * Directive idea:: .. text-replace:: "pattern" "replacement" - Support Unicode "U+XXXX" codes. - Support regexps, perhaps with alternative "regexp-replace" directive. - Flags for regexps; ":flags:" option, or individuals. - Specifically, should the default be case-sensistive or -insensitive? .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:


============================== Problems With StructuredText ============================== :Author: David Goodger :Contact: goodger@users.sourceforge.net :Revision: $Revision: 1.10 $ :Date: $Date: 2004/06/03 19:35:53 $ :Copyright: This document has been placed in the public domain. There are several problems, unresolved issues, and areas of controversy within StructuredText_ (Classic and Next Generation). In order to resolve all these issues, this analysis brings all of the issues out into the open, enumerates all the alternatives, and proposes solutions to be incorporated into the reStructuredText_ specification. .. contents:: Formal Specification ==================== The description in the original StructuredText.py has been criticized for being vague. For practical purposes, "the code *is* the spec." Tony Ibbs has been working on deducing a `detailed description`_ from the documentation and code of StructuredTextNG_. Edward Loper's STMinus_ is another attempt to formalize a spec. For this kind of a project, the specification should always precede the code. Otherwise, the markup is a moving target which can never be adopted as a standard. Of course, a specification may be revised during lifetime of the code, but without a spec there is no visible control and thus no confidence. Understanding and Extending the Code ==================================== The original StructuredText_ is a dense mass of sparsely commented code and inscrutable regular expressions. It was not designed to be extended and is very difficult to understand. StructuredTextNG_ has been designed to allow input (syntax) and output extensions, but its documentation (both internal [comments & docstrings], and external) is inadequate for the complexity of the code itself. For reStructuredText to become truly useful, perhaps even part of Python's standard library, it must have clear, understandable documentation and implementation code. For the implementation of reStructuredText to be taken seriously, it must be a sterling example of the potential of docstrings; the implementation must practice what the specification preaches. Section Structure via Indentation ================================= Setext_ required that body text be indented by 2 spaces. The original StructuredText_ and StructuredTextNG_ require that section structure be indicated through indentation, as "inspired by Python". For certain structures with a very limited, local extent (such as lists, block quotes, and literal blocks), indentation naturally indicates structure or hierarchy. For sections (which may have a very large extent), structure via indentation is unnecessary, unnatural and ambiguous. Rather, the syntax of the section title *itself* should indicate that it is a section title. The original StructuredText states that "A single-line paragraph whose immediately succeeding paragraphs are lower level is treated as a header." Requiring indentation in this way is: - Unnecessary. The vast majority of docstrings and standalone documents will have no more than one level of section structure. Requiring indentation for such docstrings is unnecessary and irritating. - Unnatural. Most published works use title style (type size, face, weight, and position) and/or section/subsection numbering rather than indentation to indicate hierarchy. This is a tradition with a very long history. - Ambiguous. A StructuredText header is indistinguishable from a one-line paragraph followed by a block quote (precluding the use of block quotes). Enumerated section titles are ambiguous (is it a header? is it a list item?). Some additional adornment must be required to confirm the line's role as a title, both to a parser and to the human reader of the source text. Python's use of significant whitespace is a wonderful (if not original) innovation, however requiring indentation in ordinary written text is hypergeneralization. reStructuredText_ indicates section structure through title adornment style (as exemplified by this document). This is far more natural. In fact, it is already in widespread use in plain text documents, including in Python's standard distribution (such as the toplevel README_ file). Character Escaping Mechanism ============================ No matter what characters are chosen for markup, some day someone will want to write documentation *about* that markup or using markup characters in a non-markup context. Therefore, any complete markup language must have an escaping or encoding mechanism. For a lightweight markup system, encoding mechanisms like SGML/XML's '*' are out. So an escaping mechanism is in. However, with carefully chosen markup, it should be necessary to use the escaping mechanism only infrequently. reStructuredText_ needs an escaping mechanism: a way to treat markup-significant characters as the characters themselves. Currently there is no such mechanism (although ZWiki uses '!'). What are the candidates? 1. ``!`` (http://dev.zope.org/Members/jim/StructuredTextWiki/NGEscaping) 2. ``\`` 3. ``~`` 4. doubling of characters The best choice for this is the backslash (``\``). It's "the single most popular escaping character in the world!", therefore familiar and unsurprising. Since characters only need to be escaped under special circumstances, which are typically those explaining technical programming issues, the use of the backslash is natural and understandable. Python docstrings can be raw (prefixed with an 'r', as in 'r""'), which would obviate the need for gratuitous doubling-up of backslashes. (On 2001-03-29 on the Doc-SIG mailing list, GvR endorsed backslash escapes, saying, "'nuff said. Backslash it is." Although neither legally binding nor irrevocable nor any kind of guarantee of anything, it is a good sign.) The rule would be: An unescaped backslash followed by any markup character escapes the character. The escaped character represents the character itself, and is prevented from playing a role in any markup interpretation. The backslash is removed from the output. A literal backslash is represented by an "escaped backslash," two backslashes in a row. A carefully constructed set of recognition rules for inline markup will obviate the need for backslash-escapes in almost all cases; see `Delimitation of Inline Markup`_ below. When an expression (requiring backslashes and other characters used for markup) becomes too complicated and therefore unreadable, a literal block may be used instead. Inside literal blocks, no markup is recognized, therefore backslashes (for the purpose of escaping markup) become unnecessary. We could allow backslashes preceding non-markup characters to remain in the output. This would make describing regular expressions and other uses of backslashes easier. However, this would complicate the markup rules and would be confusing. Blank Lines in Lists ==================== Oft-requested in Doc-SIG (the earliest reference is dated 1996-08-13) is the ability to write lists without requiring blank lines between items. In docstrings, space is at a premium. Authors want to convey their API or usage information in as compact a form as possible. StructuredText_ requires blank lines between all body elements, including list items, even when boundaries are obvious from the markup itself. In reStructuredText, blank lines are optional between list items. However, in order to eliminate ambiguity, a blank line is required before the first list item and after the last. Nested lists also require blank lines before the list start and after the list end. Bullet List Markup ================== StructuredText_ includes 'o' as a bullet character. This is dangerous and counter to the language-independent nature of the markup. There are many languages in which 'o' is a word. For example, in Spanish:: Llamame a la casa o al trabajo. (Call me at home or at work.) And in Japanese (when romanized):: Senshuu no doyoubi ni tegami o kakimashita. ([I] wrote a letter on Saturday last week.) If a paragraph containing an 'o' word wraps such that the 'o' is the first text on a line, or if a paragraph begins with such a word, it could be misinterpreted as a bullet list. In reStructuredText_, 'o' is not used as a bullet character. '-', '*', and '+' are the possible bullet characters. Enumerated List Markup ====================== StructuredText enumerated lists are allowed to begin with numbers and letters followed by a period or right-parenthesis, then whitespace. This has surprising consequences for writing styles. For example, this is recognized as an enumerated list item by StructuredText:: Mr. Creosote. People will write enumerated lists in all different ways. It is folly to try to come up with the "perfect" format for an enumerated list, and limit the docstring parser's recognition to that one format only. Rather, the parser should recognize a variety of enumerator styles. It is also recommended that the enumerator of the first list item be ordinal-1 ('1', 'A', 'a', 'I', or 'i'), as output formats may not be able to begin a list at an arbitrary enumeration. An initial idea was to require two or more consistent enumerated list items in a row. This idea proved impractical and was dropped. In practice, the presence of a proper enumerator is enough to reliably recognize an enumerated list item; any ambiguities are reported by the parser. Here's the original idea for posterity: The parser should recognize a variety of enumerator styles, mark each block as a potential enumerated list item (PELI), and interpret the enumerators of adjacent PELIs to decide whether they make up a consistent enumerated list. If a PELI is labeled with a "1.", and is immediately followed by a PELI labeled with a "2.", we've got an enumerated list. Or "(A)" followed by "(B)". Or "i)" followed by "ii)", etc. The chances of accidentally recognizing two adjacent and consistently labeled PELIs, are acceptably small. For an enumerated list to be recognized, the following must be true: - the list must consist of multiple adjacent list items (2 or more) - the enumerators must all have the same format - the enumerators must be sequential Definition List Markup ====================== StructuredText uses ' -- ' (whitespace, two hyphens, whitespace) on the first line of a paragraph to indicate a definition list item. The ' -- ' serves to separate the term (on the left) from the definition (on the right). Many people use ' -- ' as an em-dash in their text, conflicting with the StructuredText usage. Although the Chicago Manual of Style says that spaces should not be used around an em-dash, Peter Funk pointed out that this is standard usage in German (according to the Duden, the official German reference), and possibly in other languages as well. The widespread use of ' -- ' precludes its use for definition lists; it would violate the "unsurprising" criterion. A simpler, and at least equally visually distinctive construct (proposed by Guido van Rossum, who incidentally is a frequent user of ' -- ') would do just as well:: term 1 Definition. term 2 Definition 2, paragraph 1. Definition 2, paragraph 2. A reStructuredText definition list item consists of a term and a definition. A term is a simple one-line paragraph. A definition is a block indented relative to the term, and may contain multiple paragraphs and other body elements. No blank line precedes a definition (this distinguishes definition lists from block quotes). Literal Blocks ============== The StructuredText_ specification has literal blocks indicated by 'example', 'examples', or '::' ending the preceding paragraph. STNG only recognizes '::'; 'example'/'examples' are not implemented. This is good; it fixes an unnecessary language dependency. The problem is what to do with the sometimes- unwanted '::'. In reStructuredText_ '::' at the end of a paragraph indicates that subsequent *indented* blocks are treated as literal text. No further markup interpretation is done within literal blocks (not even backslash-escapes). If the '::' is preceded by whitespace, '::' is omitted from the output; if '::' was the sole content of a paragraph, the entire paragraph is removed (no 'empty' paragraph remains). If '::' is preceded by a non-whitespace character, '::' is replaced by ':' (i.e., the extra colon is removed). Thus, a section could begin with a literal block as follows:: Section Title ------------- :: print "this is example literal" Tables ====== The table markup scheme in classic StructuredText was horrible. Its omission from StructuredTextNG is welcome, and its markup will not be repeated here. However, tables themselves are useful in documentation. Alternatives: 1. This format is the most natural and obvious. It was independently invented (no great feat of creation!), and later found to be the format supported by the `Emacs table mode`_:: +------------+------------+------------+--------------+ | Header 1 | Header 2 | Header 3 | Header 4 | +============+============+============+==============+ | Column 1 | Column 2 | Column 3 & 4 span (Row 1) | +------------+------------+------------+--------------+ | Column 1 & 2 span | Column 3 | - Column 4 | +------------+------------+------------+ - Row 2 & 3 | | 1 | 2 | 3 | - span | +------------+------------+------------+--------------+ Tables are described with a visual outline made up of the characters '-', '=', '|', and '+': - The hyphen ('-') is used for horizontal lines (row separators). - The equals sign ('=') is optionally used as a header separator (as of version 1.5.24, this is not supported by the Emacs table mode). - The vertical bar ('|') is used for for vertical lines (column separators). - The plus sign ('+') is used for intersections of horizontal and vertical lines. Row and column spans are possible simply by omitting the column or row separators, respectively. The header row separator must be complete; in other words, a header cell may not span into the table body. Each cell contains body elements, and may have multiple paragraphs, lists, etc. Initial spaces for a left margin are allowed; the first line of text in a cell determines its left margin. 2. Below is a simpler table structure. It may be better suited to manual input than alternative #1, but there is no Emacs editing mode available. One disadvantage is that it resembles section titles; a one-column table would look exactly like section & subsection titles. :: ============ ============ ============ ============== Header 1 Header 2 Header 3 Header 4 ============ ============ ============ ============== Column 1 Column 2 Column 3 & 4 span (Row 1) ------------ ------------ --------------------------- Column 1 & 2 span Column 3 - Column 4 ------------------------- ------------ - Row 2 & 3 1 2 3 - span ============ ============ ============ ============== The table begins with a top border of equals signs with a space at each column boundary (regardless of spans). Each row is underlined. Internal row separators are underlines of '-', with spaces at column boundaries. The last of the optional head rows is underlined with '=', again with spaces at column boundaries. Column spans have no spaces in their underline. Row spans simply lack an underline at the row boundary. The bottom boundary of the table consists of '=' underlines. A blank line is required following a table. 3. A minimalist alternative is as follows:: ==== ===== ======== ======== ======= ==== ===== ===== Old State Input Action New State Notes ----------- -------- ----------------- ----------- ids types new type sys.msg. dupname ids types ==== ===== ======== ======== ======= ==== ===== ===== -- -- explicit -- -- new True -- -- implicit -- -- new False None False explicit -- -- new True old False explicit implicit old new True None True explicit explicit new None True old True explicit explicit new,old None True [1] None False implicit implicit new None False old False implicit implicit new,old None False None True implicit implicit new None True old True implicit implicit new old True ==== ===== ======== ======== ======= ==== ===== ===== The table begins with a top border of equals signs with one or more spaces at each column boundary (regardless of spans). There must be at least two columns in the table (to differentiate it from section headers). Each line starts a new row. The rightmost column is unbounded; text may continue past the edge of the table. Each row/line must contain spaces at column boundaries, except for explicit column spans. Underlines of '-' can be used to indicate column spans, but should be used sparingly if at all. Lines containing column span underlines may not contain any other text. The last of the optional head rows is underlined with '=', again with spaces at column boundaries. The bottom boundary of the table consists of '=' underlines. A blank line is required following a table. This table sums up the features. Using all the features in such a small space is not pretty though:: ======== ======== ======== Header 2 & 3 Span ------------------ Header 1 Header 2 Header 3 ======== ======== ======== Each line is a new row. Each row consists of one line only. Row spans are not possible. The last column may spill over to the right. Column spans are possible with an underline joining columns. ---------------------------- The span is limited to the row above the underline. ======== ======== ======== 4. As a variation of alternative 3, bullet list syntax in the first column could be used to indicate row starts. Multi-line rows are possible, but row spans are not. For example:: ===== ===== col 1 col 2 ===== ===== - 1 Second column of row 1. - 2 Second column of row 2. Second line of paragraph. - 3 Second column of row 3. Second paragraph of row 3, column 2 ===== ===== Column spans would be indicated on the line after the last line of the row. To indicate a real bullet list within a first-column cell, simply nest the bullets. 5. In a further variation, we could simply assume that whitespace in the first column implies a multi-line row; the text in other columns is continuation text. For example:: ===== ===== col 1 col 2 ===== ===== 1 Second column of row 1. 2 Second column of row 2. Second line of paragraph. 3 Second column of row 3. Second paragraph of row 3, column 2 ===== ===== Limitations of this approach: - Cells in the first column are limited to one line of text. - Cells in the first column *must* contain some text; blank cells would lead to a misinterpretation. An empty comment ("..") is sufficient. 6. Combining alternative 3 and 4, a bullet list in the first column could mean multi-line rows, and no bullet list means single-line rows only. Alternatives 1 and 5 has been adopted by reStructuredText. Delimitation of Inline Markup ============================= StructuredText specifies that inline markup must begin with whitespace, precluding such constructs as parenthesized or quoted emphatic text:: "**What?**" she cried. (*exit stage left*) The `reStructuredText markup specification`_ allows for such constructs and disambiguates inline markup through a set of recognition rules. These recognition rules define the context of markup start-strings and end-strings, allowing markup characters to be used in most non-markup contexts without a problem (or a backslash). So we can say, "Use asterisks (*) around words or phrases to *emphasisze* them." The '(*)' will not be recognized as markup. This reduces the need for markup escaping to the point where an escape character is *almost* (but not quite!) unnecessary. Underlining =========== StructuredText uses '_text_' to indicate underlining. To quote David Ascher in his 2000-01-21 Doc-SIG mailing list post, "Docstring grammar: a very revised proposal": The tagging of underlined text with _'s is suboptimal. Underlines shouldn't be used from a typographic perspective (underlines were designed to be used in manuscripts to communicate to the typesetter that the text should be italicized -- no well-typeset book ever uses underlines), and conflict with double-underscored Python variable names (__init__ and the like), which would get truncated and underlined when that effect is not desired. Note that while *complete* markup would prevent that truncation ('__init__'), I think of docstring markups much like I think of type annotations -- they should be optional and above all do no harm. In this case the underline markup does harm. Underlining is not part of the reStructuredText specification. Inline Literals =============== StructuredText's markup for inline literals (text left as-is, verbatim, usually in a monospaced font; as in HTML ) is single quotes ('literals'). The problem with single quotes is that they are too often used for other purposes: - Apostrophes: "Don't blame me, 'cause it ain't mine, it's Chris'."; - Quoting text: First Bruce: "Well Bruce, I heard the prime minister use it. 'S'hot enough to boil a monkey's bum in 'ere your Majesty,' he said, and she smiled quietly to herself." In the UK, single quotes are used for dialogue in published works. - String literals: s = '' Alternatives:: 'text' \'text\' ''text'' "text" \"text\" ""text"" #text# @text@ `text` ^text^ ``text'' ``text`` The examples below contain inline literals, quoted text, and apostrophes. Each example should evaluate to the following HTML:: Some code, with a 'quote', "double", ain't it grand? Does a[b] = 'c' + "d" + `2^3` work? 0. Some code, with a quote, double, ain't it grand? Does a[b] = 'c' + "d" + `2^3` work? 1. Some 'code', with a \'quote\', "double", ain\'t it grand? Does 'a[b] = \'c\' + "d" + `2^3`' work? 2. Some \'code\', with a 'quote', "double", ain't it grand? Does \'a[b] = 'c' + "d" + `2^3`\' work? 3. Some ''code'', with a 'quote', "double", ain't it grand? Does ''a[b] = 'c' + "d" + `2^3`'' work? 4. Some "code", with a 'quote', \"double\", ain't it grand? Does "a[b] = 'c' + "d" + `2^3`" work? 5. Some \"code\", with a 'quote', "double", ain't it grand? Does \"a[b] = 'c' + "d" + `2^3`\" work? 6. Some ""code"", with a 'quote', "double", ain't it grand? Does ""a[b] = 'c' + "d" + `2^3`"" work? 7. Some #code#, with a 'quote', "double", ain't it grand? Does #a[b] = 'c' + "d" + `2^3`# work? 8. Some @code@, with a 'quote', "double", ain't it grand? Does @a[b] = 'c' + "d" + `2^3`@ work? 9. Some `code`, with a 'quote', "double", ain't it grand? Does `a[b] = 'c' + "d" + \`2^3\`` work? 10. Some ^code^, with a 'quote', "double", ain't it grand? Does ^a[b] = 'c' + "d" + `2\^3`^ work? 11. Some ``code'', with a 'quote', "double", ain't it grand? Does ``a[b] = 'c' + "d" + `2^3`'' work? 12. Some ``code``, with a 'quote', "double", ain't it grand? Does ``a[b] = 'c' + "d" + `2^3\``` work? Backquotes (#9 & #12) are the best choice. They are unobtrusive and relatviely rarely used (more rarely than ' or ", anyhow). Backquotes have the connotation of 'quotes', which other options (like carets, #10) don't. Analogously with ``*emph*`` & ``**strong**``, double-backquotes (#12) could be used for inline literals. If single-backquotes are used for 'interpreted text' (context-sensitive domain-specific descriptive markup) such as function name hyperlinks in Python docstrings, then double-backquotes could be used for absolute-literals, wherein no processing whatsoever takes place. An advantage of double-backquotes would be that backslash-escaping would no longer be necessary for embedded single-backquotes; however, embedded double-backquotes (in an end-string context) would be illegal. See `Backquotes in Phrase-Links`__ in `Record of reStructuredText Syntax Alternatives`__. __ alternatives.html#backquotes-in-phrase-links __ alternatives.html Alternative choices are carets (#10) and TeX-style quotes (#11). For examples of TeX-style quoting, see http://www.zope.org/Members/jim/StructuredTextWiki/CustomizingTheDocumentProcessor. Some existing uses of backquotes: 1. As a synonym for repr() in Python. 2. For command-interpolation in shell scripts. 3. Used as open-quotes in TeX code (and carried over into plaintext by TeXies). The inline markup start-string and end-string recognition rules defined by the `reStructuredText markup specification`_ would allow all of these cases inside inline literals, with very few exceptions. As a fallback, literal blocks could handle all cases. Outside of inline literals, the above uses of backquotes would require backslash-escaping. However, these are all prime examples of text that should be marked up with inline literals. If either backquotes or straight single-quotes are used as markup, TeX-quotes are too troublesome to support, so no special-casing of TeX-quotes should be done (at least at first). If TeX-quotes have to be used outside of literals, a single backslash-escaped would suffice: \``TeX quote''. Ugly, true, but very infrequently used. Using literal blocks is a fallback option which removes the need for backslash-escaping:: like this:: Here, we can do ``absolutely'' anything `'`'\|/|\ we like! No mechanism for inline literals is perfect, just as no escaping mechanism is perfect. No matter what we use, complicated inline expressions involving the inline literal quote and/or the backslash will end up looking ugly. We can only choose the least often ugly option. reStructuredText will use double backquotes for inline literals, and single backqoutes for interpreted text. Hyperlinks ========== There are three forms of hyperlink currently in StructuredText_: 1. (Absolute & relative URIs.) Text enclosed by double quotes followed by a colon, a URI, and concluded by punctuation plus white space, or just white space, is treated as a hyperlink:: "Python":http://www.python.org/ 2. (Absolute URIs only.) Text enclosed by double quotes followed by a comma, one or more spaces, an absolute URI and concluded by punctuation plus white space, or just white space, is treated as a hyperlink:: "mail me", mailto:me@mail.com 3. (Endnotes.) Text enclosed by brackets link to an endnote at the end of the document: at the beginning of the line, two dots, a space, and the same text in brackets, followed by the end note itself:: Please refer to the fine manual [GVR2001]. .. [GVR2001] Python Documentation, Release 2.1, van Rossum, Drake, et al., http://www.python.org/doc/ The problem with forms 1 and 2 is that they are neither intuitive nor unobtrusive (they break design goals 5 & 2). They overload double-quotes, which are too often used in ordinary text (potentially breaking design goal 4). The brackets in form 3 are also too common in ordinary text (such as [nested] asides and Python lists like [12]). Alternatives: 1. Have no special markup for hyperlinks. 2. A. Interpret and mark up hyperlinks as any contiguous text containing '://' or ':...@' (absolute URI) or '@' (email address) after an alphanumeric word. To de-emphasize the URI, simply enclose it in parentheses: Python (http://www.python.org/) B. Leave special hyperlink markup as a domain-specific extension. Hyperlinks in ordinary reStructuredText documents would be required to be standalone (i.e. the URI text inline in the document text). Processed hyperlinks (where the URI text is hidden behind the link) are important enough to warrant syntax. 3. The original Setext_ introduced a mechanism of indirect hyperlinks. A source link word ('hot word') in the text was given a trailing underscore:: Here is some text with a hyperlink_ built in. The hyperlink itself appeared at the end of the document on a line by itself, beginning with two dots, a space, the link word with a leading underscore, whitespace, and the URI itself:: .. _hyperlink http://www.123.xyz Setext used ``underscores_instead_of_spaces_`` for phrase links. With some modification, alternative 3 best satisfies the design goals. It has the advantage of being readable and relatively unobtrusive. Since each source link must match up to a target, the odd variable ending in an underscore can be spared being marked up (although it should generate a "no such link target" warning). The only disadvantage is that phrase-links aren't possible without some obtrusive syntax. We could achieve phrase-links if we enclose the link text: 1. in double quotes:: "like this"_ 2. in brackets:: [like this]_ 3. or in backquotes:: `like this`_ Each gives us somewhat obtrusive markup, but that is unavoidable. The bracketed syntax (#2) is reminiscent of links on many web pages (intuitive), although it is somewhat obtrusive. Alternative #3 is much less obtrusive, and is consistent with interpreted text: the trailing underscore indicates the interpretation of the phrase, as a hyperlink. #3 also disambiguates hyperlinks from footnote references. Alternative #3 wins. The same trailing underscore markup can also be used for footnote and citation references, removing the problem with ordinary bracketed text and Python lists:: Please refer to the fine manual [GVR2000]_. .. [GVR2000] Python Documentation, van Rossum, Drake, et al., http://www.python.org/doc/ The two-dots-and-a-space syntax was generalized by Setext for comments, which are removed from the (visible) processed output. reStructuredText uses this syntax for comments, footnotes, and link target, collectively termed "explicit markup". For link targets, in order to eliminate ambiguity with comments and footnotes, reStructuredText specifies that a colon always follow the link target word/phrase. The colon denotes 'maps to'. There is no reason to restrict target links to the end of the document; they could just as easily be interspersed. Internal hyperlinks (links from one point to another within a single document) can be expressed by a source link as before, and a target link with a colon but no URI. In effect, these targets 'map to' the element immediately following. As an added bonus, we now have a perfect candidate for reStructuredText directives, a simple extension mechanism: explicit markup containing a single word followed by two colons and whitespace. The interpretation of subsequent data on the directive line or following is directive-dependent. To summarize:: .. This is a comment. .. The line below is an example of a directive. .. version:: 1 This is a footnote [1]_. This internal hyperlink will take us to the footnotes_ area below. Here is a one-word_ external hyperlink. Here is `a hyperlink phrase`_. .. _footnotes: .. [1] Footnote text goes here. .. external hyperlink target mappings: .. _one-word: http://www.123.xyz .. _a hyperlink phrase: http://www.123.xyz The presence or absence of a colon after the target link differentiates an indirect hyperlink from a footnote, respectively. A footnote requires brackets. Backquotes around a target link word or phrase are required if the phrase contains a colon, optional otherwise. Below are examples using no markup, the two StructuredText hypertext styles, and the reStructuredText hypertext style. Each example contains an indirect link, a direct link, a footnote/endnote, and bracketed text. In HTML, each example should evaluate to::

A URI, see [eggs2000] (in Bacon [Publisher]). Also see http://eggs.org.

[eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam"

1. No markup:: A URI http://spam.org, see eggs2000 (in Bacon [Publisher]). Also see http://eggs.org. eggs2000 "Spam, Spam, Spam, Eggs, Bacon, and Spam" 2. StructuredText absolute/relative URI syntax ("text":http://www.url.org):: A "URI":http://spam.org, see [eggs2000] (in Bacon [Publisher]). Also see "http://eggs.org":http://eggs.org. .. [eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam" Note that StructuredText does not recognize standalone URIs, forcing doubling up as shown in the second line of the example above. 3. StructuredText absolute-only URI syntax ("text", mailto:you@your.com):: A "URI", http://spam.org, see [eggs2000] (in Bacon [Publisher]). Also see "http://eggs.org", http://eggs.org. .. [eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam" 4. reStructuredText syntax:: 4. A URI_, see [eggs2000]_ (in Bacon [Publisher]). Also see http://eggs.org. .. _URI: http:/spam.org .. [eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam" The bracketed text '[Publisher]' may be problematic with StructuredText (syntax 2 & 3). reStructuredText's syntax (#4) is definitely the most readable. The text is separated from the link URI and the footnote, resulting in cleanly readable text. .. _StructuredText: http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage .. _Setext: http://docutils.sourceforge.net/mirror/setext.html .. _reStructuredText: http://docutils.sourceforge.net/rst.html .. _detailed description: http://homepage.ntlworld.com/tibsnjoan/docutils/STNG-format.html .. _STMinus: http://www.cis.upenn.edu/~edloper/pydoc/stminus.html .. _StructuredTextNG: http://dev.zope.org/Members/jim/StructuredTextWiki/StructuredTextNG .. _README: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/~checkout~/ python/python/dist/src/README .. _Emacs table mode: http://table.sourceforge.net/ .. _reStructuredText Markup Specification: ../../ref/rst/restructuredtext.html .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:


================================
 Docutils_ Internationalization
================================

:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Date: $Date: 2004/04/17 17:38:28 $
:Revision: $Revision: 1.11 $
:Copyright: This document has been placed in the public domain.

.. contents::

This document describes the internationalization facilities of the
Docutils_ project.  `Introduction to i18n`_ by Tomohiro KUBOTA is a
good general reference.  "Internationalization" is often abbreviated
as "i18n": "i" + 18 letters + "n".

.. Note::

   The i18n facilities of Docutils should be considered a "first
   draft".  They work so far, but improvements are welcome.
   Specifically, standard i18n facilities like "gettext" have yet to
   be explored.

Docutils is designed to work flexibly with text in multiple languages
(one language at a time).  Language-specific features are (or should
be [#]_) fully parameterized.  To enable a new language, two modules
have to be added to the project: one for Docutils itself (the
`Docutils Language Module`_) and one for the reStructuredText parser
(the `reStructuredText Language Module`_).

.. [#] If anything in Docutils is insufficiently parameterized, it
   should be considered a bug.  Please report bugs to the Docutils
   project bug tracker on SourceForge at
   http://sourceforge.net/tracker/?group_id=38414&atid=422030.

.. _Docutils: http://docutils.sourceforge.net/
.. _Introduction to i18n:
   http://www.debian.org/doc/manuals/intro-i18n/

Language Module Names
=====================

Language modules are named using a case-insensitive language
identifier as defined in `RFC 1766`_, converting hyphens to
underscores [#]_.  A typical language identifier consists of a
2-letter language code from `ISO 639`_ (3-letter codes can be used if
no 2-letter code exists; RFC 1766 is currently being revised to allow
3-letter codes).  The language identifier can have an optional subtag,
typically for variations based on country (from `ISO 3166`_ 2-letter
country codes).  If no language identifier is specified, the default
is "en" for English.  Examples of module names include ``en.py``,
``fr.py``, ``ja.py``, and ``pt_br.py``.

.. [#] Subtags are separated from primary tags by underscores instead
   of hyphens, to conform to Python naming rules.

.. _RFC 1766: http://www.faqs.org/rfcs/rfc1766.html
.. _ISO 639: http://lcweb.loc.gov/standards/iso639-2/englangn.html
.. _ISO 3166: http://www.iso.ch/iso/en/prods-services/iso3166ma/
   02iso-3166-code-lists/index.html

Python Code
===========

All Python code in Docutils will be ASCII-only.  In language modules,
Unicode-escapes will have to be used for non-ASCII characters.
Although it may be possible for some developers to store non-ASCII
characters directly in strings, it will cause problems for other
developers whose locales are set up differently.

`PEP 263`_ introduces source code encodings to Python modules,
implemented beginning in Python 2.3.  Until PEP 263 is fully
implemented as a well-established convention, proven robust in daily
use, and the tools (editors, CVS, email, etc.) recognize this
convention, Docutils shall remain conservative.

As mentioned in the note above, developers are invited to explore
"gettext" and other i18n technologies.

.. _PEP 263: http://www.python.org/peps/pep-0263.html

Docutils Language Module
========================

Modules in ``docutils/languages`` contain language mappings for
markup-independent language-specific features of Docutils.  To make a
new language module, just copy the ``en.py`` file, rename it with the
code for your language (see `Language Module Names`_ above), and
translate the terms as described below.

Each Docutils language module contains three module attributes:

``labels``
    This is a mapping of node class names to language-dependent
    boilerplate label text.  The label text is used by Writer
    components when they encounter document tree elements whose class
    names are the mapping keys.

    The entry values (*not* the keys) should be translated to the
    target language.

``bibliographic_fields``
    This is a mapping of language-dependent field names (converted to
    lower case) to canonical field names (keys of
    ``DocInfo.biblio_notes`` in ``docutils.transforms.frontmatter``).
    It is used when transforming bibliographic fields.

    The keys should be translated to the target language.

``author_separators``
    This is a list of strings used to parse the 'Authors'
    bibliographic field.  They separate individual authors' names, and
    are tried in order (i.e., earlier items take priority, and the
    first item that matches wins).  The English-language module
    defines them as ``[';', ',']``; semi-colons can be used to
    separate names like "Arthur Pewtie, Esq.".

    Most languages won't have to "translate" this list.

reStructuredText Language Module
================================

Modules in ``docutils/parsers/rst/languages`` contain language
mappings for language-specific features of the reStructuredText
parser.  To make a new language module, just copy the ``en.py`` file,
rename it with the code for your language (see `Language Module
Names`_ above), and translate the terms as described below.

Each reStructuredText language module contains just one module
attribute:

``directives``
    This is a mapping from language-dependent directive names to
    canonical directive names.  The canonical directive names are
    registered in ``docutils/parsers/rst/directives/__init__.py``, in
    ``_directive_registry``.

    The keys should be translated to the target language.  Synonyms
    (multiple keys with the same values) are allowed; this is useful
    for abbreviations.

``roles``
    This is a mapping language-dependent role names to canonical role
    names for interpreted text.  The canonical directive names are
    registered in ``docutils/parsers/rst/states.py``, in
    ``Inliner._interpreted_roles`` (this may change).

    The keys should be translated to the target language.  Synonyms
    (multiple keys with the same values) are allowed; this is useful
    for abbreviations.

Testing the Language Modules
============================

Whenever a new language module is added or an existing one modified,
the unit tests should be run.  The test modules can be found in the
docutils/test directory from CVS_ or from the `latest CVS snapshot`_.

The ``test_language.py`` module can be run as a script.  With no
arguments, it will test all language modules.  With one or more
language codes, it will test just those languages.  For example::

    $ python test_language.py en
    ..
    ----------------------------------------
    Ran 2 tests in 0.095s

    OK

Use the "alltests.py" script to run all test modules, exhaustively
testing the parser and other parts of the Docutils system.

.. _CVS: http://sourceforge.net/cvs/?group_id=38414
.. _latest CVS snapshot: http://docutils.sf.net/docutils-snapshot.tgz


======================================
 Creating reStructuredText Directives
======================================

:Authors: Dethe Elza, David Goodger
:Contact: delza@enfoldingsystems.com
:Date: $Date: 2004/06/05 02:59:13 $
:Revision: $Revision: 1.11 $
:Copyright: This document has been placed in the public domain.

Directives are the primary extension mechanism of reStructuredText.
This document aims to make the creation of new directives as easy and
understandable as possible.  There are only a couple of
reStructuredText-specific features the developer needs to know to
create a basic directive.

The syntax of directives is detailed in the `reStructuredText Markup
Specification`_, and standard directives are described in
`reStructuredText Directives`_.

Directives are a reStructuredText markup/parser concept.  There is no
"directive" element, no single element that corresponds exactly to the
concept of directives.  Instead, choose the most appropriate elements
from the existing Docutils elements.  Directives build structures
using the existing building blocks.  See `The Docutils Document Tree`_
and the ``docutils.nodes`` module for more about the building blocks
of Docutils documents.

.. _reStructuredText Markup Specification:
   ../ref/rst/restructuredtext.html#directives
.. _reStructuredText Directives: ../ref/rst/directives.html
.. _The Docutils Document Tree: ../ref/doctree.html

.. contents:: Table of Contents

Define the Directive Function
=============================

The directive function does any processing that the directive
requires.  This may require the use of other parts of the
reStructuredText parser.  This is where the directive actually *does*
something.

The directive implementation itself is a callback function whose
signature is as follows::

    def directive_fn(name, arguments, options, content, lineno,
                     content_offset, block_text, state, state_machine):
        code...

    # Set function attributes:
    directive_fn.arguments = ...
    directive_fn.options = ...
    direcitve_fn.content = ...

Function attributes are described below (see `Specify Directive
Arguments, Options, and Content`_).  The directive function parameters
are as follows:

- ``name`` is the directive type or name.

- ``arguments`` is a list of positional arguments, as specified in the
  ``arguments`` function attribute.

- ``options`` is a dictionary mapping option names to values.  The
  options handled by a directive function are specified in the
  ``options`` function attribute.

- ``content`` is a list of strings, the directive content.  Use the
  ``content`` function attribute to allow directive content.

- ``lineno`` is the line number of the first line of the directive.

- ``content_offset`` is the line offset of the first line of the
  content from the beginning of the current input.  Used when
  initiating a nested parse.

- ``block_text`` is a string containing the entire directive.  Include
  it as the content of a literal block in a system message if there is
  a problem.

- ``state`` is the state which called the directive function.

- ``state_machine`` is the state machine which controls the state
  which called the directive function.

Directive functions return a list of nodes which will be inserted into
the document tree at the point where the directive was encountered.
This can be an empty list if there is nothing to insert.  For ordinary
directives, the list must contain body elements or structural
elements.  Some directives are intended specifically for substitution
definitions, and must return a list of ``Text`` nodes and/or inline
elements (suitable for inline insertion, in place of the substitution
reference).  Such directives must verify substitution definition
context, typically using code like this::

    if not isinstance(state, states.SubstitutionDef):
        error = state_machine.reporter.error(
            'Invalid context: the "%s" directive can only be used '
            'within a substitution definition.' % (name),
            nodes.literal_block(block_text, block_text), line=lineno)
        return [error]

Specify Directive Arguments, Options, and Content
=================================================

Function attributes are interpreted by the directive parser (from the
``docutils.parsers.rst.states.Body.run_directive()`` method).  If
unspecified, directive function attributes are assumed to have the
value ``None``.  Three directive function attributes are recognized:

- ``arguments``: A 3-tuple specifying the expected positional
  arguments, or ``None`` if the directive has no arguments.  The 3
  items in the tuple are:

  1. The number of required arguments.
  2. The number of optional arguments.
  3. A boolean, indicating if the final argument may contain whitespace.

  Arguments are normally single whitespace-separated words.  The final
  argument may contain whitespace when indicated by the value 1 (True)
  for the third item in the argument spec tuple.  In this case, the
  final argument in the ``arguments`` parameter to the directive
  function will contain spaces and/or newlines, preserved from the
  input text.

  If the form of the arguments is more complex, specify only one
  argument (either required or optional) and indicate that final
  whitespace is OK (1/True); the client code must do any
  context-sensitive parsing.

- ``options``: The option specification.  ``None`` or an empty dict
  implies no options to parse.

  An option specification must be defined detailing the options
  available to the directive.  An option spec is a mapping of option
  name to conversion function; conversion functions are applied to
  each option value to check validity and convert them to the expected
  type.  Python's built-in conversion functions are often usable for
  this, such as ``int``, ``float``, and ``bool`` (included in Python
  from version 2.2.1).  Other useful conversion functions are included
  in the ``docutils.parsers.rst.directives`` package (in the
  ``__init__.py`` module):

  - ``flag``: For options with no option arguments.  Checks for an
    argument (raises ``ValueError`` if found), returns ``None`` for
    valid flag options.

  - ``unchanged_required``: Returns the text argument, unchanged.
    Raises ``ValueError`` if no argument is found.

  - ``unchanged``: Returns the text argument, unchanged.  Returns an
    empty string ("") if no argument is found.

  - ``path``: Returns the path argument unwrapped (with newlines
    removed).  Raises ``ValueError`` if no argument is found or if the
    path contains internal whitespace.

  - ``nonnegative_int``: Checks for a nonnegative integer argument,
    and raises ``ValueError`` if not.

  - ``class_option``: Converts the argument into an ID-compatible
    string and returns it.  Raises ``ValueError`` if no argument is
    found.

  A further utility function, ``choice``, is supplied to enable
  options whose argument must be a member of a finite set of possible
  values.  A custom conversion function must be written to use it.
  For example::

      from docutils.parsers.rst import directives

      def yesno(argument):
          return directives.choice(argument, ('yes', 'no'))

  For example, here is an option spec for a directive which allows two
  options, "name" and "value", each with an option argument::

      directive_fn.options = {'name': unchanged, 'value': int}

- ``content``: A boolean; true if content is allowed.  Directive
  functions must handle the case where content is required but not
  present in the input text (an empty content list will be supplied).

The final step of the ``run_directive()`` method is to call the
directive function itself.

Register the Directive
======================

If the directive is a general-use addition to the Docutils core, it
must be registered with the parser and language mappings added:

1. Register the new directive using its canonical name in
   ``docutils/parsers/rst/directives/__init__.py``, in the
   ``_directive_registry`` dictionary.  This allows the
   reStructuredText parser to find and use the directive.

2. Add an entry to the ``directives`` dictionary in
   ``docutils/parsers/rst/languages/en.py`` for the directive, mapping
   the English name to the canonical name (both lowercase).  Usually
   the English name and the canonical name are the same.

3. Update all the other language modules as well.  For languages in
   which you are proficient, please add translations.  For other
   languages, add the English directive name plus "(translation
   required)".

If the directive is application-specific, use the
``register_directive`` function::

    from docutils.parsers.rst import directives
    directives.register_directive(directive_name, directive_function)

Examples
========

For the most direct and accurate information, "Use the Source, Luke!".
All standard directives are documented in `reStructuredText
Directives`_, and the source code implementing them is located in the
``docutils/parsers/rst/directives`` package.  The ``__init__.py``
module contains a mapping of directive name to module & function name.
Several representative directives are described below.

Admonitions
-----------

Admonition directives, such as "note" and "caution", are quite simple.
They have no directive arguments or options.  Admonition directive
content is interpreted as ordinary reStructuredText.  The directive
function simply hands off control to a generic directive function::

   def note(*args):
       return admonition(nodes.note, *args)

   attention.content = 1

Note that the only thing distinguishing the various admonition
directives is the element (node class) generated.  In the code above,
the node class is passed as the first argument to the generic
directive function (early version), where the actual processing takes
place::

   def admonition(node_class, name, arguments, options, content, lineno,
                  content_offset, block_text, state, state_machine):
       text = '\n'.join(content)
       admonition_node = node_class(text)
       if text:
           state.nested_parse(content, content_offset, admonition_node)
           return [admonition_node]
       else:
           warning = state_machine.reporter.warning(
               'The "%s" admonition is empty; content required.'
               % (name), '',
               nodes.literal_block(block_text, block_text), line=lineno)
           return [warning]

Three things are noteworthy in the function above:

1. The ``admonition_node = node_class(text)`` line creates the wrapper
   element, using the class passed in from the initial (stub)
   directive function.

2. The call to ``state.nested_parse()`` is what does the actual
   processing.  It parses the directive content and adds any generated
   elements as child elements of ``admonition_node``.

3. If there was no directive content, a warning is generated and
   returned.  The call to ``state_machine.reporter.warning()``
   includes a literal block containing the entire directive text
   (``block_text``) and the line (``lineno``) of the top of the
   directive.

"image"
-------

The "image" directive is used to insert a picture into a document.
This directive has one argument, the path to the image file, and
supports several options.  There is no directive content.  Here's an
early version of the image directive function::

    def image(name, arguments, options, content, lineno,
              content_offset, block_text, state, state_machine):
        reference = ''.join(arguments[0].split('\n'))
        if reference.find(' ') != -1:
            error = state_machine.reporter.error(
                  'Image URI contains whitespace.', '',
                  nodes.literal_block(block_text, block_text),
                  line=lineno)
            return [error]
        options['uri'] = reference
        image_node = nodes.image(block_text, **options)
        return [image_node]

    image.arguments = (1, 0, 1)
    image.options = {'alt': directives.unchanged,
                     'height': directives.nonnegative_int,
                     'width': directives.nonnegative_int,
                     'scale': directives.nonnegative_int,
                     'align': align}

Several things are noteworthy in the code above:

1. The "image" directive requires a single argument, which is allowed
   to contain whitespace (see the argument spec above,
   ``image.arguments = (1, 0, 1)``).  This is to allow for long URLs
   which may span multiple lines.  The first line of the ``image``
   function joins the URL, discarding any embedded newlines.  Then the
   result is checked for embedded spaces, which are *not* allowed.

2. The reference is added to the ``options`` dictionary under the
   "uri" key; this becomes an attribute of the ``nodes.image`` element
   object.  Any other attributes have already been set explicitly in
   the source text.

3. The "align" option depends on the following definitions (which
   actually occur earlier in the source code)::

       align_values = ('top', 'middle', 'bottom', 'left', 'center',
                       'right')

       def align(argument):
           return directives.choice(argument, align_values)

"contents"
----------

The "contents" directive is used to insert an auto-generated table of
contents (TOC) into a document.  It takes one optional argument, a
title for the TOC.  If no title is specified, a default title is used
instead.  The directive also handles several options.  Here's an early
version of the code::

    def contents(name, arguments, options, content, lineno,
                 content_offset, block_text, state, state_machine):
        """Table of contents."""
        if arguments:
            title_text = arguments[0]
            text_nodes, messages = state.inline_text(title_text, lineno)
            title = nodes.title(title_text, '', *text_nodes)
        else:
            messages = []
            title = None
        pending = nodes.pending(parts.Contents, {'title': title},
                                block_text)
        pending.details.update(options)
        state_machine.document.note_pending(pending)
        return [pending] + messages

    contents.arguments = (0, 1, 1)
    contents.options = {'depth': directives.nonnegative_int,
                        'local': directives.flag,
                        'backlinks': backlinks}

Aspects of note include:

1. The ``contents.arguments = (0, 1, 1)`` function attribute specifies
   a single, *optional* argument.  If no argument is present, the
   ``arguments`` parameter to the directive function will be an empty
   list.

2. If an argument *is* present, its text is passed to
   ``state.inline_text()`` for parsing.  Titles may contain inline
   markup, such as emphasis or inline literals.

3. The table of contents is not generated right away.  Typically, a
   TOC is placed near the beginning of a document, and is a summary or
   outline of the section structure of the document.  The entire
   document must already be processed before a summary can be made.
   This directive leaves a ``nodes.pending`` placeholder element in
   the document tree, marking the position of the TOC and including a
   ``details`` internal attribute containing all the directive
   options, effectively communicating the options forward.  The actual
   table of contents processing is performed by a transform,
   ``docutils.transforms.parts.Contents``, after the rest of the
   document has been parsed.


==================================================
 Creating reStructuredText Interpreted Text Roles
==================================================

:Authors: David Goodger
:Contact: goodger@python.org
:Date: $Date: 2004/06/03 19:16:08 $
:Revision: $Revision: 1.5 $
:Copyright: This document has been placed in the public domain.

Interpreted text roles are an extension mechanism for inline markup in
reStructuredText.  This document aims to make the creation of new
roles as easy and understandable as possible.

Standard roles are described in `reStructuredText Interpreted Text
Roles`_.  See the `Interpreted Text`_ section in the `reStructuredText
Markup Specification`_ for syntax details.

.. _reStructuredText Interpreted Text Roles: ../ref/rst/roles.html
.. _Interpreted Text:
   ../ref/rst/restructuredtext.html#interpreted-text
.. _reStructuredText Markup Specification:
   ../ref/rst/restructuredtext.html

.. contents::

Define the Role Function
========================

The role function creates and returns inline elements (nodes) and does
any additional processing required.  Its signature is as follows::

    def role_fn(name, rawtext, text, lineno, inliner,
                options={}, content=[]):
        code...

    # Set function attributes for customization:
    role_fn.options = ...
    role_fn.content = ...

Function attributes are described below (see `Specify Role Function
Options and Content`_).  The role function parameters are as follows:

* ``name``: The local name of the interpreted role, the role name
  actually used in the document.

* ``rawtext``: A string containing the enitre interpreted text input,
  including the role and markup.  Return it as a ``problematic`` node
  linked to a system message if a problem is encountered.

* ``text``: The interpreted text content.

* ``lineno``: The line number where the interpreted text begins.

* ``inliner``: The ``docutils.parsers.rst.states.Inliner`` object that
  called role_fn.  It contains the several attributes useful for error
  reporting and document tree access.

* ``options``: A dictionary of directive options for customization
  (from the `"role" directive`_), to be interpreted by the role
  function.  Used for additional attributes for the generated elements
  and other functionality.

* ``content``: A list of strings, the directive content for
  customization (from the `"role" directive`_).  To be interpreted by
  the role function.

Role functions return a tuple of two values:

* A list of nodes which will be inserted into the document tree at the
  point where the interpreted role was encountered (can be an empty
  list).

* A list of system messages, which will be inserted into the document tree
  immediately after the end of the current block (can also be empty).

Specify Role Function Options and Content
=========================================

Function attributes are for customization, and are interpreted by the
`"role" directive`_.  If unspecified, role function attributes are
assumed to have the value ``None``.  Two function attributes are
recognized:

- ``options``: The option specification.  All role functions
  implicitly support the "class" option, unless disabled with an
  explicit ``{'class': None}``.

  An option specification must be defined detailing the options
  available to the "role" directive.  An option spec is a mapping of
  option name to conversion function; conversion functions are applied
  to each option value to check validity and convert them to the
  expected type.  Python's built-in conversion functions are often
  usable for this, such as ``int``, ``float``, and ``bool`` (included
  in Python from version 2.2.1).  Other useful conversion functions
  are included in the ``docutils.parsers.rst.directives`` package.
  For further details, see `Creating reStructuredText Directives`_.

- ``content``: A boolean; true if "role" directive content is allowed.
  Role functions must handle the case where content is required but
  not supplied (an empty content list will be supplied).

  As of this writing, no roles accept directive content.

Note that unlike directives, the "arguments" function attribute is not
supported for role customization.  Directive arguments are handled by
the "role" directive itself.

.. _"role" directive: ../ref/rst/directives.html#role
.. _Creating reStructuredText Directives:
   rst-directives.html#specify-directive-arguments-options-and-content

Register the Role
=================

If the role is a general-use addition to the Docutils core, it must be
registered with the parser and language mappings added:

1. Register the new role using the canonical name::

       from docutils.parsers.rst import roles
       roles.register_canonical_role(name, role_function)

   This code is normally placed immediately after the definition of
   the role funtion.

2. Add an entry to the ``roles`` dictionary in
   ``docutils/parsers/rst/languages/en.py`` for the role, mapping the
   English name to the canonical name (both lowercase).  Usually the
   English name and the canonical name are the same.  Abbreviations
   and other aliases may also be added here.

3. Update all the other language modules as well.  For languages in
   which you are proficient, please add translations.  For other
   languages, add the English role name plus "(translation required)".

If the role is application-specific, use the ``register_local_role``
function::

    from docutils.parsers.rst import roles
    roles.register_local_role(name, role_function)

Examples
========

For the most direct and accurate information, "Use the Source, Luke!".
All standard roles are documented in `reStructuredText Interpreted
Text Roles`_, and the source code implementing them is located in the
``docutils/parsers/rst/roles.py`` module.  Several representative
roles are described below.

Generic Roles
-------------

Many roles simply wrap a given element around the text.  There's a
special helper function, ``register_generic_role``, which generates a
role function from the canonical role name and node class::

    register_generic_role('emphasis', nodes.emphasis)

For the implementation of ``register_generic_role``, see the
``docutils.parsers.rst.roles`` module.

RFC Reference Role
------------------

This role allows easy references to RFCs_ (Request For Comments
documents) by automatically providing the base URL,
http://www.faqs.org/rfcs/, and appending the RFC document itself
(rfcXXXX.html, where XXXX is the RFC number).  For example::

    See :RFC:`2822` for information about email headers.

This is equivalent to::

    See `RFC 2822`__ for information about email headers.

    __ http://www.faqs.org/rfcs/rfc2822.html

Here is the implementation of the role::

    def rfc_reference_role(role, rawtext, text, lineno, inliner,
                           options={}, content=[]):
        try:
            rfcnum = int(text)
            if rfcnum <= 0:
                raise ValueError
        except ValueError:
            msg = inliner.reporter.error(
                'RFC number must be a number greater than or equal to 1; '
                '"%s" is invalid.' % text, line=lineno)
            prb = inliner.problematic(rawtext, rawtext, msg)
            return [prb], [msg]
        # Base URL mainly used by inliner.rfc_reference, so this is correct:
        ref = inliner.rfc_url % rfcnum
        node = nodes.reference(rawtext, 'RFC ' + text, refuri=ref, **options)
        return [node], []

    register_canonical_role('rfc-reference', rfc_reference_role)

Noteworthy in the code above are:

1. The interpreted text itself should contain the RFC number.  The
   ``try`` clause verifies by converting it to an integer.  If the
   conversion fails, the ``except`` clause is executed: a system
   message is generated, the entire interpreted text construct (in
   ``rawtext``) is wrapped in a ``problematic`` node (linked to the
   system message), and the two are returned.

2. The RFC reference itself is constructed from a stock URI, set as
   the "refuri" attribute of a "reference" element.

3. The ``options`` function parameter, a dictionary, may contain a
   "class" customization attribute; it is passed through to the
   "reference" element node constructor.

.. _RFCs: http://foldoc.doc.ic.ac.uk/foldoc/foldoc.cgi?query=rfc&action=Search&sourceid=Mozilla-search


PEP: 256
Title: Docstring Processing System Framework
Version: $Revision: 1.9 $
Last-Modified: $Date: 2004/06/05 02:55:11 $
Author: David Goodger <goodger@users.sourceforge.net>
Discussions-To: <doc-sig@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 01-Jun-2001
Post-History: 13-Jun-2001

Abstract
========

Python lends itself to inline documentation.  With its built-in
docstring syntax, a limited form of `Literate Programming`_ is easy to
do in Python.  However, there are no satisfactory standard tools for
extracting and processing Python docstrings.  The lack of a standard
toolset is a significant gap in Python's infrastructure; this PEP aims
to fill the gap.

The issues surrounding docstring processing have been contentious and
difficult to resolve.  This PEP proposes a generic Docstring
Processing System (DPS) framework, which separates out the components
(program and conceptual), enabling the resolution of individual issues
either through consensus (one solution) or through divergence (many).
It promotes standard interfaces which will allow a variety of plug-in
components (input context readers, markup parsers, and output format
writers) to be used.

The concepts of a DPS framework are presented independently of
implementation details.

Road Map to the Docstring PEPs
==============================

There are many aspects to docstring processing.  The "Docstring PEPs"
have broken up the issues in order to deal with each of them in
isolation, or as close as possible.  The individual aspects and
associated PEPs are as follows:

* Docstring syntax.  PEP 287, "reStructuredText Docstring Format"
  [#PEP-287]_, proposes a syntax for Python docstrings, PEPs, and
  other uses.

* Docstring semantics consist of at least two aspects:

  - Conventions: the high-level structure of docstrings.  Dealt with
    in PEP 257, "Docstring Conventions" [#PEP-257]_.

  - Methodology: rules for the informational content of docstrings.
    Not addressed.

* Processing mechanisms.  This PEP (PEP 256) outlines the high-level
  issues and specification of an abstract docstring processing system
  (DPS).  PEP 258, "Docutils Design Specification" [#PEP-258]_, is an
  overview of the design and implementation of one DPS under
  development.

* Output styles: developers want the documentation generated from
  their source code to look good, and there are many different ideas
  about what that means.  PEP 258 touches on "Stylist Transforms".
  This aspect of docstring processing has yet to be fully explored.

By separating out the issues, we can form consensus more easily
(smaller fights ;-), and accept divergence more readily.

Rationale
=========

There are standard inline documentation systems for some other
languages.  For example, Perl has POD_ ("Plain Old Documentation") and
Java has Javadoc_, but neither of these mesh with the Pythonic way.
POD syntax is very explicit, but takes after Perl in terms of
readability.  Javadoc is HTML-centric; except for "``@field``" tags,
raw HTML is used for markup.  There are also general tools such as
Autoduck_ and Web_ (Tangle & Weave), useful for multiple languages.

There have been many attempts to write auto-documentation systems
for Python (not an exhaustive list):

- Marc-Andre Lemburg's doc.py_

- Daniel Larsson's pythondoc_ & gendoc_

- Doug Hellmann's HappyDoc_

- Laurence Tratt's Crystal (no longer available on the web)

- Ka-Ping Yee's pydoc_ (pydoc.py is now part of the Python standard
  library; see below)

- Tony Ibbs' docutils_ (Tony has donated this name to the `Docutils
  project`_)

- Edward Loper's STminus_ formalization and related efforts

These systems, each with different goals, have had varying degrees of
success.  A problem with many of the above systems was over-ambition
combined with inflexibility.  They provided a self-contained set of
components: a docstring extraction system, a markup parser, an
internal processing system and one or more output format writers with
a fixed style.  Inevitably, one or more aspects of each system had
serious shortcomings, and they were not easily extended or modified,
preventing them from being adopted as standard tools.

It has become clear (to this author, at least) that the "all or
nothing" approach cannot succeed, since no monolithic self-contained
system could possibly be agreed upon by all interested parties.  A
modular component approach designed for extension, where components
may be multiply implemented, may be the only chance for success.
Standard inter-component APIs will make the DPS components
comprehensible without requiring detailed knowledge of the whole,
lowering the barrier for contributions, and ultimately resulting in a
rich and varied system.

Each of the components of a docstring processing system should be
developed independently.  A "best of breed" system should be chosen,
either merged from existing systems, and/or developed anew.  This
system should be included in Python's standard library.

PyDoc & Other Existing Systems
------------------------------

PyDoc became part of the Python standard library as of release 2.1.
It extracts and displays docstrings from within the Python interactive
interpreter, from the shell command line, and from a GUI window into a
web browser (HTML).  Although a very useful tool, PyDoc has several
deficiencies, including:

- In the case of the GUI/HTML, except for some heuristic hyperlinking
  of identifier names, no formatting of the docstrings is done.  They
  are presented within ``<p><small><tt>`` tags to avoid unwanted line
  wrapping.  Unfortunately, the result is not attractive.

- PyDoc extracts docstrings and structural information (class
  identifiers, method signatures, etc.) from imported module objects.
  There are security issues involved with importing untrusted code.
  Also, information from the source is lost when importing, such as
  comments, "additional docstrings" (string literals in non-docstring
  contexts; see PEP 258 [#PEP-258]_), and the order of definitions.

The functionality proposed in this PEP could be added to or used by
PyDoc when serving HTML pages.  The proposed docstring processing
system's functionality is much more than PyDoc needs in its current
form.  Either an independent tool will be developed (which PyDoc may
or may not use), or PyDoc could be expanded to encompass this
functionality and *become* the docstring processing system (or one
such system).  That decision is beyond the scope of this PEP.

Similarly for other existing docstring processing systems, their
authors may or may not choose compatibility with this framework.
However, if this framework is accepted and adopted as the Python
standard, compatibility will become an important consideration in
these systems' future.

Specification
=============

The docstring processing system framework is broken up as follows:

1. Docstring conventions.  Documents issues such as:

   - What should be documented where.

   - First line is a one-line synopsis.

   PEP 257 [#PEP-257]_ documents some of these issues.

2. Docstring processing system design specification.  Documents
   issues such as:

   - High-level spec: what a DPS does.

   - Command-line interface for executable script.

   - System Python API.

   - Docstring extraction rules.

   - Readers, which encapsulate the input context.

   - Parsers.

   - Document tree: the intermediate internal data structure.  The
     output of the Parser and Reader, and the input to the Writer all
     share the same data structure.

   - Transforms, which modify the document tree.

   - Writers for output formats.

   - Distributors, which handle output management (one file, many
     files, or objects in memory).

   These issues are applicable to any docstring processing system
   implementation.  PEP 258 [#PEP-258]_ documents these issues.

3. Docstring processing system implementation.

4. Input markup specifications: docstring syntax.  PEP 287 [#PEP-287]_
   proposes a standard syntax.

5. Input parser implementations.

6. Input context readers ("modes": Python source code, PEP, standalone
   text file, email, etc.) and implementations.

7. Stylists: certain input context readers may have associated
   stylists which allow for a variety of output document styles.

8. Output formats (HTML, XML, TeX, DocBook, info, etc.) and writer
   implementations.

Components 1, 2/3/5, and 4 are the subject of individual companion
PEPs.  If there is another implementation of the framework or
syntax/parser, additional PEPs may be required.  Multiple
implementations of each of components 6 and 7 will be required; the
PEP mechanism may be overkill for these components.

Project Web Site
================

A SourceForge project has been set up for this work at
http://docutils.sourceforge.net/.

References and Footnotes
========================

.. [#PEP-287] PEP 287, reStructuredText Docstring Format, Goodger
   (http://www.python.org/peps/pep-0287.html)

.. [#PEP-257] PEP 257, Docstring Conventions, Goodger, Van Rossum
   (http://www.python.org/peps/pep-0257.html)

.. [#PEP-258] PEP 258, Docutils Design Specification, Goodger
   (http://www.python.org/peps/pep-0258.html)

.. _Literate Programming: http://www.literateprogramming.com/

.. _POD: http://www.perldoc.com/perl5.6/pod/perlpod.html

.. _Javadoc: http://java.sun.com/j2se/javadoc/

.. _Autoduck:
   http://www.helpmaster.com/hlp-developmentaids-autoduck.htm

.. _Web: http://www-cs-faculty.stanford.edu/~knuth/cweb.html

.. _doc.py:
   http://www.egenix.com/files/python/SoftwareDescriptions.html#doc.py

.. _pythondoc:
.. _gendoc: http://starship.python.net/crew/danilo/pythondoc/

.. _HappyDoc: http://happydoc.sourceforge.net/

.. _pydoc: http://www.python.org/doc/current/lib/module-pydoc.html

.. _docutils: http://www.tibsnjoan.co.uk/docutils.html

.. _Docutils project: http://docutils.sourceforge.net/

.. _STMinus: http://www.cis.upenn.edu/~edloper/pydoc/

.. _Python Doc-SIG: http://www.python.org/sigs/doc-sig/

Copyright
=========

This document has been placed in the public domain.

Acknowledgements
================

This document borrows ideas from the archives of the `Python
Doc-SIG`_.  Thanks to all members past & present.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


PEP: 257
Title: Docstring Conventions
Version: $Revision: 1.7 $
Last-Modified: $Date: 2002/11/30 01:59:35 $
Author: David Goodger <goodger@users.sourceforge.net>,
        Guido van Rossum <guido@python.org>
Discussions-To: doc-sig@python.org
Status: Active
Type: Informational
Content-Type: text/x-rst
Created: 29-May-2001
Post-History: 13-Jun-2001

Abstract
========

This PEP documents the semantics and conventions associated with
Python docstrings.

Rationale
=========

The aim of this PEP is to standardize the high-level structure of
docstrings: what they should contain, and how to say it (without
touching on any markup syntax within docstrings).  The PEP contains
conventions, not laws or syntax.

    "A universal convention supplies all of maintainability, clarity,
    consistency, and a foundation for good programming habits too.
    What it doesn't do is insist that you follow it against your will.
    That's Python!"

    -- Tim Peters on comp.lang.python, 2001-06-16

If you violate these conventions, the worst you'll get is some dirty
looks.  But some software (such as the Docutils_ docstring processing
system [1]_ [2]_) will be aware of the conventions, so following them
will get you the best results.

Specification
=============

What is a Docstring?
--------------------

A docstring is a string literal that occurs as the first statement in
a module, function, class, or method definition.  Such a docstring
becomes the ``__doc__`` special attribute of that object.

All modules should normally have docstrings, and all functions and
classes exported by a module should also have docstrings.  Public
methods (including the ``__init__`` constructor) should also have
docstrings.  A package may be documented in the module docstring of
the ``__init__.py`` file in the package directory.

String literals occurring elsewhere in Python code may also act as
documentation.  They are not recognized by the Python bytecode
compiler and are not accessible as runtime object attributes (i.e. not
assigned to ``__doc__``), but two types of extra docstrings may be
extracted by software tools:

1. String literals occurring immediately after a simple assignment at
   the top level of a module, class, or ``__init__`` method are called
   "attribute docstrings".

2. String literals occurring immediately after another docstring are
   called "additional docstrings".

Please see PEP 258, "Docutils Design Specification" [2]_, for a
detailed description of attribute and additional docstrings.

XXX Mention docstrings of 2.2 properties.

For consistency, always use ``"""triple double quotes"""`` around
docstrings.  Use ``r"""raw triple double quotes"""`` if you use any
backslashes in your docstrings.  For Unicode docstrings, use
``u"""Unicode triple-quoted strings"""``.

There are two forms of docstrings: one-liners and multi-line
docstrings.

One-line Docstrings
--------------------

One-liners are for really obvious cases.  They should really fit on
one line.  For example::

    def kos_root():
        """Return the pathname of the KOS root directory."""
        global _kos_root
        if _kos_root: return _kos_root
        ...

Notes:

- Triple quotes are used even though the string fits on one line.
  This makes it easy to later expand it.

- The closing quotes are on the same line as the opening quotes.  This
  looks better for one-liners.

- There's no blank line either before or after the docstring.

- The docstring is a phrase ending in a period.  It prescribes the
  function or method's effect as a command ("Do this", "Return that"),
  not as a description; e.g. don't write "Returns the pathname ...".

- The one-line docstring should NOT be a "signature" reiterating the
  function/method parameters (which can be obtained by introspection).
  Don't do::

      def function(a, b):
          """function(a, b) -> list"""

  This type of docstring is only appropriate for C functions (such as
  built-ins), where introspection is not possible.  However, the
  nature of the *return value* cannot be determined by introspection,
  so it should be mentioned.  The preferred form for such a docstring
  would be something like::

      def function(a, b):
          """Do X and return a list."""

  (Of course "Do X" should be replaced by a useful description!)

Multi-line Docstrings
----------------------

Multi-line docstrings consist of a summary line just like a one-line
docstring, followed by a blank line, followed by a more elaborate
description.  The summary line may be used by automatic indexing
tools; it is important that it fits on one line and is separated from
the rest of the docstring by a blank line.  The summary line may be on
the same line as the opening quotes or on the next line.  The entire
docstring is indented the same as the quotes at its first line (see
example below).

Insert a blank line before and after all docstrings (one-line or
multi-line) that document a class -- generally speaking, the class's
methods are separated from each other by a single blank line, and the
docstring needs to be offset from the first method by a blank line;
for symmetry, put a blank line between the class header and the
docstring.  Docstrings documenting functions or methods generally
don't have this requirement, unless the function or method's body is
written as a number of blank-line separated sections -- in this case,
treat the docstring as another section, and precede it with a blank
line.

The docstring of a script (a stand-alone program) should be usable as
its "usage" message, printed when the script is invoked with incorrect
or missing arguments (or perhaps with a "-h" option, for "help").
Such a docstring should document the script's function and command
line syntax, environment variables, and files.  Usage messages can be
fairly elaborate (several screens full) and should be sufficient for a
new user to use the command properly, as well as a complete quick
reference to all options and arguments for the sophisticated user.

The docstring for a module should generally list the classes,
exceptions and functions (and any other objects) that are exported by
the module, with a one-line summary of each.  (These summaries
generally give less detail than the summary line in the object's
docstring.)  The docstring for a package (i.e., the docstring of the
package's ``__init__.py`` module) should also list the modules and
subpackages exported by the package.

The docstring for a function or method should summarize its behavior
and document its arguments, return value(s), side effects, exceptions
raised, and restrictions on when it can be called (all if applicable).
Optional arguments should be indicated.  It should be documented
whether keyword arguments are part of the interface.

The docstring for a class should summarize its behavior and list the
public methods and instance variables.  If the class is intended to be
subclassed, and has an additional interface for subclasses, this
interface should be listed separately (in the docstring).  The class
constructor should be documented in the docstring for its ``__init__``
method.  Individual methods should be documented by their own
docstring.

If a class subclasses another class and its behavior is mostly
inherited from that class, its docstring should mention this and
summarize the differences.  Use the verb "override" to indicate that a
subclass method replaces a superclass method and does not call the
superclass method; use the verb "extend" to indicate that a subclass
method calls the superclass method (in addition to its own behavior).

*Do not* use the Emacs convention of mentioning the arguments of
functions or methods in upper case in running text.  Python is case
sensitive and the argument names can be used for keyword arguments, so
the docstring should document the correct argument names.  It is best
to list each argument on a separate line.  For example::

    def complex(real=0.0, imag=0.0):
        """Form a complex number.

        Keyword arguments:
        real -- the real part (default 0.0)
        imag -- the imaginary part (default 0.0)

        """
        if imag == 0.0 and real == 0.0: return complex_zero
        ...

The BDFL [3]_ recommends inserting a blank line between the last
paragraph in a multi-line docstring and its closing quotes, placing
the closing quotes on a line by themselves.  This way, Emacs'
``fill-paragraph`` command can be used on it.

Handling Docstring Indentation
------------------------------

Docstring processing tools will strip a uniform amount of indentation
from the second and further lines of the docstring, equal to the
minimum indentation of all non-blank lines after the first line.  Any
indentation in the first line of the docstring (i.e., up to the first
newline) is insignificant and removed.  Relative indentation of later
lines in the docstring is retained.  Blank lines should be removed
from the beginning and end of the docstring.

Since code is much more precise than words, here is an implementation
of the algorithm::

    def trim(docstring):
        if not docstring:
            return ''
        # Convert tabs to spaces (following the normal Python rules)
        # and split into a list of lines:
        lines = docstring.expandtabs().splitlines()
        # Determine minimum indentation (first line doesn't count):
        indent = sys.maxint
        for line in lines[1:]:
            stripped = line.lstrip()
            if stripped:
                indent = min(indent, len(line) - len(stripped))
        # Remove indentation (first line is special):
        trimmed = [lines[0].strip()]
        if indent < sys.maxint:
            for line in lines[1:]:
                trimmed.append(line[indent:].rstrip())
        # Strip off trailing and leading blank lines:
        while trimmed and not trimmed[-1]:
            trimmed.pop()
        while trimmed and not trimmed[0]:
            trimmed.pop(0)
        # Return a single string:
        return '\n'.join(trimmed)

The docstring in this example contains two newline characters and is
therefore 3 lines long.  The first and last lines are blank::

    def foo():
        """
        This is the second line of the docstring.
        """

To illustrate::

    >>> print repr(foo.__doc__)
    '\n    This is the second line of the docstring.\n    '
    >>> foo.__doc__.splitlines()
    ['', '    This is the second line of the docstring.', '    ']
    >>> trim(foo.__doc__)
    'This is the second line of the docstring.'

Once trimmed, these docstrings are equivalent::

    def foo():
        """A multi-line
        docstring.
        """

    def bar():
        """
        A multi-line
        docstring.
        """

References and Footnotes
========================

.. [1] PEP 256, Docstring Processing System Framework, Goodger
   (http://www.python.org/peps/pep-0256.html)

.. [2] PEP 258, Docutils Design Specification, Goodger
   (http://www.python.org/peps/pep-0258.html)

.. [3] Guido van Rossum, Python's creator and Benevolent Dictator For
   Life.

.. _Docutils: http://docutils.sourceforge.net/

.. _Python Style Guide:
   http://www.python.org/doc/essays/styleguide.html

.. _Doc-SIG: http://www.python.org/sigs/doc-sig/

Copyright
=========

This document has been placed in the public domain.

Acknowledgements
================

The "Specification" text comes mostly verbatim from the `Python Style
Guide`_ essay by Guido van Rossum.

This document borrows ideas from the archives of the Python Doc-SIG_.
Thanks to all members past and present.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   fill-column: 70
   sentence-end-double-space: t
   End:


PEP: 258
Title: Docutils Design Specification
Version: $Revision: 1.31 $
Last-Modified: $Date: 2004/06/04 21:31:18 $
Author: David Goodger <goodger@users.sourceforge.net>
Discussions-To: <doc-sig@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Requires: 256, 257
Created: 31-May-2001
Post-History: 13-Jun-2001

==========
 Abstract
==========

This PEP documents design issues and implementation details for
Docutils, a Python Docstring Processing System (DPS).  The rationale
and high-level concepts of a DPS are documented in PEP 256, "Docstring
Processing System Framework" [#PEP-256]_.  Also see PEP 256 for a
"Road Map to the Docstring PEPs".

Docutils is being designed modularly so that any of its components can
be replaced easily.  In addition, Docutils is not limited to the
processing of Python docstrings; it processes standalone documents as
well, in several contexts.

No changes to the core Python language are required by this PEP.  Its
deliverables consist of a package for the standard library and its
documentation.

===============
 Specification
===============

Docutils Project Model
======================

Project components and data flow::

                     +---------------------------+
                     |        Docutils:          |
                     | docutils.core.Publisher,  |
                     | docutils.core.publish_*() |
                     +---------------------------+
                      /            |            \
                     /             |             \
            1,3,5   /        6     |              \ 7
           +--------+       +-------------+       +--------+
           | READER | ----> | TRANSFORMER | ====> | WRITER |
           +--------+       +-------------+       +--------+
            /     \\                                  |
           /       \\                                 |
     2    /      4  \\                             8  |
    +-------+   +--------+                        +--------+
    | INPUT |   | PARSER |                        | OUTPUT |
    +-------+   +--------+                        +--------+

The numbers above each component indicate the path a document's data
takes.  Double-width lines between Reader & Parser and between
Transformer & Writer indicate that data sent along these paths should
be standard (pure & unextended) Docutils doc trees.  Single-width
lines signify that internal tree extensions or completely unrelated
representations are possible, but they must be supported at both ends.

Publisher
---------

The ``docutils.core`` module contains a "Publisher" facade class and
several convenience functions: "publish_cmdline()" (for command-line
front ends), "publish_file()" (for programmatic use with file-like
I/O), and "publish_string()" (for programmatic use with string I/O).
The Publisher class encapsulates the high-level logic of a Docutils
system.  The Publisher class has overall responsibility for
processing, controlled by the ``Publisher.publish()`` method:

1. Set up internal settings (may include config files & command-line
   options) and I/O objects.

2. Call the Reader object to read data from the source Input object
   and parse the data with the Parser object.  A document object is
   returned.

3. Set up and apply transforms via the Transformer object attached to
   the document.

4. Call the Writer object which translates the document to the final
   output format and writes the formatted data to the destination
   Output object.  Depending on the Output object, the output may be
   returned from the Writer, and then from the ``publish()`` method.

Calling the "publish" function (or instantiating a "Publisher" object)
with component names will result in default behavior.  For custom
behavior (customizing component settings), create custom component
objects first, and pass *them* to the Publisher or ``publish_*``
convenience functions.

Readers
-------

Readers understand the input context (where the data is coming from),
send the whole input or discrete "chunks" to the parser, and provide
the context to bind the chunks together back into a cohesive whole.

Each reader is a module or package exporting a "Reader" class with a
"read" method.  The base "Reader" class can be found in the
``docutils/readers/__init__.py`` module.

Most Readers will have to be told what parser to use.  So far (see the
list of examples below), only the Python Source Reader ("PySource";
still incomplete) will be able to determine the parser on its own.

Responsibilities:

* Get input text from the source I/O.

* Pass the input text to the parser, along with a fresh `document
  tree`_ root.

Examples:

* Standalone (Raw/Plain): Just read a text file and process it.
  The reader needs to be told which parser to use.

  The "Standalone Reader" has been implemented in module
  ``docutils.readers.standalone``.

* Python Source: See `Python Source Reader`_ below.  This Reader is
  currently in development in the Docutils sandbox.

* Email: RFC-822 headers, quoted excerpts, signatures, MIME parts.

* PEP: RFC-822 headers, "PEP xxxx" and "RFC xxxx" conversion to URIs.
  The "PEP Reader" has been implemented in module
  ``docutils.readers.pep``; see PEP 287 and PEP 12.

* Wiki: Global reference lookups of "wiki links" incorporated into
  transforms.  (CamelCase only or unrestricted?)  Lazy
  indentation?

* Web Page: As standalone, but recognize meta fields as meta tags.
  Support for templates of some sort?  (After ``<body>``, before
  ``</body>``?)

* FAQ: Structured "question & answer(s)" constructs.

* Compound document: Merge chapters into a book.  Master manifest
  file?

Parsers
-------

Parsers analyze their input and produce a Docutils `document tree`_.
They don't know or care anything about the source or destination of
the data.

Each input parser is a module or package exporting a "Parser" class
with a "parse" method.  The base "Parser" class can be found in the
``docutils/parsers/__init__.py`` module.

Responsibilities: Given raw input text and a doctree root node,
populate the doctree by parsing the input text.

Example: The only parser implemented so far is for the
reStructuredText markup.  It is implemented in the
``docutils/parsers/rst/`` package.

The development and integration of other parsers is possible and
encouraged.

.. _transforms:

Transformer
-----------

The Transformer class, in ``docutils/transforms/__init__.py``, stores
transforms and applies them to documents.  A transformer object is
attached to every new document tree.  The Publisher_ calls
``Transformer.apply_transforms()`` to apply all stored transforms to
the document tree.  Transforms change the document tree from one form
to another, add to the tree, or prune it.  Transforms resolve
references and footnote numbers, process interpreted text, and do
other context-sensitive processing.

Some transforms are specific to components (Readers, Parser, Writers,
Input, Output).  Standard component-specific transforms are specified
in the ``default_transforms`` attribute of component classes.  After
the Reader has finished processing, the Publisher_ calls
``Transformer.populate_from_components()`` with a list of components
and all default transforms are stored.

Each transform is a class in a module in the ``docutils/transforms/``
package, a subclass of ``docutils.tranforms.Transform``.  Transform
classes each have a ``default_priority`` attribute which is used by
the Transformer to apply transforms in order (low to high).  The
default priority can be overridden when adding transforms to the
Transformer object.

Transformer responsibilities:

* Apply transforms to the document tree, in priority order.

* Store a mapping of component type name ('reader', 'writer', etc.) to
  component objects.  These are used by certain transforms (such as
  "components.Filter") to determine suitability.

Transform responsibilities:

* Modify a doctree in-place, either purely transforming one structure
  into another, or adding new structures based on the doctree and/or
  external data.

Examples of transforms (in the ``docutils/transforms/`` package):

* frontmatter.DocInfo: Conversion of document metadata (bibliographic
  information).

* references.AnonymousHyperlinks: Resolution of anonymous references
  to corresponding targets.

* parts.Contents: Generates a table of contents for a document.

* document.Merger: Combining multiple populated doctrees into one.
  (Not yet implemented or fully understood.)

* document.Splitter: Splits a document into a tree-structure of
  subdocuments, perhaps by section.  It will have to transform
  references appropriately.  (Neither implemented not remotely
  understood.)

* components.Filter: Includes or excludes elements which depend on a
  specific Docutils component.

Writers
-------

Writers produce the final output (HTML, XML, TeX, etc.).  Writers
translate the internal `document tree`_ structure into the final data
format, possibly running Writer-specific transforms_ first.

By the time the document gets to the Writer, it should be in final
form.  The Writer's job is simply (and only) to translate from the
Docutils doctree structure to the target format.  Some small
transforms may be required, but they should be local and
format-specific.

Each writer is a module or package exporting a "Writer" class with a
"write" method.  The base "Writer" class can be found in the
``docutils/writers/__init__.py`` module.

Responsibilities:

* Translate doctree(s) into specific output formats.

  - Transform references into format-native forms.

* Write the translated output to the destination I/O.

Examples:

* XML: Various forms, such as:

  - Docutils XML (an expression of the internal document tree,
    implemented as ``docutils.writers.docutils_xml``).

  - DocBook (being implemented in the Docutils sandbox).

* HTML (XHTML implemented as ``docutils.writers.html4css1``).

* PDF (a ReportLabs interface is being developed in the Docutils
  sandbox).

* TeX (a LaTeX Writer is being implemented in the sandbox).

* Docutils-native pseudo-XML (implemented as
  ``docutils.writers.pseudoxml``, used for testing).

* Plain text

* reStructuredText?

Input/Output
------------

I/O classes provide a uniform API for low-level input and output.
Subclasses will exist for a variety of input/output mechanisms.
However, they can be considered an implementation detail.  Most
applications should be satisfied using one of the convenience
functions associated with the Publisher_.

I/O classes are currently in the preliminary stages; there's a lot of
work yet to be done.  Issues:

* How to represent multi-file input (files & directories) in the API?

* How to represent multi-file output?  Perhaps "Writer" variants, one
  for each output distribution type?  Or Output objects with
  associated transforms?

Responsibilities:

* Read data from the input source (Input objects) or write data to the
  output destination (Output objects).

Examples of input sources:

* A single file on disk or a stream (implemented as
  ``docutils.io.FileInput``).

* Multiple files on disk (``MultiFileInput``?).

* Python source files: modules and packages.

* Python strings, as received from a client application
  (implemented as ``docutils.io.StringInput``).

Examples of output destinations:

* A single file on disk or a stream (implemented as
  ``docutils.io.FileOutput``).

* A tree of directories and files on disk.

* A Python string, returned to a client application (implemented as
  ``docutils.io.StringOutput``).

* No output; useful for programmatic applications where only a portion
  of the normal output is to be used (implemented as
  ``docutils.io.NullOutput``).

* A single tree-shaped data structure in memory.

* Some other set of data structures in memory.

Docutils Package Structure
==========================

* Package "docutils".

  - Module "__init__.py" contains: class "Component", a base class for
    Docutils components; class "SettingsSpec", a base class for
    specifying runtime settings (used by docutils.frontend); and class
    "TransformSpec", a base class for specifying transforms.

  - Module "docutils.core" contains facade class "Publisher" and
    convenience functions.  See `Publisher`_ above.

  - Module "docutils.frontend" provides runtime settings support, for
    programmatic use and front-end tools (including configuration file
    support, and command-line argument and option processing).

  - Module "docutils.io" provides a uniform API for low-level input
    and output.  See `Input/Output`_ above.

  - Module "docutils.nodes" contains the Docutils document tree
    element class library plus tree-traversal Visitor pattern base
    classes.  See `Document Tree`_ below.

  - Module "docutils.statemachine" contains a finite state machine
    specialized for regular-expression-based text filters and parsers.
    The reStructuredText parser implementation is based on this
    module.

  - Module "docutils.urischemes" contains a mapping of known URI
    schemes ("http", "ftp", "mail", etc.).

  - Module "docutils.utils" contains utility functions and classes,
    including a logger class ("Reporter"; see `Error Handling`_
    below).

  - Package "docutils.parsers": markup parsers_.

    - Function "get_parser_class(parser_name)" returns a parser module
      by name.  Class "Parser" is the base class of specific parsers.
      (``docutils/parsers/__init__.py``)

    - Package "docutils.parsers.rst": the reStructuredText parser.

    - Alternate markup parsers may be added.

    See `Parsers`_ above.

  - Package "docutils.readers": context-aware input readers.

    - Function "get_reader_class(reader_name)" returns a reader module
      by name or alias.  Class "Reader" is the base class of specific
      readers.  (``docutils/readers/__init__.py``)

    - Module "docutils.readers.standalone" reads independent document
      files.

    - Module "docutils.readers.pep" reads PEPs (Python Enhancement
      Proposals).

    - Readers to be added for: Python source code (structure &
      docstrings), email, FAQ, and perhaps Wiki and others.

    See `Readers`_ above.

  - Package "docutils.writers": output format writers.

    - Function "get_writer_class(writer_name)" returns a writer module
      by name.  Class "Writer" is the base class of specific writers.
      (``docutils/writers/__init__.py``)

    - Module "docutils.writers.html4css1" is a simple HyperText Markup
      Language document tree writer for HTML 4.01 and CSS1.

    - Module "docutils.writers.docutils_xml" writes the internal
      document tree in XML form.

    - Module "docutils.writers.pseudoxml" is a simple internal
      document tree writer; it writes indented pseudo-XML.

    - Writers to be added: HTML 3.2 or 4.01-loose, XML (various forms,
      such as DocBook), PDF, TeX, plaintext, reStructuredText, and
      perhaps others.

    See `Writers`_ above.

  - Package "docutils.transforms": tree transform classes.

    - Class "Transformer" stores transforms and applies them to
      document trees.  (``docutils/transforms/__init__.py``)

    - Class "Transform" is the base class of specific transforms.
      (``docutils/transforms/__init__.py``)

    - Each module contains related transform classes.

    See `Transforms`_ above.

  - Package "docutils.languages": Language modules contain
    language-dependent strings and mappings.  They are named for their
    language identifier (as defined in `Choice of Docstring Format`_
    below), converting dashes to underscores.

    - Function "get_language(language_code)", returns matching
      language module.  (``docutils/languages/__init__.py``)

    - Modules: en.py (English), de.py (German), fr.py (French), it.py
      (Italian), sk.py (Slovak), sv.py (Swedish).

    - Other languages to be added.

* Third-party modules: "extras" directory.  These modules are
  installed only if they're not already present in the Python
  installation.

  - ``extras/optparse.py`` and ``extras/textwrap.py`` provide
    option parsing and command-line help; from Greg Ward's
    http://optik.sf.net/ project, included for convenience.

  - ``extras/roman.py`` contains Roman numeral conversion routines.

Front-End Tools
===============

The ``tools/`` directory contains several front ends for common
Docutils processing.  See `Docutils Front-End Tools`_ for details.

.. _Docutils Front-End Tools:
   http://docutils.sourceforge.net/docs/user/tools.html

Document Tree
=============

A single intermediate data structure is used internally by Docutils,
in the interfaces between components; it is defined in the
``docutils.nodes`` module.  It is not required that this data
structure be used *internally* by any of the components, just
*between* components as outlined in the diagram in the `Docutils
Project Model`_ above.

Custom node types are allowed, provided that either (a) a transform
converts them to standard Docutils nodes before they reach the Writer
proper, or (b) the custom node is explicitly supported by certain
Writers, and is wrapped in a filtered "pending" node.  An example of
condition (a) is the `Python Source Reader`_ (see below), where a
"stylist" transform converts custom nodes.  The HTML ``<meta>`` tag is
an example of condition (b); it is supported by the HTML Writer but
not by others.  The reStructuredText "meta" directive creates a
"pending" node, which contains knowledge that the embedded "meta" node
can only be handled by HTML-compatible writers.  The "pending" node is
resolved by the ``docutils.transforms.components.Filter`` transform,
which checks that the calling writer supports HTML; if it doesn't, the
"pending" node (and enclosed "meta" node) is removed from the
document.

The document tree data structure is similar to a DOM tree, but with
specific node names (classes) instead of DOM's generic nodes. The
schema is documented in an XML DTD (eXtensible Markup Language
Document Type Definition), which comes in two parts:

* the Docutils Generic DTD, docutils.dtd_, and

* the OASIS Exchange Table Model, soextbl.dtd_.

The DTD defines a rich set of elements, suitable for many input and
output formats.  The DTD retains all information necessary to
reconstruct the original input text, or a reasonable facsimile
thereof.

See `The Docutils Document Tree`_ for details (incomplete).

Error Handling
==============

When the parser encounters an error in markup, it inserts a system
message (DTD element "system_message").  There are five levels of
system messages:

* Level-0, "DEBUG": an internal reporting issue.  There is no effect
  on the processing.  Level-0 system messages are handled separately
  from the others.

* Level-1, "INFO": a minor issue that can be ignored.  There is little
  or no effect on the processing.  Typically level-1 system messages
  are not reported.

* Level-2, "WARNING": an issue that should be addressed.  If ignored,
  there may be minor problems with the output.  Typically level-2
  system messages are reported but do not halt processing

* Level-3, "ERROR": a major issue that should be addressed.  If
  ignored, the output will contain unpredictable errors.  Typically
  level-3 system messages are reported but do not halt processing

* Level-4, "SEVERE": a critical error that must be addressed.
  Typically level-4 system messages are turned into exceptions which
  halt processing.  If ignored, the output will contain severe errors.

Although the initial message levels were devised independently, they
have a strong correspondence to `VMS error condition severity
levels`_; the names in quotes for levels 1 through 4 were borrowed
from VMS.  Error handling has since been influenced by the `log4j
project`_.

Python Source Reader
====================

The Python Source Reader ("PySource") is the Docutils component that
reads Python source files, extracts docstrings in context, then
parses, links, and assembles the docstrings into a cohesive whole.  It
is a major and non-trivial component, currently under experimental
development in the Docutils sandbox.  High-level design issues are
presented here.

Processing Model
----------------

This model will evolve over time, incorporating experience and
discoveries.

1. The PySource Reader uses an Input class to read in Python packages
   and modules, into a tree of strings.

2. The Python modules are parsed, converting the tree of strings into
   a tree of abstract syntax trees with docstring nodes.

3. The abstract syntax trees are converted into an internal
   representation of the packages/modules.  Docstrings are extracted,
   as well as code structure details.  See `AST Mining`_ below.
   Namespaces are constructed for lookup in step 6.

4. One at a time, the docstrings are parsed, producing standard
   Docutils doctrees.

5. PySource assembles all the individual docstrings' doctrees into a
   Python-specific custom Docutils tree paralleling the
   package/module/class structure; this is a custom Reader-specific
   internal representation (see the `Docutils Python Source DTD`_).
   Namespaces must be merged: Python identifiers, hyperlink targets.

6. Cross-references from docstrings (interpreted text) to Python
   identifiers are resolved according to the Python namespace lookup
   rules.  See `Identifier Cross-References`_ below.

7. A "Stylist" transform is applied to the custom doctree (by the
   Transformer_), custom nodes are rendered using standard nodes as
   primitives, and a standard document tree is emitted.  See `Stylist
   Transforms`_ below.

8. Other transforms are applied to the standard doctree by the
   Transformer_.

9. The standard doctree is sent to a Writer, which translates the
   document into a concrete format (HTML, PDF, etc.).

10. The Writer uses an Output class to write the resulting data to its
    destination (disk file, directories and files, etc.).

AST Mining
----------

Abstract Syntax Tree mining code will be written (or adapted) that
scans a parsed Python module, and returns an ordered tree containing
the names, docstrings (including attribute and additional docstrings;
see below), and additional info (in parentheses below) of all of the
following objects:

* packages
* modules
* module attributes (+ initial values)
* classes (+ inheritance)
* class attributes (+ initial values)
* instance attributes (+ initial values)
* methods (+ parameters & defaults)
* functions (+ parameters & defaults)

(Extract comments too?  For example, comments at the start of a module
would be a good place for bibliographic field lists.)

In order to evaluate interpreted text cross-references, namespaces for
each of the above will also be required.

See the python-dev/docstring-develop thread "AST mining", started on
2001-08-14.

Docstring Extraction Rules
--------------------------

1. What to examine:

   a) If the "``__all__``" variable is present in the module being
      documented, only identifiers listed in "``__all__``" are
      examined for docstrings.

   b) In the absence of "``__all__``", all identifiers are examined,
      except those whose names are private (names begin with "_" but
      don't begin and end with "__").

   c) 1a and 1b can be overridden by runtime settings.

2. Where:

   Docstrings are string literal expressions, and are recognized in
   the following places within Python modules:

   a) At the beginning of a module, function definition, class
      definition, or method definition, after any comments.  This is
      the standard for Python ``__doc__`` attributes.

   b) Immediately following a simple assignment at the top level of a
      module, class definition, or ``__init__`` method definition,
      after any comments.  See `Attribute Docstrings`_ below.

   c) Additional string literals found immediately after the
      docstrings in (a) and (b) will be recognized, extracted, and
      concatenated.  See `Additional Docstrings`_ below.

   d) @@@ 2.2-style "properties" with attribute docstrings?  Wait for
      syntax?

3. How:

   Whenever possible, Python modules should be parsed by Docutils, not
   imported.  There are several reasons:

   - Importing untrusted code is inherently insecure.

   - Information from the source is lost when using introspection to
     examine an imported module, such as comments and the order of
     definitions.

   - Docstrings are to be recognized in places where the byte-code
     compiler ignores string literal expressions (2b and 2c above),
     meaning importing the module will lose these docstrings.

   Of course, standard Python parsing tools such as the "parser"
   library module should be used.

   When the Python source code for a module is not available
   (i.e. only the ``.pyc`` file exists) or for C extension modules, to
   access docstrings the module can only be imported, and any
   limitations must be lived with.

Since attribute docstrings and additional docstrings are ignored by
the Python byte-code compiler, no namespace pollution or runtime bloat
will result from their use.  They are not assigned to ``__doc__`` or
to any other attribute.  The initial parsing of a module may take a
slight performance hit.

Attribute Docstrings
''''''''''''''''''''

(This is a simplified version of PEP 224 [#PEP-224]_.)

A string literal immediately following an assignment statement is
interpreted by the docstring extraction machinery as the docstring of
the target of the assignment statement, under the following
conditions:

1. The assignment must be in one of the following contexts:

   a) At the top level of a module (i.e., not nested inside a compound
      statement such as a loop or conditional): a module attribute.

   b) At the top level of a class definition: a class attribute.

   c) At the top level of the "``__init__``" method definition of a
      class: an instance attribute.  Instance attributes assigned in
      other methods are assumed to be implementation details.  (@@@
      ``__new__`` methods?)

   d) A function attribute assignment at the top level of a module or
      class definition.

   Since each of the above contexts are at the top level (i.e., in the
   outermost suite of a definition), it may be necessary to place
   dummy assignments for attributes assigned conditionally or in a
   loop.

2. The assignment must be to a single target, not to a list or a tuple
   of targets.

3. The form of the target:

   a) For contexts 1a and 1b above, the target must be a simple
      identifier (not a dotted identifier, a subscripted expression,
      or a sliced expression).

   b) For context 1c above, the target must be of the form
      "``self.attrib``", where "``self``" matches the "``__init__``"
      method's first parameter (the instance parameter) and "attrib"
      is a simple identifier as in 3a.

   c) For context 1d above, the target must be of the form
      "``name.attrib``", where "``name``" matches an already-defined
      function or method name and "attrib" is a simple identifier as
      in 3a.

Blank lines may be used after attribute docstrings to emphasize the
connection between the assignment and the docstring.

Examples::

    g = 'module attribute (module-global variable)'
    """This is g's docstring."""

    class AClass:

        c = 'class attribute'
        """This is AClass.c's docstring."""

        def __init__(self):
            """Method __init__'s docstring."""

            self.i = 'instance attribute'
            """This is self.i's docstring."""

    def f(x):
        """Function f's docstring."""
        return x**2

    f.a = 1
    """Function attribute f.a's docstring."""

Additional Docstrings
'''''''''''''''''''''

(This idea was adapted from PEP 216 [#PEP-216]_.)

Many programmers would like to make extensive use of docstrings for
API documentation.  However, docstrings do take up space in the
running program, so some programmers are reluctant to "bloat up" their
code.  Also, not all API documentation is applicable to interactive
environments, where ``__doc__`` would be displayed.

Docutils' docstring extraction tools will concatenate all string
literal expressions which appear at the beginning of a definition or
after a simple assignment.  Only the first strings in definitions will
be available as ``__doc__``, and can be used for brief usage text
suitable for interactive sessions; subsequent string literals and all
attribute docstrings are ignored by the Python byte-code compiler and
may contain more extensive API information.

Example::

    def function(arg):
        """This is __doc__, function's docstring."""
        """
        This is an additional docstring, ignored by the byte-code
        compiler, but extracted by Docutils.
        """
        pass

.. topic:: Issue: ``from __future__ import``

   This would break "``from __future__ import``" statements introduced
   in Python 2.1 for multiple module docstrings (main docstring plus
   additional docstring(s)).  The Python Reference Manual specifies:

       A future statement must appear near the top of the module.  The
       only lines that can appear before a future statement are:

       * the module docstring (if any),
       * comments,
       * blank lines, and
       * other future statements.

   Resolution?

   1. Should we search for docstrings after a ``__future__``
      statement?  Very ugly.

   2. Redefine ``__future__`` statements to allow multiple preceding
      string literals?

   3. Or should we not even worry about this?  There probably
      shouldn't be ``__future__`` statements in production code, after
      all.  Perhaps modules with ``__future__`` statements will simply
      have to put up with the single-docstring limitation.

Choice of Docstring Format
--------------------------

Rather than force everyone to use a single docstring format, multiple
input formats are allowed by the processing system.  A special
variable, ``__docformat__``, may appear at the top level of a module
before any function or class definitions.  Over time or through
decree, a standard format or set of formats should emerge.

A module's ``__docformat__`` variable only applies to the objects
defined in the module's file.  In particular, the ``__docformat__``
variable in a package's ``__init__.py`` file does not apply to objects
defined in subpackages and submodules.

The ``__docformat__`` variable is a string containing the name of the
format being used, a case-insensitive string matching the input
parser's module or package name (i.e., the same name as required to
"import" the module or package), or a registered alias.  If no
``__docformat__`` is specified, the default format is "plaintext" for
now; this may be changed to the standard format if one is ever
established.

The ``__docformat__`` string may contain an optional second field,
separated from the format name (first field) by a single space: a
case-insensitive language identifier as defined in RFC 1766.  A
typical language identifier consists of a 2-letter language code from
`ISO 639`_ (3-letter codes used only if no 2-letter code exists; RFC
1766 is currently being revised to allow 3-letter codes).  If no
language identifier is specified, the default is "en" for English.
The language identifier is passed to the parser and can be used for
language-dependent markup features.

Identifier Cross-References
---------------------------

In Python docstrings, interpreted text is used to classify and mark up
program identifiers, such as the names of variables, functions,
classes, and modules.  If the identifier alone is given, its role is
inferred implicitly according to the Python namespace lookup rules.
For functions and methods (even when dynamically assigned),
parentheses ('()') may be included::

    This function uses `another()` to do its work.

For class, instance and module attributes, dotted identifiers are used
when necessary.  For example (using reStructuredText markup)::

    class Keeper(Storer):

        """
        Extend `Storer`.  Class attribute `instances` keeps track
        of the number of `Keeper` objects instantiated.
        """

        instances = 0
        """How many `Keeper` objects are there?"""

        def __init__(self):
            """
            Extend `Storer.__init__()` to keep track of instances.

            Keep count in `Keeper.instances`, data in `self.data`.
            """
            Storer.__init__(self)
            Keeper.instances += 1

            self.data = []
            """Store data in a list, most recent last."""

        def store_data(self, data):
            """
            Extend `Storer.store_data()`; append new `data` to a
            list (in `self.data`).
            """
            self.data = data

Each of the identifiers quoted with backquotes ("`") will become
references to the definitions of the identifiers themselves.

Stylist Transforms
------------------

Stylist transforms are specialized transforms specific to the PySource
Reader.  The PySource Reader doesn't have to make any decisions as to
style; it just produces a logically constructed document tree, parsed
and linked, including custom node types.  Stylist transforms
understand the custom nodes created by the Reader and convert them
into standard Docutils nodes.

Multiple Stylist transforms may be implemented and one can be chosen
at runtime (through a "--style" or "--stylist" command-line option).
Each Stylist transform implements a different layout or style; thus
the name.  They decouple the context-understanding part of the Reader
from the layout-generating part of processing, resulting in a more
flexible and robust system.  This also serves to "separate style from
content", the SGML/XML ideal.

By keeping the piece of code that does the styling small and modular,
it becomes much easier for people to roll their own styles.  The
"barrier to entry" is too high with existing tools; extracting the
stylist code will lower the barrier considerably.

==========================
 References and Footnotes
==========================

.. [#PEP-256] PEP 256, Docstring Processing System Framework, Goodger
   (http://www.python.org/peps/pep-0256.html)

.. [#PEP-224] PEP 224, Attribute Docstrings, Lemburg
   (http://www.python.org/peps/pep-0224.html)

.. [#PEP-216] PEP 216, Docstring Format, Zadka
   (http://www.python.org/peps/pep-0216.html)

.. _docutils.dtd:
   http://docutils.sourceforge.net/docs/ref/docutils.dtd

.. _soextbl.dtd:
   http://docutils.sourceforge.net/docs/ref/soextblx.dtd

.. _The Docutils Document Tree:
   http://docutils.sourceforge.net/docs/ref/doctree.html

.. _VMS error condition severity levels:
   http://www.openvms.compaq.com:8000/73final/5841/841pro_027.html
   #error_cond_severity

.. _log4j project: http://logging.apache.org/log4j/docs/index.html

.. _Docutils Python Source DTD:
   http://docutils.sourceforge.net/docs/dev/pysource.dtd

.. _ISO 639: http://lcweb.loc.gov/standards/iso639-2/englangn.html

.. _Python Doc-SIG: http://www.python.org/sigs/doc-sig/

==================
 Project Web Site
==================

A SourceForge project has been set up for this work at
http://docutils.sourceforge.net/.

===========
 Copyright
===========

This document has been placed in the public domain.

==================
 Acknowledgements
==================

This document borrows ideas from the archives of the `Python
Doc-SIG`_.  Thanks to all members past & present.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


PEP: 287
Title: reStructuredText Docstring Format
Version: $Revision: 1.22 $
Last-Modified: $Date: 2004/06/03 18:27:51 $
Author: David Goodger <goodger@users.sourceforge.net>
Discussions-To: <doc-sig@python.org>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 25-Mar-2002
Post-History: 02-Apr-2002
Replaces: 216

Abstract
========

When plaintext hasn't been expressive enough for inline documentation,
Python programmers have sought out a format for docstrings.  This PEP
proposes that the `reStructuredText markup`_ be adopted as a standard
markup format for structured plaintext documentation in Python
docstrings, and for PEPs and ancillary documents as well.
reStructuredText is a rich and extensible yet easy-to-read,
what-you-see-is-what-you-get plaintext markup syntax.

Only the low-level syntax of docstrings is addressed here.  This PEP
is not concerned with docstring semantics or processing at all (see
PEP 256 for a "Road Map to the Docstring PEPs").  Nor is it an attempt
to deprecate pure plaintext docstrings, which are always going to be
legitimate.  The reStructuredText markup is an alternative for those
who want more expressive docstrings.

Benefits
========

Programmers are by nature a lazy breed.  We reuse code with functions,
classes, modules, and subsystems.  Through its docstring syntax,
Python allows us to document our code from within.  The "holy grail"
of the Python Documentation Special Interest Group (Doc-SIG_) has been
a markup syntax and toolset to allow auto-documentation, where the
docstrings of Python systems can be extracted in context and processed
into useful, high-quality documentation for multiple purposes.

Document markup languages have three groups of customers: the authors
who write the documents, the software systems that process the data,
and the readers, who are the final consumers and the most important
group.  Most markups are designed for the authors and software
systems; readers are only meant to see the processed form, either on
paper or via browser software.  ReStructuredText is different: it is
intended to be easily readable in source form, without prior knowledge
of the markup.  ReStructuredText is entirely readable in plaintext
format, and many of the markup forms match common usage (e.g.,
``*emphasis*``), so it reads quite naturally.  Yet it is rich enough
to produce complex documents, and extensible so that there are few
limits.  Of course, to write reStructuredText documents some prior
knowledge is required.

The markup offers functionality and expressivity, while maintaining
easy readability in the source text.  The processed form (HTML etc.)
makes it all accessible to readers: inline live hyperlinks; live links
to and from footnotes; automatic tables of contents (with live
links!); tables; images for diagrams etc.; pleasant, readable styled
text.

The reStructuredText parser is available now, part of the Docutils_
project.  Standalone reStructuredText documents and PEPs can be
converted to HTML; other output format writers are being worked on and
will become available over time.  Work is progressing on a Python
source "Reader" which will implement auto-documentation from
docstrings.  Authors of existing auto-documentation tools are
encouraged to integrate the reStructuredText parser into their
projects, or better yet, to join forces to produce a world-class
toolset for the Python standard library.

Tools will become available in the near future, which will allow
programmers to generate HTML for online help, XML for multiple
purposes, and eventually PDF, DocBook, and LaTeX for printed
documentation, essentially "for free" from the existing docstrings.
The adoption of a standard will, at the very least, benefit docstring
processing tools by preventing further "reinventing the wheel".

Eventually PyDoc, the one existing standard auto-documentation tool,
could have reStructuredText support added.  In the interim it will
have no problem with reStructuredText markup, since it treats all
docstrings as preformatted plaintext.

Goals
=====

These are the generally accepted goals for a docstring format, as
discussed in the Doc-SIG:

1. It must be readable in source form by the casual observer.

2. It must be easy to type with any standard text editor.

3. It must not need to contain information which can be deduced from
   parsing the module.

4. It must contain sufficient information (structure) so it can be
   converted to any reasonable markup format.

5. It must be possible to write a module's entire documentation in
   docstrings, without feeling hampered by the markup language.

reStructuredText meets and exceeds all of these goals, and sets its
own goals as well, even more stringent.  See `Docstring-Significant
Features`_ below.

The goals of this PEP are as follows:

1. To establish reStructuredText as a standard structured plaintext
   format for docstrings (inline documentation of Python modules and
   packages), PEPs, README-type files and other standalone documents.
   "Accepted" status will be sought through Python community consensus
   and eventual BDFL pronouncement.

   Please note that reStructuredText is being proposed as *a*
   standard, not *the only* standard.  Its use will be entirely
   optional.  Those who don't want to use it need not.

2. To solicit and address any related concerns raised by the Python
   community.

3. To encourage community support.  As long as multiple competing
   markups are out there, the development community remains fractured.
   Once a standard exists, people will start to use it, and momentum
   will inevitably gather.

4. To consolidate efforts from related auto-documentation projects.
   It is hoped that interested developers will join forces and work on
   a joint/merged/common implementation.

Once reStructuredText is a Python standard, effort can be focused on
tools instead of arguing for a standard.  Python needs a standard set
of documentation tools.

With regard to PEPs, one or both of the following strategies may be
applied:

a) Keep the existing PEP section structure constructs (one-line
   section headers, indented body text).  Subsections can either be
   forbidden, or supported with reStructuredText-style underlined
   headers in the indented body text.

b) Replace the PEP section structure constructs with the
   reStructuredText syntax.  Section headers will require underlines,
   subsections will be supported out of the box, and body text need
   not be indented (except for block quotes).

Strategy (b) is recommended, and its implementation is complete.

Support for RFC 2822 headers has been added to the reStructuredText
parser for PEPs (unambiguous given a specific context: the first
contiguous block of the document).  It may be desired to concretely
specify what over/underline styles are allowed for PEP section
headers, for uniformity.

Rationale
=========

The lack of a standard syntax for docstrings has hampered the
development of standard tools for extracting and converting docstrings
into documentation in standard formats (e.g., HTML, DocBook, TeX).
There have been a number of proposed markup formats and variations,
and many tools tied to these proposals, but without a standard
docstring format they have failed to gain a strong following and/or
floundered half-finished.

Throughout the existence of the Doc-SIG, consensus on a single
standard docstring format has never been reached.  A lightweight,
implicit markup has been sought, for the following reasons (among
others):

1. Docstrings written within Python code are available from within the
   interactive interpreter, and can be "print"ed.  Thus the use of
   plaintext for easy readability.

2. Programmers want to add structure to their docstrings, without
   sacrificing raw docstring readability.  Unadorned plaintext cannot
   be transformed ("up-translated") into useful structured formats.

3. Explicit markup (like XML or TeX) is widely considered unreadable
   by the uninitiated.

4. Implicit markup is aesthetically compatible with the clean and
   minimalist Python syntax.

Many alternative markups for docstrings have been proposed on the
Doc-SIG over the years; a representative sample is listed below.  Each
is briefly analyzed in terms of the goals stated above.  Please note
that this is *not* intended to be an exclusive list of all existing
markup systems; there are many other markups (Texinfo, Doxygen, TIM,
YODL, AFT, ...) which are not mentioned.

- XML_, SGML_, DocBook_, HTML_, XHTML_

  XML and SGML are explicit, well-formed meta-languages suitable for
  all kinds of documentation.  XML is a variant of SGML.  They are
  best used behind the scenes, because to untrained eyes they are
  verbose, difficult to type, and too cluttered to read comfortably as
  source.  DocBook, HTML, and XHTML are all applications of SGML
  and/or XML, and all share the same basic syntax and the same
  shortcomings.

- TeX_

  TeX is similar to XML/SGML in that it's explicit, but not very easy
  to write, and not easy for the uninitiated to read.

- `Perl POD`_

  Most Perl modules are documented in a format called POD (Plain Old
  Documentation).  This is an easy-to-type, very low level format with
  strong integration with the Perl parser.  Many tools exist to turn
  POD documentation into other formats: info, HTML and man pages,
  among others.  However, the POD syntax takes after Perl itself in
  terms of readability.

- JavaDoc_

  Special comments before Java classes and functions serve to document
  the code.  A program to extract these, and turn them into HTML
  documentation is called javadoc, and is part of the standard Java
  distribution.  However, JavaDoc has a very intimate relationship
  with HTML, using HTML tags for most markup.  Thus it shares the
  readability problems of HTML.

- Setext_, StructuredText_

  Early on, variants of Setext (Structure Enhanced Text), including
  Zope Corp's StructuredText, were proposed for Python docstring
  formatting.  Hereafter these variants will collectively be called
  "STexts".  STexts have the advantage of being easy to read without
  special knowledge, and relatively easy to write.

  Although used by some (including in most existing Python
  auto-documentation tools), until now STexts have failed to become
  standard because:

  - STexts have been incomplete.  Lacking "essential" constructs that
    people want to use in their docstrings, STexts are rendered less
    than ideal.  Note that these "essential" constructs are not
    universal; everyone has their own requirements.

  - STexts have been sometimes surprising.  Bits of text are
    unexpectedly interpreted as being marked up, leading to user
    frustration.

  - SText implementations have been buggy.

  - Most STexts have have had no formal specification except for the
    implementation itself.  A buggy implementation meant a buggy spec,
    and vice-versa.

  - There has been no mechanism to get around the SText markup rules
    when a markup character is used in a non-markup context.  In other
    words, no way to escape markup.

Proponents of implicit STexts have vigorously opposed proposals for
explicit markup (XML, HTML, TeX, POD, etc.), and the debates have
continued off and on since 1996 or earlier.

reStructuredText is a complete revision and reinterpretation of the
SText idea, addressing all of the problems listed above.

Specification
=============

The specification and user documentaton for reStructuredText is
quite extensive.  Rather than repeating or summarizing it all
here, links to the originals are provided.

Please first take a look at `A ReStructuredText Primer`_, a short and
gentle introduction.  The `Quick reStructuredText`_ user reference
quickly summarizes all of the markup constructs.  For complete and
extensive details, please refer to the following documents:

- `An Introduction to reStructuredText`_

- `reStructuredText Markup Specification`_

- `reStructuredText Directives`_

In addition, `Problems With StructuredText`_ explains many markup
decisions made with regards to StructuredText, and `A Record of
reStructuredText Syntax Alternatives`_ records markup decisions made
independently.

Docstring-Significant Features
==============================

- A markup escaping mechanism.

  Backslashes (``\``) are used to escape markup characters when needed
  for non-markup purposes.  However, the inline markup recognition
  rules have been constructed in order to minimize the need for
  backslash-escapes.  For example, although asterisks are used for
  *emphasis*, in non-markup contexts such as "*" or "(*)" or "x * y",
  the asterisks are not interpreted as markup and are left unchanged.
  For many non-markup uses of backslashes (e.g., describing regular
  expressions), inline literals or literal blocks are applicable; see
  the next item.

- Markup to include Python source code and Python interactive
  sessions: inline literals, literal blocks, and doctest blocks.

  Inline literals use ``double-backquotes`` to indicate program I/O or
  code snippets.  No markup interpretation (including backslash-escape
  [``\``] interpretation) is done within inline literals.

  Literal blocks (block-level literal text, such as code excerpts or
  ASCII graphics) are indented, and indicated with a double-colon
  ("::") at the end of the preceding paragraph (right here -->)::

      if literal_block:
          text = 'is left as-is'
          spaces_and_linebreaks = 'are preserved'
          markup_processing = None

  Doctest blocks begin with ">>> " and end with a blank line.  Neither
  indentation nor literal block double-colons are required.  For
  example::

      Here's a doctest block:

      >>> print 'Python-specific usage examples; begun with ">>>"'
      Python-specific usage examples; begun with ">>>"
      >>> print '(cut and pasted from interactive sessions)'
      (cut and pasted from interactive sessions)

- Markup that isolates a Python identifier: interpreted text.

  Text enclosed in single backquotes is recognized as "interpreted
  text", whose interpretation is application-dependent.  In the
  context of a Python docstring, the default interpretation of
  interpreted text is as Python identifiers.  The text will be marked
  up with a hyperlink connected to the documentation for the
  identifier given.  Lookup rules are the same as in Python itself:
  LGB namespace lookups (local, global, builtin).  The "role" of the
  interpreted text (identifying a class, module, function, etc.) is
  determined implicitly from the namespace lookup.  For example::

      class Keeper(Storer):

          """
          Keep data fresher longer.

          Extend `Storer`.  Class attribute `instances` keeps track
          of the number of `Keeper` objects instantiated.
          """

          instances = 0
          """How many `Keeper` objects are there?"""

          def __init__(self):
              """
              Extend `Storer.__init__()` to keep track of
              instances.  Keep count in `self.instances` and data
              in `self.data`.
              """
              Storer.__init__(self)
              self.instances += 1

              self.data = []
              """Store data in a list, most recent last."""

          def storedata(self, data):
              """
              Extend `Storer.storedata()`; append new `data` to a
              list (in `self.data`).
              """
              self.data = data

  Each piece of interpreted text is looked up according to the local
  namespace of the block containing its docstring.

- Markup that isolates a Python identifier and specifies its type:
  interpreted text with roles.

  Although the Python source context reader is designed not to require
  explicit roles, they may be used.  To classify identifiers
  explicitly, the role is given along with the identifier in either
  prefix or suffix form::

      Use :method:`Keeper.storedata` to store the object's data in
      `Keeper.data`:instance_attribute:.

  The syntax chosen for roles is verbose, but necessarily so (if
  anyone has a better alternative, please post it to the Doc-SIG_).
  The intention of the markup is that there should be little need to
  use explicit roles; their use is to be kept to an absolute minimum.

- Markup for "tagged lists" or "label lists": field lists.

  Field lists represent a mapping from field name to field body.
  These are mostly used for extension syntax, such as "bibliographic
  field lists" (representing document metadata such as author, date,
  and version) and extension attributes for directives (see below).
  They may be used to implement methodologies (docstring semantics),
  such as identifying parameters, exceptions raised, etc.; such usage
  is beyond the scope of this PEP.

  A modified RFC 2822 syntax is used, with a colon *before* as well as
  *after* the field name.  Field bodies are more versatile as well;
  they may contain multiple field bodies (even nested field lists).
  For example::

      :Date: 2002-03-22
      :Version: 1
      :Authors:
          - Me
          - Myself
          - I

  Standard RFC 2822 header syntax cannot be used for this construct
  because it is ambiguous.  A word followed by a colon at the
  beginning of a line is common in written text.

- Markup extensibility: directives and substitutions.

  Directives are used as an extension mechanism for reStructuredText,
  a way of adding support for new block-level constructs without
  adding new syntax.  Directives for images, admonitions (note,
  caution, etc.), and tables of contents generation (among others)
  have been implemented.  For example, here's how to place an image::

      .. image:: mylogo.png

  Substitution definitions allow the power and flexibility of
  block-level directives to be shared by inline text.  For example::

      The |biohazard| symbol must be used on containers used to
      dispose of medical waste.

      .. |biohazard| image:: biohazard.png

- Section structure markup.

  Section headers in reStructuredText use adornment via underlines
  (and possibly overlines) rather than indentation.  For example::

      This is a Section Title
      =======================

      This is a Subsection Title
      --------------------------

      This paragraph is in the subsection.

      This is Another Section Title
      =============================

      This paragraph is in the second section.

Questions & Answers
===================

1. Is reStructuredText rich enough?

   Yes, it is for most people.  If it lacks some construct that is
   required for a specific application, it can be added via the
   directive mechanism.  If a useful and common construct has been
   overlooked and a suitably readable syntax can be found, it can be
   added to the specification and parser.

2. Is reStructuredText *too* rich?

   For specific applications or individuals, perhaps.  In general, no.

   Since the very beginning, whenever a docstring markup syntax has
   been proposed on the Doc-SIG_, someone has complained about the
   lack of support for some construct or other.  The reply was often
   something like, "These are docstrings we're talking about, and
   docstrings shouldn't have complex markup."  The problem is that a
   construct that seems superfluous to one person may be absolutely
   essential to another.

   reStructuredText takes the opposite approach: it provides a rich
   set of implicit markup constructs (plus a generic extension
   mechanism for explicit markup), allowing for all kinds of
   documents.  If the set of constructs is too rich for a particular
   application, the unused constructs can either be removed from the
   parser (via application-specific overrides) or simply omitted by
   convention.

3. Why not use indentation for section structure, like StructuredText
   does?  Isn't it more "Pythonic"?

   Guido van Rossum wrote the following in a 2001-06-13 Doc-SIG post:

       I still think that using indentation to indicate sectioning is
       wrong.  If you look at how real books and other print
       publications are laid out, you'll notice that indentation is
       used frequently, but mostly at the intra-section level.
       Indentation can be used to offset lists, tables, quotations,
       examples, and the like.  (The argument that docstrings are
       different because they are input for a text formatter is wrong:
       the whole point is that they are also readable without
       processing.)

       I reject the argument that using indentation is Pythonic: text
       is not code, and different traditions and conventions hold.
       People have been presenting text for readability for over 30
       centuries.  Let's not innovate needlessly.

   See `Section Structure via Indentation`__ in `Problems With
   StructuredText`_ for further elaboration.

   __ http://docutils.sourceforge.net/docs/dev/rst/problems.html
      #section-structure-via-indentation

4. Why use reStructuredText for PEPs?  What's wrong with the existing
   standard?

   The existing standard for PEPs is very limited in terms of general
   expressibility, and referencing is especially lacking for such a
   reference-rich document type.  PEPs are currently converted into
   HTML, but the results (mostly monospaced text) are less than
   attractive, and most of the value-added potential of HTML
   (especially inline hyperlinks) is untapped.

   Making reStructuredText a standard markup for PEPs will enable much
   richer expression, including support for section structure, inline
   markup, graphics, and tables.  In several PEPs there are ASCII
   graphics diagrams, which are all that plaintext documents can
   support.  Since PEPs are made available in HTML form, the ability
   to include proper diagrams would be immediately useful.

   Current PEP practices allow for reference markers in the form "[1]"
   in the text, and the footnotes/references themselves are listed in
   a section toward the end of the document.  There is currently no
   hyperlinking between the reference marker and the
   footnote/reference itself (it would be possible to add this to
   pep2html.py, but the "markup" as it stands is ambiguous and
   mistakes would be inevitable).  A PEP with many references (such as
   this one ;-) requires a lot of flipping back and forth.  When
   revising a PEP, often new references are added or unused references
   deleted.  It is painful to renumber the references, since it has to
   be done in two places and can have a cascading effect (insert a
   single new reference 1, and every other reference has to be
   renumbered; always adding new references to the end is suboptimal).
   It is easy for references to go out of sync.

   PEPs use references for two purposes: simple URL references and
   footnotes.  reStructuredText differentiates between the two.  A PEP
   might contain references like this::

       Abstract

           This PEP proposes adding frungible doodads [1] to the core.
           It extends PEP 9876 [2] via the BCA [3] mechanism.

       ...

       References and Footnotes

           [1] http://www.example.org/

           [2] PEP 9876, Let's Hope We Never Get Here
               http://www.python.org/peps/pep-9876.html

           [3] "Bogus Complexity Addition"

   Reference 1 is a simple URL reference.  Reference 2 is a footnote
   containing text and a URL.  Reference 3 is a footnote containing
   text only.  Rewritten using reStructuredText, this PEP could look
   like this::

       Abstract
       ========

       This PEP proposes adding `frungible doodads`_ to the core.  It
       extends PEP 9876 [#pep9876]_ via the BCA [#]_ mechanism.

       ...

       References & Footnotes
       ======================

       .. _frungible doodads: http://www.example.org/

       .. [#pep9876] PEP 9876, Let's Hope We Never Get Here

       .. [#] "Bogus Complexity Addition"

   URLs and footnotes can be defined close to their references if
   desired, making them easier to read in the source text, and making
   the PEPs easier to revise.  The "References and Footnotes" section
   can be auto-generated with a document tree transform.  Footnotes
   from throughout the PEP would be gathered and displayed under a
   standard header.  If URL references should likewise be written out
   explicitly (in citation form), another tree transform could be
   used.

   URL references can be named ("frungible doodads"), and can be
   referenced from multiple places in the document without additional
   definitions.  When converted to HTML, references will be replaced
   with inline hyperlinks (HTML <a> tags).  The two footnotes are
   automatically numbered, so they will always stay in sync.  The
   first footnote also contains an internal reference name, "pep9876",
   so it's easier to see the connection between reference and footnote
   in the source text.  Named footnotes can be referenced multiple
   times, maintaining consistent numbering.

   The "#pep9876" footnote could also be written in the form of a
   citation::

       It extends PEP 9876 [PEP9876]_ ...

       .. [PEP9876] PEP 9876, Let's Hope We Never Get Here

   Footnotes are numbered, whereas citations use text for their
   references.

5. Wouldn't it be better to keep the docstring and PEP proposals
   separate?

   The PEP markup proposal may be removed if it is deemed that there
   is no need for PEP markup, or it could be made into a separate PEP.
   If accepted, PEP 1, PEP Purpose and Guidelines [#PEP-1]_, and PEP
   9, Sample PEP Template [#PEP-9]_ will be updated.

   It seems natural to adopt a single consistent markup standard for
   all uses of structured plaintext in Python, and to propose it all
   in one place.

6. The existing pep2html.py script converts the existing PEP format to
   HTML.  How will the new-format PEPs be converted to HTML?

   A new version of pep2html.py with integrated reStructuredText
   parsing has been completed.  The Docutils project supports PEPs
   with a "PEP Reader" component, including all functionality
   currently in pep2html.py (auto-recognition of PEP & RFC references,
   email masking, etc.).

7. Who's going to convert the existing PEPs to reStructuredText?

   PEP authors or volunteers may convert existing PEPs if they like,
   but there is no requirement to do so.  The reStructuredText-based
   PEPs will coexist with the old PEP standard.  The pep2html.py
   mentioned in answer 6 processes both old and new standards.

8. Why use reStructuredText for README and other ancillary files?

   The reasoning given for PEPs in answer 4 above also applies to
   README and other ancillary files.  By adopting a standard markup,
   these files can be converted to attractive cross-referenced HTML
   and put up on python.org.  Developers of other projects can also
   take advantage of this facility for their own documentation.

9. Won't the superficial similarity to existing markup conventions
   cause problems, and result in people writing invalid markup (and
   not noticing, because the plaintext looks natural)?  How forgiving
   is reStructuredText of "not quite right" markup?

   There will be some mis-steps, as there would be when moving from
   one programming language to another.  As with any language,
   proficiency grows with experience.  Luckily, reStructuredText is a
   very little language indeed.

   As with any syntax, there is the possibility of syntax errors.  It
   is expected that a user will run the processing system over their
   input and check the output for correctness.

   In a strict sense, the reStructuredText parser is very unforgiving
   (as it should be; "In the face of ambiguity, refuse the temptation
   to guess" [#Zen]_ applies to parsing markup as well as computer
   languages).  Here's design goal 3 from `An Introduction to
   reStructuredText`_:

       Unambiguous.  The rules for markup must not be open for
       interpretation.  For any given input, there should be one and
       only one possible output (including error output).

   While unforgiving, at the same time the parser does try to be
   helpful by producing useful diagnostic output ("system messages").
   The parser reports problems, indicating their level of severity
   (from least to most: debug, info, warning, error, severe).  The
   user or the client software can decide on reporting thresholds;
   they can ignore low-level problems or cause high-level problems to
   bring processing to an immediate halt.  Problems are reported
   during the parse as well as included in the output, often with
   two-way links between the source of the problem and the system
   message explaining it.

10. Will the docstrings in the Python standard library modules be
    converted to reStructuredText?

    No.  Python's library reference documentation is maintained
    separately from the source.  Docstrings in the Python standard
    library should not try to duplicate the library reference
    documentation.  The current policy for docstrings in the Python
    standard library is that they should be no more than concise
    hints, simple and markup-free (although many *do* contain ad-hoc
    implicit markup).

11. I want to write all my strings in Unicode.  Will anything
    break?

    The parser fully supports Unicode.  Docutils supports arbitrary
    input and output encodings.

12. Why does the community need a new structured text design?

    The existing structured text designs are deficient, for the
    reasons given in "Rationale" above.  reStructuredText aims to be a
    complete markup syntax, within the limitations of the "readable
    plaintext" medium.

13. What is wrong with existing documentation methodologies?

    What existing methodologies?  For Python docstrings, there is
    **no** official standard markup format, let alone a documentation
    methodology akin to JavaDoc.  The question of methodology is at a
    much higher level than syntax (which this PEP addresses).  It is
    potentially much more controversial and difficult to resolve, and
    is intentionally left out of this discussion.

References & Footnotes
======================

.. [#PEP-1] PEP 1, PEP Guidelines, Warsaw, Hylton
   (http://www.python.org/peps/pep-0001.html)

.. [#PEP-9] PEP 9, Sample PEP Template, Warsaw
   (http://www.python.org/peps/pep-0009.html)

.. [#Zen] From `The Zen of Python (by Tim Peters)`__ (or just
   "``import this``" in Python)

__ http://www.python.org/doc/Humor.html#zen

.. [#PEP-216] PEP 216, Docstring Format, Zadka
   (http://www.python.org/peps/pep-0216.html)

.. _reStructuredText markup: http://docutils.sourceforge.net/rst.html

.. _Doc-SIG: http://www.python.org/sigs/doc-sig/

.. _XML: http://www.w3.org/XML/

.. _SGML: http://www.oasis-open.org/cover/general.html

.. _DocBook: http://docbook.org/tdg/en/html/docbook.html

.. _HTML: http://www.w3.org/MarkUp/

.. _XHTML: http://www.w3.org/MarkUp/#xhtml1

.. _TeX: http://www.tug.org/interest.html

.. _Perl POD: http://www.perldoc.com/perl5.6/pod/perlpod.html

.. _JavaDoc: http://java.sun.com/j2se/javadoc/

.. _Setext: http://docutils.sourceforge.net/mirror/setext.html

.. _StructuredText:
   http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage

.. _A ReStructuredText Primer:
   http://docutils.sourceforge.net/docs/user/rst/quickstart.html

.. _Quick reStructuredText:
   http://docutils.sourceforge.net/docs/user/rst/quickref.html

.. _An Introduction to reStructuredText:
   http://docutils.sourceforge.net/docs/ref/rst/introduction.html

.. _reStructuredText Markup Specification:
   http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html

.. _reStructuredText Directives:
   http://docutils.sourceforge.net/docs/ref/rst/directives.html

.. _Problems with StructuredText:
   http://docutils.sourceforge.net/docs/dev/rst/problems.html

.. _A Record of reStructuredText Syntax Alternatives:
   http://docutils.sourceforge.net/docs/dev/rst/alternatives.html

.. _Docutils: http://docutils.sourceforge.net/

Copyright
=========

This document has been placed in the public domain.

Acknowledgements
================

Some text is borrowed from PEP 216, Docstring Format [#PEP-216]_, by
Moshe Zadka.

Special thanks to all members past & present of the Python Doc-SIG_.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


A Title

============================ The Docutils Document Tree ============================ A Guide to the Docutils DTD *************************** :Author: David Goodger :Contact: goodger@users.sourceforge.net :Revision: $Revision: 1.41 $ :Date: $Date: 2004/06/03 17:07:11 $ :Copyright: This document has been placed in the public domain. .. contents:: :depth: 1 This document describes the XML data structure of Docutils_ documents: the relationships and semantics of elements and attributes. The Docutils document structure is formally defined by the `Docutils Generic DTD`_ XML document type definition, docutils.dtd_, which is the definitive source for details of element structural relationships. This document does not discuss implementation details. Those can be found in internal documentation (docstrings) for the ``docutils.nodes`` module, where the document tree data structure is implemented in a class library. The reader is assumed to have some familiarity with XML or SGML, and an understanding of the data structure meaning of "tree". For a list of introductory articles, see `Introducing the Extensible Markup Language (XML)`_. The reStructuredText_ markup is used for illustrative examples throughout this document. For a gentle introduction, see `A ReStructuredText Primer`_. For complete technical details, see the `reStructuredText Markup Specification`_. .. _Docutils: http://docutils.sourceforge.net/ .. _Docutils Generic DTD: .. _Docutils DTD: .. _docutils.dtd: docutils.dtd .. _Introducing the Extensible Markup Language (XML): http://xml.coverpages.org/xmlIntro.html .. _reStructuredText: http://docutils.sourceforge.net/rst.html .. _A ReStructuredText Primer: ../user/rst/quickstart.html .. _reStructuredText Markup Specification: rst/restructuredtext.html ------------------- Element Hierarchy ------------------- .. contents:: :local: Below is a simplified diagram of the hierarchy of elements in the Docutils document tree structure. An element may contain any other elements immediately below it in the diagram. Notes are written in square brackets. Element types in parentheses indicate recursive or one-to-many relationships; sections may contain (sub)sections, tables contain further body elements, etc. :: +--------------------------------------------------------------------+ | document [may begin with a title, subtitle, docinfo, decoration] | | +--------------------------------------+ | | sections [each begins with a title] | +-----------------------------+-------------------------+------------+ | [body elements:] | (sections) | | | - literal | - lists | | - hyperlink +------------+ | | blocks | - tables | | targets | | para- | - doctest | - block | foot- | - sub. defs | | graphs | blocks | quotes | notes | - comments | +---------+-----------+----------+-------+--------------+ | [text]+ | [text] | (body elements) | [text] | | (inline +-----------+------------------+--------------+ | markup) | +---------+ The Docutils document model uses a simple, recursive model for section structure. A document_ node may contain body elements and section_ elements. Sections in turn may contain body elements and sections. The level (depth) of a section element is determined from its physical nesting level; unlike other document models (``

`` in HTML_, ```` in DocBook_, ```` in XMLSpec_) the level is not incorporated into the element name. The Docutils document model uses strict element content models. Every element has a unique structure and semantics, but elements may be classified into general categories (below). Only elements which are meant to directly contain text data have a mixed content model, where text data and inline elements may be intermixed. This is unlike the much looser HTML_ document model, where paragraphs and text data may occur at the same level. Structural Elements =================== Structural elements may only contain child elements; they do not directly contain text data. Structural elements may contain body elements or further structural elements. Structural elements can only be child elements of other structural elements. Category members: document_, section_, topic_, sidebar_ Structural Subelements ---------------------- Structural subelements are child elements of structural elements. Simple structuctural subelements (title_, subtitle_) contain text data; the others are compound and do not directly contain text data. Category members: title_, subtitle_, docinfo_, decoration_, transition_ Bibliographic Elements `````````````````````` The docinfo_ element is an optional child of document_. It groups bibliographic elements together. All bibliographic elements except authors_ and field_ contain text data. authors_ contains further bibliographic elements (most notably author_). field_ contains field_name_ and field_body_ body subelements. Category members: address_, author_, authors_, contact_, copyright_, date_, field_, organization_, revision_, status_, version_ Decorative Elements ``````````````````` The decoration_ element is also an optional child of document_. It groups together elements used to generate page headers and footers. Category members: footer_, header_ Body Elements ============= Body elements are contained within structural elements and compound body elements. There are two subcategories of body elements: simple and compound. Category members: admonition_, attention_, block_quote_, bullet_list_, caution_, citation_, comment_, danger_, definition_list_, doctest_block_, enumerated_list_, error_, field_list_, figure_, footnote_, hint_, image_, important_, line_block_, literal_block_, note_, option_list_, paragraph_, pending_, raw_, rubric_, substitution_definition_, system_message_, table_, target_, tip_, warning_ Simple Body Elements -------------------- Simple body elements directly are empty or contain text data. Those that contain text data may also contain inline elements. Such elements therefore have a "mixed content model". Category members: comment_, doctest_block_, image_, line_block_, literal_block_, paragraph_, pending_, raw_, rubric_, substitution_definition_, target_ Compound Body Elements ---------------------- Compound body elements contain local substructure (body subelements) and further body elements. They do not directly contain text data. Category members: admonition_, attention_, block_quote_, bullet_list_, caution_, citation_, danger_, definition_list_, enumerated_list_, error_, field_list_, figure_, footnote_, hint_, important_, note_, option_list_, system_message_, table_, tip_, warning_ Body Subelements ```````````````` Compound body elements contain specific subelements (e.g. bullet_list_ contains list_item_). Subelements may themselves be compound elements (containing further child elements, like field_) or simple data elements (containing text data, like field_name_). These subelements always occur within specific parent elements, never at the body element level (beside paragraphs, etc.). Category members (simple): attribution_, caption_, classifier_, colspec_, field_name_, label_, option_argument_, option_string_, term_ Category members (compound): definition_, definition_list_item_, description_, entry_, field_, field_body_, legend_, list_item_, option_, option_group_, option_list_item_, row_, tbody_, tgroup_, thead_ Inline Elements =============== Inline elements directly contain text data, and may also contain further inline elements. Inline elements are contained within simple body elements. Most inline elements have a "mixed content model". Category members: abbreviation_, acronym_, citation_reference_, emphasis_, footnote_reference_, generated_, image_, inline_, literal_, problematic_, reference_, strong_, subscript_, substitution_reference_, superscript_, target_, title_reference_, raw_ .. _HTML: http://www.w3.org/MarkUp/ .. _DocBook: http://docbook.org/tdg/en/html/docbook.html .. _XMLSpec: http://www.w3.org/XML/1998/06/xmlspec-report.htm ------------------- Element Reference ------------------- .. contents:: :local: :depth: 1 Each element in the DTD (document type definition) is described in its own section below. Each section contains an introduction plus the following subsections: * Details (of element relationships and semantics): - Category: One or more references to the element categories in `Element Hierarchy`_ above. Some elements belong to more than one category. - Parents: A list of elements which may contain the element. - Children: A list of elements which may occur within the element. - Analogues: Describes analogous elements in well-known document models such as HTML_ or DocBook_. Lists similarities and differences. - Processing: Lists formatting or rendering recommendations for the element. * Content Model: The formal XML content model from the `Docutils DTD`_, followed by: - Attributes: Describes (or refers to descriptions of) the possible values and semantics of each attribute. - Parameter Entities: Lists the parameter entities which directly or indirectly include the element. * Examples: reStructuredText_ examples are shown along with fragments of the document trees resulting from parsing. _`Pseudo-XML` is used for the results of parsing and processing. Pseudo-XML is a representation of XML where nesting is indicated by indentation and end-tags are not shown. Some of the precision of real XML is given up in exchange for easier readability. For example, the following are equivalent: - Real XML::
A Title A paragraph.
- Pseudo-XML::
A Title <paragraph> A paragraph. -------------------- Many of the element reference sections below are marked "_`to be completed`". Please help complete this document by contributing to its writing. ``abbreviation`` ================ `To be completed`_. ``acronym`` =========== `To be completed`_. ``address`` =========== The ``address`` element holds the surface mailing address information for the author (individual or group) of the document, or a third-party contact address. Its structure is identical to that of the line_block_ element: whitespace is significant, especially newlines. Details ------- :Category: `Bibliographic Elements`_ :Parents: The following elements may contain ``address``: docinfo_, authors_ :Children: ``address`` elements contain text data plus `inline elements`_. :Analogues: ``address`` is analogous to the DocBook "address" element. :Processing: As with the line_block_ element, newlines and other whitespace is significant and must be preserved. However, a monospaced typeface need not be used. See also docinfo_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``address`` element contains the `common attributes`_ (id_, name_, dupname_, source_, and class_), plus `xml:space`_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``address``. Examples -------- reStructuredText_ source:: Document Title ============== :Address: 123 Example Ave. Example, EX Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <address> 123 Example Ave. Example, EX See docinfo_ for a more complete example, including processing context. ``admonition`` ============== This element is a generic, titled admonition. Also see the specific admonition elements Docutils offers (in alphabetical order): caution_, danger_, error_, hint_, important_, note_, tip_, warning_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``admonition``. :Children: ``admonition`` elements begin with a title_ and may contain one or more `body elements`_. :Analogues: ``admonition`` has no direct analogues in common DTDs. It can be emulated with primitives and type effects. :Processing: Rendered distinctly (inset and/or in a box, etc.). Content Model ------------- .. parsed-literal:: (title_, (`%body.elements;`_)+) :Attributes: The ``admonition`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``admonition``. The `%structure.model;`_ parameter entity indirectly includes ``admonition``. Examples -------- reStructuredText source:: .. admonition:: And, by the way... You can make up your own admonition too. Pseudo-XML_ fragment from simple parsing:: <admonition class="admonition-and-by-the-way"> <title> And, by the way... <paragraph> You can make up your own admonition too. ``attention`` ============= The ``attention`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): caution_, danger_, error_, hint_, important_, note_, tip_, warning_, and the generic admonition_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``attention``. :Children: ``attention`` elements contain one or more `body elements`_. :Analogues: ``attention`` has no direct analogues in common DTDs. It can be emulated with primitives and type effects. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "Attention!" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``attention`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``attention``. The `%structure.model;`_ parameter entity indirectly includes ``attention``. Examples -------- reStructuredText source:: .. Attention:: All your base are belong to us. Pseudo-XML_ fragment from simple parsing:: <attention> <paragraph> All your base are belong to us. ``attribution`` =============== `To be completed`_. ``author`` ========== The ``author`` element holds the name of the author of the document. Details ------- :Category: `Bibliographic Elements`_ :Parents: The following elements may contain ``author``: docinfo_, authors_ :Children: ``author`` elements may contain text data plus `inline elements`_. :Analogues: ``author`` is analogous to the DocBook "author" element. :Processing: See docinfo_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``author`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``author``. Examples -------- reStructuredText_ source:: Document Title ============== :Author: J. Random Hacker Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <author> J. Random Hacker See docinfo_ for a more complete example, including processing context. ``authors`` =========== The ``authors`` element is a container for author information for documents with multiple authors. Details ------- :Category: `Bibliographic Elements`_ :Parents: Only the docinfo_ element contains ``authors``. :Children: ``authors`` elements may contain the following elements: author_, organization_, address_, contact_ :Analogues: ``authors`` is analogous to the DocBook "authors" element. :Processing: See docinfo_. Content Model ------------- .. parsed-literal:: ((author_, organization_?, address_?, contact_?)+) :Attributes: The ``authors`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``authors``. Examples -------- reStructuredText_ source:: Document Title ============== :Authors: J. Random Hacker; Jane Doe Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <authors> <author> J. Random Hacker <author> Jane Doe In reStructuredText, multiple author's names are separated with semicolons (";") or commas (","); semicolons take precedence. There is currently no way to represent the author's organization, address, or contact in a reStructuredText "Authors" field. See docinfo_ for a more complete example, including processing context. ``block_quote`` =============== The ``block_quote`` element is used for quotations set off from the main text (standalone). Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``block_quote``. :Children: ``block_quote`` elements contain `body elements`_ followed by an optional attribution_ element. :Analogues: ``block_quote`` is analogous to the "blockquote" element in both HTML and DocBook. :Processing: ``block_quote`` elements serve to set their contents off from the main text, typically with indentation and/or other decoration. Content Model ------------- .. parsed-literal:: ((`%body.elements;`_)+, attribution_?) :Attributes: The ``block_quote`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``block_quote``. The `%structure.model;`_ parameter entity indirectly includes ``block_quote``. Examples -------- reStructuredText source:: As a great paleontologist once said, This theory, that is mine, is mine. -- Anne Elk (Miss) Pseudo-XML_ fragment from simple parsing:: <paragraph> As a great paleontologist once said, <block_quote> <paragraph> This theory, that is mine, is mine. <attribution> Anne Elk (Miss) ``bullet_list`` =============== The ``bullet_list`` element contains list_item_ elements which are uniformly marked with bullets. Bullets are typically simple dingbats (symbols) such as circles and squares. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``bullet_list``. :Children: ``bullet_list`` elements contain one or more list_item_ elements. :Analogues: ``bullet_list`` is analogous to the HTML "ul" element and to the DocBook "itemizedlist" element. HTML's "ul" is short for "unordered list", which we consider to be a misnomer. "Unordered" implies that the list items may be randomly rearranged without affecting the meaning of the list. Bullet lists *are* often ordered; the ordering is simply left implicit. :Processing: Each list item should begin a new vertical block, prefaced by a bullet/dingbat. Content Model ------------- .. parsed-literal:: (list_item_ +) :Attributes: The ``bullet_list`` element contains the `common attributes`_ (id_, name_, dupname_, source_, and class_), plus bullet_. ``bullet`` is used to record the style of bullet from the input data. In documents processed from reStructuredText_, it contains one of "-", "+", or "*". It may be ignored in processing. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``bullet_list``. The `%structure.model;`_ parameter entity indirectly includes ``bullet_list``. Examples -------- reStructuredText_ source:: - Item 1, paragraph 1. Item 1, paragraph 2. - Item 2. Pseudo-XML_ fragment from simple parsing:: <bullet_list bullet="-"> <list_item> <paragraph> Item 1, paragraph 1. <paragraph> Item 1, paragraph 2. <list_item> <paragraph> Item 2. See list_item_ for another example. ``caption`` =========== `To be completed`_. ``caution`` =========== The ``caution`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention_, danger_, error_, hint_, important_, note_, tip_, warning_, and the generic admonition_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``caution``. :Children: ``caution`` elements contain one or more `body elements`_. :Analogues: ``caution`` is analogous to the DocBook "caution" element. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "Caution" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``caution`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``caution``. The `%structure.model;`_ parameter entity indirectly includes ``caution``. Examples -------- reStructuredText source:: .. Caution:: Don't take any wooden nickels. Pseudo-XML_ fragment from simple parsing:: <caution> <paragraph> Don't take any wooden nickels. ``citation`` ============ `To be completed`_. ``citation_reference`` ====================== `To be completed`_. ``classifier`` ============== The ``classifier`` element contains the classification or type of the term_ being defined in a definition_list_. For example, it can be used to indicate the type of a variable. Details ------- :Category: `Body Subelements`_ (simple) :Parents: Only the definition_list_item_ element contains ``classifier``. :Children: ``classifier`` elements may contain text data plus `inline elements`_. :Analogues: ``classifier`` has no direct analogues in common DTDs. It can be emulated with primitives or type effects. :Processing: See definition_list_item_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``classifier`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- Here is a hypothetical data dictionary. reStructuredText_ source:: name : string Customer name. i : int Temporary index variable. Pseudo-XML_ fragment from simple parsing:: <definition_list> <definition_list_item> <term> name <classifier> string <definition> <paragraph> Customer name. <definition_list_item> <term> i <classifier> int <definition> <paragraph> Temporary index variable. ``colspec`` =========== `To be completed`_. ``comment`` =========== `To be completed`_. ``contact`` =========== The ``contact`` element holds contact information for the author (individual or group) of the document, or a third-party contact. It is typically used for an email or web address. Details ------- :Category: `Bibliographic Elements`_ :Parents: The following elements may contain ``contact``: docinfo_, authors_ :Children: ``contact`` elements may contain text data plus `inline elements`_. :Analogues: ``contact`` is analogous to the DocBook "email" element. The HTML "address" element serves a similar purpose. :Processing: See docinfo_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``contact`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``contact``. Examples -------- reStructuredText_ source:: Document Title ============== :Contact: jrh@example.com Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <contact> <reference refuri="mailto:jrh@example.com"> jrh@example.com See docinfo_ for a more complete example, including processing context. ``copyright`` ============= The ``copyright`` element contains the document's copyright statement. Details ------- :Category: `Bibliographic Elements`_ :Parents: Only the docinfo_ element contains ``copyright``. :Children: ``copyright`` elements may contain text data plus `inline elements`_. :Analogues: ``copyright`` is analogous to the DocBook "copyright" element. :Processing: See docinfo_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``copyright`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``copyright``. Examples -------- reStructuredText_ source:: Document Title ============== :Copyright: This document has been placed in the public domain. Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <copyright> This document has been placed in the public domain. See docinfo_ for a more complete example, including processing context. ``danger`` ========== The ``danger`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention_, caution_, error_, hint_, important_, note_, tip_, warning_, and the generic admonition_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``danger``. :Children: ``danger`` elements contain one or more `body elements`_. :Analogues: ``danger`` has no direct analogues in common DTDs. It can be emulated with primitives and type effects. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "!DANGER!" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``danger`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``danger``. The `%structure.model;`_ parameter entity indirectly includes ``danger``. Examples -------- reStructuredText source:: .. DANGER:: Mad scientist at work! Pseudo-XML_ fragment from simple parsing:: <danger> <paragraph> Mad scientist at work! ``date`` ======== The ``date`` element contains the date of publication, release, or last modification of the document. Details ------- :Category: `Bibliographic Elements`_ :Parents: Only the docinfo_ element contains ``date``. :Children: ``date`` elements may contain text data plus `inline elements`_. :Analogues: ``date`` is analogous to the DocBook "date" element. :Processing: Often used with the RCS/CVS keyword "Date". See docinfo_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``date`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``date``. Examples -------- reStructuredText_ source:: Document Title ============== :Date: 2002-08-20 Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <date> 2002-08-20 See docinfo_ for a more complete example, including processing context. ``decoration`` ============== The ``decoration`` element is a container for header_ and footer_ elements and potential future extensions. These elements are used for page navigation, notes, time/datestamp, etc. Currently only the footer_ element is implemented, populated with processing information (datestamp, a link to Docutils_, etc.). Details ------- :Category: `Structural Subelements`_ :Parents: Only the document_ element contains ``decoration``. :Children: ``decoration`` elements may contain `decorative elements`_. :Analogues: There are no direct analogies to ``decoration`` in HTML or in DocBook. Equivalents are typically constructed from primitives and/or generated by the processing system. :Processing: See the individual `decorative elements`_. Content Model ------------- .. parsed-literal:: (header_?, footer_?) Although the content model doesn't specifically require contents, no empty ``decoration`` elements are ever created. :Attributes: The ``decoration`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- reStructuredText_ source:: A paragraph. Complete pseudo-XML_ result after parsing and applying transforms, assuming that the datestamp command-line option or configuration setting has been supplied:: <document> <decoration> <footer> <paragraph> Generated on: 2002-08-20. <paragraph> A paragraph. ``definition`` ============== The ``definition`` element is a container for the body elements used to define a term_ in a definition_list_. Details ------- :Category: `Body Subelements`_ (compound) :Parents: Only definition_list_item_ elements contain ``definition``. :Children: ``definition`` elements may contain `body elements`_. :Analogues: ``definition`` is analogous to the HTML "dd" element and to the DocBook "listitem" element (inside a "variablelistentry" element). :Processing: See definition_list_item_. Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``definition`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the definition_list_, definition_list_item_, and classifier_ elements. ``definition_list`` =================== The ``definition_list`` element contains a list of terms and their definitions. It can be used for glossaries or dictionaries, to describe or classify things, for dialogues, or to itemize subtopics (such as in this reference). Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``definition_list``. :Children: ``definition_list`` elements contain one or more definition_list_item_ elements. :Analogues: ``definition_list`` is analogous to the HTML "dl" element and to the DocBook "variablelist" element. :Processing: See definition_list_item_. Content Model ------------- .. parsed-literal:: (definition_list_item_ +) :Attributes: The ``definition_list`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``definition_list``. The `%structure.model;`_ parameter entity indirectly includes ``definition_list``. Examples -------- reStructuredText_ source:: Term Definition. Term : classifier The ' : ' indicates a classifier in definition list item terms only. Pseudo-XML_ fragment from simple parsing:: <definition_list> <definition_list_item> <term> Term <definition> <paragraph> Definition. <definition_list_item> <term> Term <classifier> classifier <definition> <paragraph> The ' : ' indicates a classifier in definition list item terms only. See definition_list_item_ and classifier_ for further examples. ``definition_list_item`` ======================== The ``definition_list_item`` element contains a single term_/definition_ pair (with optional classifier_). Details ------- :Category: `Body Subelements`_ (compound) :Parents: Only the definition_list_ element contains ``definition_list_item``. :Children: ``definition_list_item`` elements each contain a single term_, an optional classifier_, and a definition_. :Analogues: ``definition_list_item`` is analogous to the DocBook "variablelistentry" element. :Processing: The optional classifier_ can be rendered differently from the term_. They should be separated visually, typically by spaces plus a colon or dash. Content Model ------------- .. parsed-literal:: (term_, classifier_?, definition_) :Attributes: The ``definition_list_item`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- reStructuredText_ source:: Tyrannosaurus Rex : carnivore Big and scary; the "Tyrant King". Brontosaurus : herbivore All brontosauruses are thin at one end, much much thicker in the middle and then thin again at the far end. -- Anne Elk (Miss) Pseudo-XML_ fragment from simple parsing:: <definition_list> <definition_list_item> <term> Tyrannosaurus Rex <classifier> carnivore <definition> <paragraph> Big and scary; the "Tyrant King". <definition_list_item> <term> Brontosaurus <classifier> herbivore <definition> <paragraph> All brontosauruses are thin at one end, much much thicker in the middle and then thin again at the far end. <paragraph> -- Anne Elk (Miss) See definition_list_ and classifier_ for further examples. ``description`` =============== The ``description`` element contains body elements, describing the purpose or effect of a command-line option or group of options. Details ------- :Category: `Body Subelements`_ :Parents: Only the option_list_item_ element contains ``description``. :Children: ``description`` elements may contain `body elements`_. :Analogues: ``description`` has no direct analogues in common DTDs. :Processing: See option_list_. Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``description`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the option_list_ element. ``docinfo`` =========== The ``docinfo`` element is a container for document bibliographic data, or meta-data (data about the document). It corresponds to the front matter of a book, such as the title page and copyright page. Details ------- :Category: `Structural Subelements`_ :Parents: Only the document_ element contains ``docinfo``. :Children: ``docinfo`` elements contain `bibliographic elements`_. :Analogues: ``docinfo`` is analogous to DocBook "info" elements ("bookinfo" etc.). There are no directly analogous HTML elements; the "meta" element carries some of the same information, albeit invisibly. :Processing: The ``docinfo`` element may be rendered as a two-column table or in other styles. It may even be invisible or omitted from the processed output. Meta-data may be extracted from ``docinfo`` children; for example, HTML ``<meta>`` tags may be constructed. When Docutils_ transforms a reStructuredText_ field_list_ into a ``docinfo`` element (see the examples below), RCS/CVS keywords are normally stripped from simple (one paragraph) field bodies. For complete details, please see `RCS Keywords`_ in the `reStructuredText Markup Specification`_. .. _RCS Keywords: rst/restructuredtext.html#rcs-keywords Content Model ------------- .. parsed-literal:: (`%bibliographic.elements;`_)+ :Attributes: The ``docinfo`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- Docinfo is represented in reStructuredText_ by a field_list_ in a bibliographic context: the first non-comment element of a document_, after any document title_/subtitle_. The field list is transformed into a ``docinfo`` element and its children by a transform. Source:: Docinfo Example =============== :Author: J. Random Hacker :Contact: jrh@example.com :Date: 2002-08-18 :Status: Work In Progress :Version: 1 :Filename: $RCSfile: doctree.txt,v $ :Copyright: This document has been placed in the public domain. Complete pseudo-XML_ result after parsing and applying transforms:: <document id="docinfo-example" name="docinfo example"> <title> Docinfo Example <docinfo> <author> J. Random Hacker <contact> <reference refuri="mailto:jrh@example.com"> jrh@example.com <date> 2002-08-18 <status> Work In Progress <version> 1 <field> <field_name> Filename <field_body> <paragraph> doctree.txt <copyright> This document has been placed in the public domain. Note that "Filename" is a non-standard ``docinfo`` field, so becomes a generic ``field`` element. Also note that the "RCSfile" keyword syntax has been stripped from the "Filename" data. See field_list_ for an example in a non-bibliographic context. Also see the individual examples for the various `bibliographic elements`_. ``doctest_block`` ================= The ``doctest_block`` element is a Python-specific variant of literal_block_. It is a block of text where line breaks and whitespace are significant and must be preserved. ``doctest_block`` elements are used for interactive Python interpreter sessions, which are distinguished by their input prompt: ``>>>``. They are meant to illustrate usage by example, and provide an elegant and powerful testing environment via the `doctest module`_ in the Python standard library. .. _doctest module: http://www.python.org/doc/current/lib/module-doctest.html Details ------- :Category: `Simple Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``doctest_block``. :Children: ``doctest_block`` elements may contain text data plus `inline elements`_. :Analogues: ``doctest_block`` is analogous to the HTML "pre" element and to the DocBook "programlisting" and "screen" elements. :Processing: As with literal_block_, ``doctest_block`` elements are typically rendered in a monospaced typeface. It is crucial that all whitespace and line breaks are preserved in the rendered form. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``doctest_block`` element contains the `common attributes`_ (id_, name_, dupname_, source_, and class_), plus `xml:space`_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``doctest_block``. The `%structure.model;`_ parameter entity indirectly includes ``doctest_block``. Examples -------- reStructuredText source:: This is an ordinary paragraph. >>> print 'this is a Doctest block' this is a Doctest block Pseudo-XML_ fragment from simple parsing:: <paragraph> This is an ordinary paragraph. <doctest_block xml:space="preserve"> >>> print 'this is a Doctest block' this is a Doctest block ``document`` ============ The ``document`` element is the root (topmost) element of the Docutils document tree. ``document`` is the direct or indirect ancestor of every other element in the tree. It encloses the entire document tree. It is the starting point for a document. Details ------- :Category: `Structural Elements`_ :Parents: The ``document`` element has no parents. :Children: ``document`` elements may contain `structural subelements`_, `structural elements`_, and `body elements`_. :Analogues: ``document`` is analogous to the HTML "html" element and to several DocBook elements such as "book". Content Model ------------- .. parsed-literal:: ((title_, subtitle_?)?, docinfo_?, decoration_?, `%structure.model;`_) Depending on the source of the data and the stage of processing, the "document" may not initially contain a "title". A document title is not directly representable in reStructuredText_. Instead, a lone top-level section may have its title promoted to become the document title_, and similarly for a lone second-level (sub)section's title to become the document subtitle_. The "docinfo_" may be transformed from an initial field_list_, and "decoration_" is usually constructed programmatically. See the `%structure.model;`_ parameter entity for details of the body of a ``document``. :Attributes: The ``document`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- reStructuredText_ source:: A Title ======= A paragraph. Complete pseudo-XML_ result from simple parsing:: <document> <section id="a-title" name="a title"> <title> A Title <paragraph> A paragraph. After applying transforms, the section title is promoted to become the document title:: <document id="a-title" name="a title"> <title> A Title <paragraph> A paragraph. ``emphasis`` ============ `To be completed`_. ``entry`` ========= `To be completed`_. ``enumerated_list`` =================== The ``enumerated_list`` element contains list_item_ elements which are uniformly marked with enumerator labels. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``enumerated_list``. :Children: ``enumerated_list`` elements contain one or more list_item_ elements. :Analogues: ``enumerated_list`` is analogous to the HTML "ol" element and to the DocBook "orderedlist" element. :Processing: Each list item should begin a new vertical block, prefaced by a enumeration marker (such as "1."). Content Model ------------- .. parsed-literal:: (list_item_ +) :Attributes: The ``enumerated_list`` element contains the `common attributes`_ (id_, name_, dupname_, source_, and class_), plus enumtype_, prefix_, suffix_, and start_. ``enumtype`` is used to record the intended enumeration sequence, one of "arabic" (1, 2, 3, ...), "loweralpha" (a, b, c, ..., z), "upperalpha" (A, B, C, ..., Z), "lowerroman" (i, ii, iii, iv, ..., mmmmcmxcix [4999]), or "upperroman" (I, II, III, IV, ..., MMMMCMXCIX [4999]). ``prefix`` stores the formatting characters used before the enumerator. In documents originating from reStructuredText_ data, it will contain either "" (empty string) or "(" (left parenthesis). It may or may not affect processing. ``suffix`` stores the formatting characters used after the enumerator. In documents originating from reStructuredText_ data, it will contain either "." (period) or ")" (right parenthesis). Depending on the capabilities of the output format, this attribute may or may not affect processing. ``start`` contains the ordinal value of the first item in the list, in decimal. For lists beginning at value 1 ("1", "a", "A", "i", or "I"), this attribute may be omitted. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``enumerated_list``. The `%structure.model;`_ parameter entity indirectly includes ``enumerated_list``. Examples -------- reStructuredText_ source:: 1. Item 1. (A) Item A. (B) Item B. (C) Item C. 2. Item 2. Pseudo-XML_ fragment from simple parsing:: <enumerated_list enumtype="arabic" prefix="" suffix="."> <list_item> <paragraph> Item 1. <enumerated_list enumtype="upperalpha" prefix="(" suffix=")"> <list_item> <paragraph> Item A. <list_item> <paragraph> Item B. <list_item> <paragraph> Item C. <list_item> <paragraph> Item 2. See list_item_ for another example. ``error`` ========= The ``error`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention_, caution_, danger_, hint_, important_, note_, tip_, warning_, and the generic admonition_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``error``. :Children: ``error`` elements contain one or more `body elements`_. :Analogues: ``error`` has no direct analogues in common DTDs. It can be emulated with primitives and type effects. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "Error" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``error`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``error``. The `%structure.model;`_ parameter entity indirectly includes ``error``. Examples -------- reStructuredText source:: .. Error:: Does not compute. Pseudo-XML_ fragment from simple parsing:: <error> <paragraph> Does not compute. ``field`` ========= The ``field`` element contains a pair of field_name_ and field_body_ elements. Details ------- :Category: `Body Subelements`_ :Parents: The following elements may contain ``field``: docinfo_, field_list_ :Children: Each ``field`` element contains one field_name_ and one field_body_ element. :Analogues: ``field`` has no direct analogues in common DTDs. :Processing: See field_list_. Content Model ------------- .. parsed-literal:: (field_name_, field_body_) :Attributes: The ``field`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``field``. Examples -------- See the examples for the field_list_ and docinfo_ elements. ``field_body`` ============== The ``field_body`` element contains body elements. It is analogous to a database field's data. Details ------- :Category: `Body Subelements`_ :Parents: Only the field_ element contains ``field_body``. :Children: ``field_body`` elements may contain `body elements`_. :Analogues: ``field_body`` has no direct analogues in common DTDs. :Processing: See field_list_. Content Model ------------- .. parsed-literal:: (`%body.elements;`_)* :Attributes: The ``field_body`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the field_list_ and docinfo_ elements. ``field_list`` ============== The ``field_list`` element contains two-column table-like structures resembling database records (label & data pairs). Field lists are often meant for further processing. In reStructuredText_, field lists are used to represent bibliographic fields (contents of the docinfo_ element) and directive options. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``field_list``. :Children: ``field_list`` elements contain one or more field_ elements. :Analogues: ``field_list`` has no direct analogues in common DTDs. It can be emulated with primitives such as tables. :Processing: A ``field_list`` is typically rendered as a two-column list, where the first column contains "labels" (usually with a colon suffix). However, field lists are often used for extension syntax or special processing. Such structures do not survive as field lists to be rendered. Content Model ------------- .. parsed-literal:: (field_ +) :Attributes: The ``field_list`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``field_list``. The `%structure.model;`_ parameter entity indirectly includes ``field_list``. Examples -------- reStructuredText_ source:: :Author: Me :Version: 1 :Date: 2001-08-11 :Parameter i: integer Pseudo-XML_ fragment from simple parsing:: <field_list> <field> <field_name> Author <field_body> <paragraph> Me <field> <field_name> Version <field_body> <paragraph> 1 <field> <field_name> Date <field_body> <paragraph> 2001-08-11 <field> <field_name> Parameter i <field_body> <paragraph> integer ``field_name`` ============== The ``field_name`` element contains text; it is analogous to a database field's name. Details ------- :Category: `Body Subelements`_ (simple) :Parents: Only the field_ element contains ``field_name``. :Children: ``field_name`` elements may contain text data plus `inline elements`_. :Analogues: ``field_name`` has no direct analogues in common DTDs. :Processing: See field_list_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``field_name`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the field_list_ and docinfo_ elements. ``figure`` ========== `To be completed`_. ``footer`` ========== The ``footer`` element is a container element whose contents are meant to appear at the bottom of a web page, or repeated at the bottom of every printed page. Currently the ``footer`` element may contain processing information (datestamp, a link to Docutils_, etc.). Details ------- :Category: `Decorative Elements`_ :Parents: Only the decoration_ element contains ``footer``. :Children: ``footer`` elements may contain `body elements`_. :Analogues: There are no direct analogies to ``footer`` in HTML or DocBook. Equivalents are typically constructed from primitives and/or generated by the processing system. Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``footer`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- reStructuredText_ source:: A paragraph. Complete pseudo-XML_ result after parsing and applying transforms, assuming that the datestamp command-line option or configuration setting has been supplied:: <document> <decoration> <footer> <paragraph> Generated on: 2002-08-20. <paragraph> A paragraph. ``footnote`` ============ `To be completed`_. ``footnote_reference`` ====================== `To be completed`_. ``generated`` ============= Docutils wraps ``generated`` elements around text that is inserted (generated) by Docutils; i.e., text that was not in the document, like section numbers inserted by the "sectnum" directive. `To be completed`_. ``header`` ========== The ``header`` element is a container element whose contents are meant to appear at the top of a web page, or at the top of every printed page. Docutils does not yet make use of the ``header`` element. Details ------- :Category: `Decorative Elements`_ :Parents: Only the decoration_ element contains ``header``. :Children: ``header`` elements may contain `body elements`_. :Analogues: There are no direct analogies to ``header`` in HTML or DocBook. Equivalents are typically constructed from primitives and/or generated by the processing system. Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``header`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- None. ``hint`` ======== The ``hint`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention_, caution_, danger_, error_, important_, note_, tip_, warning_, and the generic admonition_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``hint``. :Children: ``hint`` elements contain one or more `body elements`_. :Analogues: ``hint`` has no direct analogues in common DTDs. It can be emulated with primitives and type effects. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "Hint" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``hint`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``hint``. The `%structure.model;`_ parameter entity indirectly includes ``hint``. Examples -------- reStructuredText source:: .. Hint:: It's bigger than a bread box. Pseudo-XML_ fragment from simple parsing:: <hint> <paragraph> It's bigger than a bread box. ``image`` ========= `To be completed`_. ``important`` ============= The ``important`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention_, caution_, danger_, error_, hint_, note_, tip_, warning_, and the generic admonition_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``important``. :Children: ``important`` elements contain one or more `body elements`_. :Analogues: ``important`` is analogous to the DocBook "important" element. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "Important" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``important`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``important``. The `%structure.model;`_ parameter entity indirectly includes ``important``. Examples -------- reStructuredText source:: .. Important:: * Wash behind your ears. * Clean up your room. * Back up your data. * Call your mother. Pseudo-XML_ fragment from simple parsing:: <important> <bullet_list> <list_item> <paragraph> Wash behind your ears. <list_item> <paragraph> Clean up your room. <list_item> <paragraph> Back up your data. <list_item> <paragraph> Call your mother. ``inline`` ========== `To be completed`_. ``label`` ========= `To be completed`_. ``legend`` ========== `To be completed`_. ``line_block`` ============== The ``line_block`` element contains a block of text where line breaks and whitespace are significant and must be preserved. ``line_block`` elements are commonly used for verse and addresses. See `line_block`_ for an alternative useful for program listings and interactive computer sessions. Details ------- :Category: `Simple Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``line_block``. :Children: ``line_block`` elements may contain text data plus `inline elements`_. :Analogues: ``line_block`` is analogous to the DocBook "literallayout" element and to the HTML "pre" element (with modifications to typeface styles). :Processing: Unline ``literal_block``, ``line_block`` elements are typically rendered in an ordinary text typeface. It is crucial that all whitespace and line breaks are preserved in the rendered form. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``line_block`` element contains the `common attributes`_ (id_, name_, dupname_, source_, and class_), plus `xml:space`_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``line_block``. The `%structure.model;`_ parameter entity indirectly includes ``line_block``. Examples -------- reStructuredText uses a directive to indicate a ``line_block``. Example source:: Take it away, Eric the Orchestra Leader! .. line-block:: A one, two, a one two three four Half a bee, philosophically, must, *ipso facto*, half not be. But half the bee has got to be, *vis a vis* its entity. D'you see? But can a bee be said to be or not to be an entire bee, when half the bee is not a bee, due to some ancient injury? Singing... Pseudo-XML_ fragment from simple parsing:: <paragraph> Take it away, Eric the Orchestra Leader! <line_block xml:space="preserve"> A one, two, a one two three four Half a bee, philosophically, must, <emphasis>ipso facto</emphasis>, half not be. But half the bee has got to be, <emphasis>vis a vis</emphasis>its entity. D'you see? But can a bee be said to be or not to be an entire bee, when half the bee is not a bee, due to some ancient injury? Singing... ``list_item`` ============= The ``list_item`` element is a container for the elements of a list item. Details ------- :Category: `Body Subelements`_ (compound) :Parents: The bullet_list_ and enumerated_list_ elements contain ``list_item``. :Children: ``list_item`` elements may contain `body elements`_. :Analogues: ``list_item`` is analogous to the HTML "li" element and to the DocBook "listitem" element. :Processing: See bullet_list_ or enumerated_list_. Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``list_item`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- reStructuredText_ source:: 1. Outer list, item 1. * Inner list, item 1. * Inner list, item 2. 2. Outer list, item 2. Pseudo-XML_ fragment from simple parsing:: <enumerated_list enumtype="arabic" prefix="" suffix="."> <list_item> <paragraph> Outer list, item 1. <bullet_list bullet="*"> <list_item> <paragraph> Inner list, item 1. <list_item> <paragraph> Inner list, item 2. <list_item> <paragraph> Outer list, item 2. See bullet_list_ or enumerated_list_ for further examples. ``literal`` =========== `To be completed`_. ``literal_block`` ================= The ``literal_block`` element contains a block of text where line breaks and whitespace are significant and must be preserved. ``literal_block`` elements are commonly used for program listings and interactive computer sessions. See `line_block`_ for an alternative useful for verse and addresses. Details ------- :Category: `Simple Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``literal_block``. :Children: ``literal_block`` elements may contain text data plus `inline elements`_. :Analogues: ``literal_block`` is analogous to the HTML "pre" element and to the DocBook "programlisting" and "screen" elements. :Processing: ``literal_block`` elements are typically rendered in a monospaced typeface. It is crucial that all whitespace and line breaks are preserved in the rendered form. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``literal_block`` element contains the `common attributes`_ (id_, name_, dupname_, source_, and class_), plus `xml:space`_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``literal_block``. The `%structure.model;`_ parameter entity indirectly includes ``literal_block``. Examples -------- reStructuredText source:: Here is a literal block:: if literal_block: text = 'is left as-is' spaces_and_linebreaks = 'are preserved' markup_processing = None Pseudo-XML_ fragment from simple parsing:: <paragraph> Here is a literal block: <literal_block xml:space="preserve"> if literal_block: text = 'is left as-is' spaces_and_linebreaks = 'are preserved' markup_processing = None ``note`` ======== The ``note`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention_, caution_, danger_, error_, hint_, important_, tip_, warning_, and the generic admonition_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``note``. :Children: ``note`` elements contain one or more `body elements`_. :Analogues: ``note`` is analogous to the DocBook "note" element. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "Note" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``note`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``note``. The `%structure.model;`_ parameter entity indirectly includes ``note``. Examples -------- reStructuredText source:: .. Note:: Admonitions can be handy to break up a long boring technical document. Pseudo-XML_ fragment from simple parsing:: <note> <paragraph> Admonitions can be handy to break up a long boring technical document. ``option`` ========== The ``option`` element groups an option string together with zero or more option argument placeholders. Note that reStructuredText_ currently supports only one argument per option. Details ------- :Category: `Body Subelements`_ :Parents: Only the option_group_ element contains ``option``. :Children: Each ``option`` element contains one option_string_ and zero or more option_argument_ elements. :Analogues: ``option`` has no direct analogues in common DTDs. :Processing: See option_list_. Content Model ------------- .. parsed-literal:: (option_string_, option_argument_ \*) :Attributes: The ``option`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the option_list_ element. ``option_argument`` =================== The ``option_argument`` element contains placeholder text for option arguments. Details ------- :Category: `Body Subelements`_ :Parents: Only the option_ element contains ``option_argument``. :Children: ``option_argument`` elements contain text data only. :Analogues: ``option_argument`` has no direct analogues in common DTDs. :Processing: The value of the "delimiter" attribute is prefixed to the ``option_argument``, separating it from its option_string_ or a preceding ``option_argument``. The ``option_argument`` text is typically rendered in a monospaced typeface, possibly italicized or otherwise altered to indicate its placeholder nature. Content Model ------------- .. parsed-literal:: (#PCDATA) :Attributes: The ``option_argument`` element contains the `common attributes`_ (id_, name_, dupname_, source_, and class_), plus delimiter_. ``delimiter`` contains the text preceding the ``option_argument``: either the text separating it from the option_string_ (typically either "=" or " ") or the text between option arguments (typically either "," or " "). Examples -------- See the examples for the option_list_ element. ``option_group`` ================ The ``option_group`` element groups together one or more option_ elements, all synonyms. Details ------- :Category: `Body Subelements`_ :Parents: Only the option_list_item_ element contains ``option_group``. :Children: ``option_group`` elements contain one or more option_ elements. ``option_group`` is an empty element and has no children. Each ``option_group`` element contains one _ and one _ element. :Analogues: ``option_group`` has no direct analogues in common DTDs. :Processing: Typically option_ elements within an ``option_group`` are joined together in a comma-separated list. Content Model ------------- .. parsed-literal:: (option_group_, description_) :Attributes: The ``option_group`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the option_list_ element. ``option_list`` =============== Each ``option_list`` element contains a two-column list of command-line options and descriptions, documenting a program's options. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``option_list``. :Children: ``option_list`` elements contain one or more option_list_item_ elements. :Analogues: ``option_list`` has no direct analogues in common DTDs. It can be emulated with primitives such as tables. :Processing: An ``option_list`` is typically rendered as a two-column list, where the first column contains option strings and arguments, and the second column contains descriptions. Content Model ------------- .. parsed-literal:: (option_list_item_ +) :Attributes: The ``option_list`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``option_list``. The `%structure.model;`_ parameter entity indirectly includes ``option_list``. Examples -------- reStructuredText_ source:: -a command-line option "a" -1 file, --one=file, --two file Multiple options with arguments. Pseudo-XML_ fragment from simple parsing:: <option_list> <option_list_item> <option_group> <option> <option_string> -a <description> <paragraph> command-line option "a" <option_list_item> <option_group> <option> <option_string> -1 <option_argument delimiter=" "> file <option> <option_string> --one <option_argument delimiter="="> file <option> <option_string> --two <option_argument delimiter=" "> file <description> <paragraph> Multiple options with arguments. ``option_list_item`` ==================== The ``option_list_item`` element is a container for a pair of option_group_ and description_ elements. Details ------- :Category: `Body Subelements`_ :Parents: Only the option_list_ element contains ``option_list_item``. :Children: Each ``option_list_item`` element contains one option_group_ and one description_ element. :Analogues: ``option_list_item`` has no direct analogues in common DTDs. :Processing: See option_list_. Content Model ------------- .. parsed-literal:: (option_group_, description_) :Attributes: The ``option_list_item`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the option_list_ element. ``option_string`` ================= The ``option_string`` element contains the text of a command-line option. Details ------- :Category: `Body Subelements`_ :Parents: Only the option_ element contains ``option_string``. :Children: ``option_string`` elements contain text data only. :Analogues: ``option_string`` has no direct analogues in common DTDs. :Processing: The ``option_string`` text is typically rendered in a monospaced typeface. Content Model ------------- .. parsed-literal:: (#PCDATA) :Attributes: The ``option_string`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the option_list_ element. ``organization`` ================ The ``organization`` element contains the name of document author's organization, or the organization responsible for the document. Details ------- :Category: `Bibliographic Elements`_ :Parents: Only the docinfo_ element contains ``organization``. :Children: ``organization`` elements may contain text data plus `inline elements`_. :Analogues: ``organization`` is analogous to the DocBook "orgname", "corpname", or "publishername" elements. :Processing: See docinfo_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``organization`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``organization``. Examples -------- reStructuredText_ source:: Document Title ============== :Organization: Humankind Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <organization> Humankind See docinfo_ for a more complete example, including processing context. ``paragraph`` ============= The ``paragraph`` element contains the text and inline elements of a single paragraph, a fundamental building block of documents. Details ------- :Category: `Simple Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``paragraph``. :Children: ``paragraph`` elements may contain text data plus `inline elements`_. :Analogues: ``paragraph`` is analogous to the HTML "p" element and to the DocBook "para" elements. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``paragraph`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``paragraph``. The `%structure.model;`_ parameter entity indirectly includes ``paragraph``. Examples -------- reStructuredText_ source:: A paragraph. Pseudo-XML_ fragment from simple parsing:: <paragraph> A paragraph. ``pending`` =========== `To be completed`_. ``problematic`` =============== `To be completed`_. ``raw`` ======= `To be completed`_. ``reference`` ============= `To be completed`_. ``revision`` ============ The ``revision`` element contains the revision number of the document. It can be used alone or in conjunction with version_. Details ------- :Category: `Bibliographic Elements`_ :Parents: Only the docinfo_ element contains ``revision``. :Children: ``revision`` elements may contain text data plus `inline elements`_. :Analogues: ``revision`` is analogous to but simpler than the DocBook "revision" element. It closely matches the DocBook "revnumber" element, but in a simpler context. :Processing: Often used with the RCS/CVS keyword "Revision". See docinfo_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``revision`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``revision``. Examples -------- reStructuredText_ source:: Document Title ============== :Version: 1 :Revision: b Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <version> 1 <revision> b See docinfo_ for a more complete example, including processing context. ``row`` ======= `To be completed`_. ``rubric`` ========== rubric n. 1. a title, heading, or the like, in a manuscript, book, statute, etc., written or printed in red or otherwise distinguished from the rest of the text. ... -- Random House Webster's College Dictionary, 1991 A rubric is like an informal heading that doesn't correspond to the document's structure. `To be completed`_. ``section`` =========== The ``section`` element is the main unit of hierarchy for Docutils documents. Docutils ``section`` elements are a recursive structure; a ``section`` may contain other ``section`` elements, without limit. Paragraphs and other body elements may occur before a ``section``, but not after it. Details ------- :Category: `Structural Elements`_ :Parents: The following elements may contain ``section``: document_, section_ :Children: ``section`` elements begin with a title_, and may contain `body elements`_ as well as transition_, topic_, and sidebar_ elements. :Analogues: ``section`` is analogous to DocBook recursive "section" elements, and to HTML "div" elements combined with "h1" etc. title elements. Content Model ------------- .. parsed-literal:: (title_, `%structure.model;`_) See the `%structure.model;`_ parameter entity for details of the body of a ``section``. :Attributes: The ``section`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%section.elements;`_ parameter entity directly includes ``section``. The `%structure.model;`_ parameter entity indirectly includes ``section``. Examples -------- reStructuredText_ source:: Title 1 ======= Paragraph 1. Title 2 ------- Paragraph 2. Title 3 ======= Paragraph 3. Title 4 ------- Paragraph 4. Complete pseudo-XML_ result after parsing:: <document> <section id="title-1" name="title 1"> <title> Title 1 <paragraph> Paragraph 1. <section id="title-2" name="title 2"> <title> Title 2 <paragraph> Paragraph 2. <section id="title-3" name="title 3"> <title> Title 3 <paragraph> Paragraph 3. <section id="title-4" name="title 4"> <title> Title 4 <paragraph> Paragraph 4. ``sidebar`` =========== Sidebars are like miniature, parallel documents that occur inside other documents, providing related or reference material. A ``sidebar`` is typically offset by a border and "floats" to the side of the page; the document's main text may flow around it. Sidebars can also be likened to super-footnotes; their content is outside of the flow of the document's main text. The ``sidebar`` element is a nonrecursive section_-like construct which may occur at the top level of a section_ wherever a body element (list, table, etc.) is allowed. In other words, ``sidebar`` elements cannot nest inside body elements, so you can't have a ``sidebar`` inside a ``table`` or a ``list``, or inside another ``sidebar`` (or topic_). Details ------- :Category: `Structural Elements`_ :Parents: The following elements may contain ``sidebar``: document_, section_ :Children: ``sidebar`` elements begin with a title_ and an optional subtitle_ and contain `body elements`_. :Analogues: ``sidebar`` is analogous to the DocBook "sidebar" element. :Processing: A ``sidebar`` element should be set off from the rest of the document somehow, typically with a border. Sidebars typically "float" to the side of the page and the document's main text flows around them. Content Model ------------- .. parsed-literal:: (title_, subtitle_?, (`%body.elements;`_)+) :Attributes: The ``sidebar`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%structure.model;`_ parameter entity directly includes ``sidebar``. Examples -------- The `"sidebar" directive`_ is used to create a ``sidebar`` element. reStructuredText_ source:: .. sidebar:: Title :subtitle: If Desired Body. Pseudo-XML_ fragment from simple parsing:: <sidebar> <title> Title <subtitle> If Desired <paragraph> Body. .. _"sidebar" directive: rst/directives.html#sidebar ``status`` ========== The ``status`` element contains a status statement for the document, such as "Draft", "Final", "Work In Progress", etc. Details ------- :Category: `Bibliographic Elements`_ :Parents: Only the docinfo_ element contains ``status``. :Children: ``status`` elements may contain text data plus `inline elements`_. :Analogues: ``status`` is analogous to the DocBook "status" element. :Processing: See docinfo_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``status`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``status``. Examples -------- reStructuredText_ source:: Document Title ============== :Status: Work In Progress Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <status> Work In Progress See docinfo_ for a more complete example, including processing context. ``strong`` ========== `To be completed`_. ``subscript`` ============= `To be completed`_. ``substitution_definition`` =========================== `To be completed`_. ``substitution_reference`` ========================== `To be completed`_. ``subtitle`` ============ The ``subtitle`` element stores the subtitle of a document_. Details ------- :Category: `Structural Subelements`_ :Parents: The document_ and sidebar_ elements may contain ``subtitle``. :Children: ``subtitle`` elements may contain text data plus `inline elements`_. :Analogues: ``subtitle`` is analogous to HTML header elements ("h2" etc.) and to the DocBook "subtitle" element. :Processing: A document's subtitle is usually rendered smaller than its title_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``subtitle`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- reStructuredText_ source:: ======= Title ======= ---------- Subtitle ---------- A paragraph. Complete pseudo-XML_ result after parsing and applying transforms:: <document id="title" name="title"> <title> Title <subtitle id="subtitle" name="subtitle"> Subtitle <paragraph> A paragraph. Note how two section levels have collapsed, promoting their titles to become the document's title and subtitle. Since there is only one structural element (document), the subsection's ``id`` and ``name`` attributes are stored in the ``subtitle`` element. ``superscript`` =============== `To be completed`_. ``system_message`` ================== `To be completed`_. ``table`` ========= `To be completed`_. ``target`` ========== `To be completed`_. ``tbody`` ========= `To be completed`_. ``term`` ======== The ``term`` element contains a word or phrase being defined in a definition_list_. Details ------- :Category: `Body Subelements`_ (simple) :Parents: Only the definition_list_item_ element contains ``term``. :Children: ``term`` elements may contain text data plus `inline elements`_. :Analogues: ``term`` is analogous to the HTML "dt" element and to the DocBook "term" element. :Processing: See definition_list_item_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``term`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. Examples -------- See the examples for the definition_list_, definition_list_item_, and classifier_ elements. ``tgroup`` ========== `To be completed`_. ``thead`` ========= `To be completed`_. ``tip`` ======= The ``tip`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention_, caution_, danger_, error_, hint_, important_, note_, warning_, and the generic admonition_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``tip``. :Children: ``tip`` elements contain one or more `body elements`_. :Analogues: ``tip`` is analogous to the DocBook "tip" element. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "Tip" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``tip`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``tip``. The `%structure.model;`_ parameter entity indirectly includes ``tip``. Examples -------- reStructuredText source:: .. Tip:: 15% if the service is good. Pseudo-XML_ fragment from simple parsing:: <tip> <paragraph> 15% if the service is good. ``title`` ========= The ``title`` element stores the title of a document_, section_, topic_, sidebar_, or generic admonition_. Details ------- :Category: `Structural Subelements`_ :Parents: The following elements may contain ``title``: document_, section_, topic_, sidebar_, admonition_ :Children: ``title`` elements may contain text data plus `inline elements`_. :Analogues: ``title`` is analogous to HTML "title" and header ("h1" etc.) elements, and to the DocBook "title" element. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``title`` element contains the `common attributes`_ (id_, name_, dupname_, source_, and class_), plus refid_ and auto_. ``refid`` is used as a backlink to a table of contents entry. ``auto`` is used to indicate (with value "1") that the ``title`` has been numbered automatically. Examples -------- reStructuredText_ source:: A Title ======= A paragraph. Pseudo-XML_ fragment from simple parsing:: <section id="a-title" name="a title"> <title> A Title <paragraph> A paragraph. ``title_reference`` =================== `To be completed`_. ``topic`` ========= The ``topic`` element is a nonrecursive section_-like construct which may occur at the top level of a section_ wherever a body element (list, table, etc.) is allowed. In other words, ``topic`` elements cannot nest inside body elements, so you can't have a ``topic`` inside a ``table`` or a ``list``, or inside another ``topic`` (or sidebar_). Details ------- :Category: `Structural Elements`_ :Parents: The following elements may contain ``topic``: document_, section_ :Children: ``topic`` elements begin with a title_ and may contain `body elements`_. :Analogues: ``topic`` is analogous to the DocBook "simplesect" element. :Processing: A ``topic`` element should be set off from the rest of the document somehow, such as with indentation or a border. Content Model ------------- .. parsed-literal:: (title_?, (`%body.elements;`_)+) :Attributes: The ``topic`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%structure.model;`_ parameter entity directly includes ``topic``. Examples -------- The `"topic" directive`_ is used to create a ``topic`` element. reStructuredText_ source:: .. topic:: Title Body. Pseudo-XML_ fragment from simple parsing:: <topic> <title> Title <paragraph> Body. .. _"topic" directive: rst/directives.html#topic ``transition`` ============== The ``transition`` element is commonly seen in novels and short fiction, as a gap spanning one or more lines, with or without a type ornament such as a row of asterisks. Transitions separate other body elements, dividing a section into untitled divisions. A transition may not begin or end a section or document, nor may two transitions be immediately adjacent. See `Doctree Representation of Transitions`__ in `A Record of reStructuredText Syntax Alternatives`__. __ ../dev/rst/alternatives.html#doctree-representation-of-transitions __ ../dev/rst/alternatives.html Details ------- :Category: `Structural Subelements`_ :Parents: The following elements may contain ``transition``: document_, section_ :Children: ``transition`` is an empty element and has no children. :Analogues: ``transition`` is analogous to the HTML "hr" element. :Processing: The ``transition`` element is typically rendered as vertical whitespace (more than that separating paragraphs), with or without a horizontal line or row of asterisks. In novels, transitions are often represented as a row of three well-spaced asterisks with vertical space above and below. Content Model ------------- :: EMPTY The ``transition`` element has no content; it is a "point element". :Attributes: The ``transition`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%structure.model;`_ parameter entity directly includes ``transition``. Examples -------- reStructuredText_ source:: Paragraph 1. -------- Paragraph 2. Complete pseudo-XML_ result after parsing:: <document> <paragraph> Paragraph 1. <transition> <paragraph> Paragraph 2. ``version`` =========== The ``version`` element contains the version number of the document. It can be used alone or in conjunction with revision_. Details ------- :Category: `Bibliographic Elements`_ :Parents: Only the docinfo_ element contains ``version``. :Children: ``version`` elements may contain text data plus `inline elements`_. :Analogues: ``version`` may be considered analogous to the DocBook "revision", "revnumber", or "biblioid" elements. :Processing: Sometimes used with the RCS/CVS keyword "Revision". See docinfo_ and revision_. Content Model ------------- .. parsed-literal:: `%text.model;`_ :Attributes: The ``version`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%bibliographic.elements;`_ parameter entity directly includes ``version``. Examples -------- reStructuredText_ source:: Document Title ============== :Version: 1.1 Complete pseudo-XML_ result after parsing and applying transforms:: <document id="document-title" name="document title"> <title> Document Title <docinfo> <version> 1.1 See docinfo_ for a more complete example, including processing context. ``warning`` =========== The ``warning`` element is an admonition, a distinctive and self-contained notice. Also see the other admonition elements Docutils offers (in alphabetical order): attention_, caution_, danger_, error_, hint_, important_, note_, tip_. Details ------- :Category: `Compound Body Elements`_ :Parents: All elements employing the `%body.elements;`_ or `%structure.model;`_ parameter entities in their content models may contain ``warning``. :Children: ``warning`` elements contain one or more `body elements`_. :Analogues: ``warning`` is analogous to the DocBook "warning" element. :Processing: Rendered distinctly (inset and/or in a box, etc.), with the generated title "Warning" (or similar). Content Model ------------- .. parsed-literal:: (`%body.elements;`_)+ :Attributes: The ``warning`` element contains only the `common attributes`_: id_, name_, dupname_, source_, and class_. :Parameter Entities: The `%body.elements;`_ parameter entity directly includes ``warning``. The `%structure.model;`_ parameter entity indirectly includes ``warning``. Examples -------- reStructuredText source:: .. WARNING:: Reader discretion is strongly advised. Pseudo-XML_ fragment from simple parsing:: <warning> <paragraph> Reader discretion is strongly advised. --------------------- Attribute Reference --------------------- .. contents:: :local: :depth: 1 _`Common Attributes`: Through the `%basic.atts;`_ parameter entity, all elements contain the following attributes: id_, name_, dupname_, source_, and class_. .. _attribute type: Attribute types: ``CDATA`` Character data. ``CDATA`` attributes may contain arbitrary text. ``ID`` Like a ``NMTOKEN``, but it must begin with a letter (a "name production"). Identical ``ID`` values must not appear more than once in a document; i.e., ID values must uniquely identify their elements. ``IDREF`` A reference to an ``ID`` value (a name production) of another element. ``IDREFS`` One or more space-separated ``ID`` references (name productions). ``NMTOKEN`` A "name token". One or more of letters, digits, ".", "-", and "_". ``NMTOKENS`` One or more space-separated ``NMTOKEN`` names. ``%yesorno;`` No if zero ("0"), yes if any other value. This is a parameter entity which resolves to a ``NMTOKEN`` attribute type. ``%number;`` This emphasizes that the attribute value must be a number. This is a parameter entity which resolves to a ``NMTOKEN`` attribute type. enumeration The attribute value may be one of a specified list of values. ``anonymous`` ============= `Attribute type`_: ``%yesorno;``. Default value: none (implies no). The ``anonymous`` attribute is used for unnamed hyperlinks in the target_ and reference_ elements (via the `%anonymous.att;`_ parameter entity). ``auto`` ======== `Attribute type`_: ``CDATA``. Default value: none. The ``auto`` attribute is used to indicate automatically-numbered footnote_, footnote_reference_ and title_ elements (via the `%auto.att;`_ parameter entity). ``backrefs`` ============ `Attribute type`_: ``IDREFS``. Default value: none. The ``backrefs`` attribute contains a space-separated list of id_ references, used for backlinks from footnote_, citation_, and system_message_ elements (via the `%backrefs.att;`_ parameter entity). ``bullet`` ========== `Attribute type`_: ``CDATA``. Default value: none. The ``bullet`` attribute is used in the bullet_list_ element. ``class`` ========= `Attribute type`_: ``NMTOKENS``. Default value: none. The ``class`` attribute contains one or more names used to classify an element. The purpose of the attribute is to indicate an "is-a" variant relationship, to allow an extensible way of defining sub-classes of existing elements. It can be used to carry context forward between a Docutils Reader and Writer, when a custom structure is reduced to a standardized document tree. One common use is in conjunction with stylesheets, to add selection criteria. It should not be used to carry formatting instructions or arbitrary content. The ``class`` attribute's contents should be ignorable. Writers that are not familiar with the variant expressed should be able to ignore the attribute. ``class`` is one of the `common attributes`_, shared by all Docutils elements. ``delimiter`` ============= `Attribute type`_: ``CDATA``. Default value: none. The ``delimiter`` attribute is used in the option_argument_ element. ``dupname`` =========== `Attribute type`_: ``NMTOKENS``. Default value: none. The ``dupname`` attribute contains the name of an element when there has been a naming conflict. The contents of the ``dupname`` attribute would have been transferred from the `name`_ attribute. An element may have at most one of the ``name`` or ``dupname`` attributes, but not both. ``dupname`` is one of the `common attributes`_, shared by all Docutils elements. ``enumtype`` ============ `Attribute type`_: enumeration, one of "arabic", "loweralpha", "upperalpha", "lowerroman", or "upperroman". Default value: none. The ``enumtype`` attribute is used in the enumerated_list_ element. ``id`` ====== `Attribute type`_: ``ID``. Default value: none. The ``id`` attribute contains a unique identifier key. ``id`` is one of the `common attributes`_, shared by all Docutils elements. ``name`` ======== `Attribute type`_: ``NMTOKENS``. Default value: none. The ``name`` attribute contains the name of an element, typically originating from the element's title or content. ``name`` must be unique; if there are name conflicts (two or more elements want to the same name), the contents will be transferred to the `dupname`_ attribute on the duplicate elements. An element may have at most one of the ``name`` or ``dupname`` attributes, but not both. ``name`` is one of the `common attributes`_, shared by all Docutils elements. ``prefix`` ========== `Attribute type`_: ``CDATA``. Default value: none. The ``prefix`` attribute is used in the enumerated_list_ element. ``refid`` ========= `Attribute type`_: ``IDREF``. Default value: none. The ``refid`` attribute contains references to `id`_ attributes in other elements. It is used by the target_, reference_, footnote_reference_, citation_reference_, title_ and problematic_ elements (via the `%refid.att;`_ and `%reference.atts;`_ parameter entities). ``refname`` =========== `Attribute type`_: ``NMTOKENS``. Default value: none. The ``refname`` attribute contains an internal reference to the `name`_ attribute of another element. On a `target`_ element, ``refname`` indicates an indirect target which may resolve to either an internal or external reference. ``refname`` is used by the target_, reference_, footnote_reference_, citation_reference_, and substitution_reference_ elements (via the `%refname.att;`_ and `%reference.atts;`_ parameter entities). ``refuri`` ========== `Attribute type`_: ``CDATA``. Default value: none. The ``refuri`` attribute contains an external reference to a URI/URL. It is used by the target_, reference_, footnote_reference_, and citation_reference_ elements (via the `%reference.atts;`_ parameter entity). ``source`` ========== `Attribute type`_: ``CDATA``. Default value: none. The ``source`` attribute is used to store the path or URL to the source text that was used to produce the document tree. It is one of the `common attributes`_, shared by all Docutils elements. ``start`` ========= `Attribute type`_: ``%number;``. Default value: none. The ``start`` attribute is used in the enumerated_list_ element. ``suffix`` ========== `Attribute type`_: ``CDATA``. Default value: none. The ``suffix`` attribute is used in the enumerated_list_ element. ``xml:space`` ============= `Attribute type`_: one of "default" or "preserve". Default value: "preserve" (fixed). The ``xml:space`` attribute is a standard XML attribute for whitespace-preserving elements. It is used by the literal_block_, line_block_, doctest_block_, comment_, and raw_ elements (via the `%fixedspace.att;`_ parameter entity). It is a fixed attribute, meant to communicate to an XML parser that the element contains significant whitespace. The attribute value should not be set in a document instance. ---------------------------- Parameter Entity Reference ---------------------------- .. contents:: :local: :depth: 1 Parameter entities are used to simplify the DTD (to share definitions and reduce duplication) and to allow the DTD to be customized by wrapper DTDs (external client DTDs that use or import the Docutils DTD). Parameter entities may be overridden by wrapper DTDs, replacing the definitions below with custom definitions. Parameter entities whose names begin with "additional" are meant to allow easy extension by wrapper DTDs. ``%anonymous.att;`` =================== The ``%anonymous.att;`` parameter entity contains the anonymous_ attribute, used for unnamed hyperlinks. Entity definition: .. parsed-literal:: anonymous_ %yesorno; #IMPLIED The reference_ and target_ elements directly employ the ``%anonymous.att;`` parameter entity in their attribute lists. ``%auto.att;`` ============== The ``%auto.att;`` parameter entity contains the auto_ attribute, used to indicate an automatically-numbered footnote or title. Entity definition: .. parsed-literal:: auto_ CDATA #IMPLIED The footnote_, footnote_reference_, and title_ elements directly employ the ``%auto.att;`` parameter entity in their attribute lists. ``%backrefs.att;`` ================== The ``%backrefs.att;`` parameter entity contains the backrefs_ attribute, a space-separated list of id references, for backlinks. Entity definition: .. parsed-literal:: backrefs_ IDREFS #IMPLIED The citation_, footnote_, and system_message_ elements directly employ the ``%backrefs.att;`` parameter entity in their attribute lists. ``%basic.atts;`` ================ The ``%basic.atts;`` parameter entity lists attributes common to all Docutils elements. See `Common Attributes`_. Entity definition: .. parsed-literal:: id_ ID #IMPLIED name_ NMTOKENS #IMPLIED dupname_ NMTOKENS #IMPLIED source_ CDATA #IMPLIED class_ NMTOKENS #IMPLIED %additional.basic.atts; The ``%additional.basic.atts;`` parameter entity can be used by wrapper DTDs to extend ``%basic.atts;``. ``%bibliographic.elements;`` ============================ The ``%bibliographic.elements;`` parameter entity contains an OR-list of all `bibliographic elements`_. Entity definition: .. parsed-literal:: author_ | authors_ | organization_ | contact_ | address_ | version_ | revision_ | status_ | date_ | copyright_ | field_ %additional.bibliographic.elements; The ``%additional.bibliographic.elements;`` parameter entity can be used by wrapper DTDs to extend ``%bibliographic.elements;``. Only the docinfo_ element directly employs the ``%bibliographic.elements;`` parameter entity in its content model. ``%body.elements;`` =================== The ``%body.elements;`` parameter entity contains an OR-list of all `body elements`_. ``%body.elements;`` is itself contained within the `%structure.model;`_ parameter entity. Entity definition: .. parsed-literal:: paragraph_ | literal_block_ | doctest_block_ | line_block_ | block_quote_ | table_ | figure_ | image_ | footnote_ | citation_ | rubric_ | bullet_list_ | enumerated_list_ | definition_list_ | field_list_ | option_list_ | attention_ | caution_ | danger_ | error_ | hint_ | important_ | note_ | tip_ | warning_ | admonition_ | target_ | substitution_definition_ | comment_ | pending_ | system_message_ | raw_ %additional.body.elements; The ``%additional.body.elements;`` parameter entity can be used by wrapper DTDs to extend ``%body.elements;``. The ``%body.elements;`` parameter entity is directly employed in the content models of the following elements: admonition_, attention_, block_quote_, caution_, citation_, danger_, definition_, description_, entry_, error_, field_body_, footer_, footnote_, header_, hint_, important_, legend_, list_item_, note_, sidebar_, system_message_, tip_, topic_, warning_ Via `%structure.model;`_, the ``%body.elements;`` parameter entity is indirectly employed in the content models of the document_ and section_ elements. ``%fixedspace.att;`` ==================== The ``%fixedspace.att;`` parameter entity contains the `xml:space`_ attribute, a standard XML attribute for whitespace-preserving elements. Entity definition: .. parsed-literal:: `xml:space`_ (default | preserve) #FIXED 'preserve' The ``%fixedspace.att;`` parameter entity is directly employed in the attribute lists of the following elements: address_, comment_, doctest_block_, line_block_, literal_block_, raw_ ``%inline.elements;`` ===================== The ``%inline.elements;`` parameter entity contains an OR-list of all `inline elements`_. Entity definition: .. parsed-literal:: emphasis_ | strong_ | literal_ | reference_ | footnote_reference_ | citation_reference_ | substitution_reference_ | title_reference_ | abbreviation_ | acronym_ | subscript_ | superscript_ | inline_ | problematic_ | generated_ | target_ | image_ | raw_ %additional.inline.elements; The ``%additional.inline.elements;`` parameter entity can be used by wrapper DTDs to extend ``%inline.elements;``. Via `%text.model;`_, the ``%inline.elements;`` parameter entity is indirectly employed in the content models of the following elements: abbreviation_, acronym_, address_, attribution_, author_, caption_, classifier_, contact_, copyright_, date_, doctest_block_, emphasis_, generated_, inline_, line_block_, literal_block_, organization_, paragraph_, problematic_, raw_, reference_, revision_, rubric_, status_, strong_, subscript_, substitution_definition_, substitution_reference_, subtitle_, superscript_, target_, term_, title_, title_reference_, version_ ``%reference.atts;`` ==================== The ``%reference.atts;`` parameter entity groups together the refuri_, refid_, and refname_ attributes. Entity definition: .. parsed-literal:: `%refuri.att;`_ `%refid.att;`_ `%refname.att;`_ %additional.reference.atts; The ``%additional.reference.atts;`` parameter entity can be used by wrapper DTDs to extend ``%additional.reference.atts;``. The citation_reference_, footnote_reference_, reference_, and target_ elements directly employ the ``%reference.att;`` parameter entity in their attribute lists. ``%refid.att;`` ================ The ``%refid.att;`` parameter entity contains the refid_ attribute, an internal reference to the `id`_ attribute of another element. Entity definition: .. parsed-literal:: refid_ CDATA #IMPLIED The title_ and problematic_ elements directly employ the ``%refid.att;`` parameter entity in their attribute lists. Via `%reference.atts;`_, the ``%refid.att;`` parameter entity is indirectly employed in the attribute lists of the citation_reference_, footnote_reference_, reference_, and target_ elements. ``%refname.att;`` ================= The ``%refname.att;`` parameter entity contains the refname_ attribute, an internal reference to the `name`_ attribute of another element. On a `target`_ element, ``refname`` indicates an indirect target which may resolve to either an internal or external reference. Entity definition: .. parsed-literal:: refname_ NMTOKENS #IMPLIED The substitution_reference_ element directly employs the ``%refname.att;`` parameter entity in its attribute list. Via `%reference.atts;`_, the ``%refname.att;`` parameter entity is indirectly employed in the attribute lists of the citation_reference_, footnote_reference_, reference_, and target_ elements. ``%refuri.att;`` ================ The ``%refuri.att;`` parameter entity contains the refuri_ attribute, an external reference to a URI/URL. Entity definition: .. parsed-literal:: refuri_ CDATA #IMPLIED Via `%reference.atts;`_, the ``%refuri.att;`` parameter entity is indirectly employed in the attribute lists of the citation_reference_, footnote_reference_, reference_, and target_ elements. ``%section.elements;`` ====================== The ``%section.elements;`` parameter entity contains an OR-list of all section_-equivalent elements. ``%section.elements;`` is itself contained within the `%structure.model;`_ parameter entity. Entity definition: .. parsed-literal:: section_ %additional.section.elements; The ``%additional.section.elements;`` parameter entity can be used by wrapper DTDs to extend ``%section.elements;``. Via `%structure.model;`_, the ``%section.elements;`` parameter entity is indirectly employed in the content models of the document_ and section_ elements. ``%structure.model;`` ===================== The ``%structure.model;`` parameter entity encapsulates the hierarchical structure of a document and of its constituent parts. See the discussion of the `element hierarchy`_ above. Entity definition: .. parsed-literal:: ( ( (`%body.elements;`_ | topic_ | sidebar_)+, (transition_, (`%body.elements;`_ | topic_ | sidebar_)+ )*, (`%section.elements;`_)* ) | (`%section.elements;`_)+ ) Each document_ or section_ contains either: - multiple body elements, topics, and/or sidebars, optionally interspersed with transitions (but transitions cannot occur at the beginning or end, nor may there be two transitions in a row), followed by zero or more sections; or - one or more sections (whose contents are recursively the same as this model). The `%structure.model;`_ parameter entity is directly employed in the content models of the document_ and section_ elements. ``%text.model;`` ================ The ``%text.model;`` parameter entity is used by many elements to represent text data mixed with `inline elements`_. Entity definition: .. parsed-literal:: (#PCDATA | `%inline.elements;`_)* The ``%text.model;`` parameter entity is directly employed in the content models of the following elements: abbreviation_, acronym_, address_, author_, caption_, classifier_, contact_, copyright_, date_, doctest_block_, emphasis_, field_name_, generated_, line_block_, literal_block_, organization_, paragraph_, problematic_, raw_, reference_, revision_, status_, strong_, substitution_definition_, substitution_reference_, subtitle_, target_, term_, title_, version_ .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:


=====================
 Docutils Transforms
=====================

:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 1.7 $
:Date: $Date: 2004/06/03 18:59:21 $
:Copyright: This document has been placed in the public domain.

.. contents::

For background about transforms and the Transformer object, see `PEP
258`_.

.. _PEP 258: ../peps/pep-0258.html#transformer

Transforms Listed in Priority Order
===================================

==============================  ============================  ========
Transform: module.Class         Added By                      Priority
==============================  ============================  ========
misc.class                      "class" (d/p)                 210

references.Substitutions        standalone (r), pep (r)       220

frontmatter.DocTitle            standalone (r)                320

frontmatter.DocInfo             standalone (r)                340

peps.Headers                    pep (r)                       360

peps.Contents                   pep (r)                       380

references.ChainedTargets       standalone (r), pep (r)       420

references.AnonymousHyperlinks  standalone (r), pep (r)       440

references.IndirectHyperlinks   standalone (r), pep (r)       460

peps.TargetNotes                pep (r)                       520

references.TargetNotes          peps.TargetNotes (t/p)        0

misc.CallBack                   peps.TargetNotes (t/p)        1

references.TargetNotes          "target-notes" (d/p)          540

references.Footnotes            standalone (r), pep (r)       620

references.ExternalTargets      standalone (r), pep (r)       640

references.InternalTargets      standalone (r), pep (r)       660

parts.SectNum                   "sectnum" (d/p)               710

parts.Contents                  "contents" (d/p),             720
                                peps.Contents (t/p)

peps.PEPZero                    peps.Headers (t/p)            760

components.Filter               "meta" (d/p)                  780

universal.Decorations           Transformer                   820

universal.FinalChecks           Transformer                   840

universal.Messages              Transformer                   860

universal.FilterMessages        Transformer                   870

universal.TestMessages          DocutilsTestSupport           880

misc.CallBack                   n/a                           990
==============================  ============================  ========

Key:

* (r): Reader
* (d): Directive
* (t): Transform
* (/p): Via a "pending" node

Transform Priority Range Categories
===================================

====  ====  ================================================
 Priority
----------  ------------------------------------------------
From   To   Category
====  ====  ================================================
   0    99  immediate execution (added by another transform)
 100   199  very early (non-standard)
 200   299  very early
 300   399  early
 400   699  main
 700   799  late
 800   899  very late
 900   999  very late (non-standard)
====  ====  ================================================


=============================
 reStructuredText Directives
=============================
:Author: David Goodger
:Contact: goodger@python.org
:Revision: $Revision: 1.55 $
:Date: $Date: 2004/06/19 22:54:20 $
:Copyright: This document has been placed in the public domain.

.. contents::

This document describes the directives implemented in the reference
reStructuredText parser.

Directives have the following syntax::

    +-------+-------------------------------+
    | ".. " | directive type "::" directive |
    +-------+ block                         |
            |                               |
            +-------------------------------+

Directives begin with an explicit markup start (two periods and a
space), followed by the directive type and two colons (collectively,
the "directive marker").  The directive block begins immediately after
the directive marker, and includes all subsequent indented lines.  The
directive block is divided into arguments, options (a field list), and
content (in that order), any of which may appear.  See the Directives_
section in the `reStructuredText Markup Specification`_ for syntax
details.

Descriptions below list "doctree elements" (document tree element
names; XML DTD generic identifiers) corresponding to individual
directives.  For details on the hierarchy of elements, please see `The
Docutils Document Tree`_ and the `Docutils Generic DTD`_ XML document
type definition.  For directive implementation details, see `Creating
reStructuredText Directives`_.

.. _Directives: restructuredtext.html#directives
.. _reStructuredText Markup Specification: restructuredtext.html
.. _The Docutils Document Tree: ../doctree.html
.. _Docutils Generic DTD: ../docutils.dtd
.. _Creating reStructuredText Directives:
   ../../howto/rst-directives.html

-------------
 Admonitions
-------------

.. _attention:
.. _caution:
.. _danger:
.. _error:
.. _hint:
.. _important:
.. _note:
.. _tip:
.. _warning:

Specific Admonitions
====================

:Directive Types: "attention", "caution", "danger", "error", "hint",
                  "important", "note", "tip", "warning", "admonition"
:Doctree Elements: attention, caution, danger, error, hint, important,
                   note, tip, warning, admonition, title
:Directive Arguments: None.
:Directive Options: None.
:Directive Content: Interpreted as body elements.

Admonitions are specially marked "topics" that can appear anywhere an
ordinary body element can.  They contain arbitrary body elements.
Typically, an admonition is rendered as an offset block in a document,
sometimes outlined or shaded, with a title matching the admonition
type.  For example::

    .. DANGER::
       Beware killer rabbits!

This directive might be rendered something like this::

    +------------------------+
    |        !DANGER!        |
    |                        |
    | Beware killer rabbits! |
    +------------------------+

The following admonition directives have been implemented:

- attention
- caution
- danger
- error
- hint
- important
- note
- tip
- warning

Any text immediately following the directive indicator (on the same
line and/or indented on following lines) is interpreted as a directive
block and is parsed for normal body elements.  For example, the
following "note" admonition directive contains one paragraph and a
bullet list consisting of two list items::

    .. note:: This is a note admonition.
       This is the second line of the first paragraph.

       - The note contains all indented body elements
         following.
       - It includes this bullet list.

.. _admonition:

Generic Admonition
==================

:Directive Type: "admonition"
:Doctree Elements: admonition, title
:Directive Arguments: One, required (admonition title)
:Directive Options: Possible.
:Directive Content: Interpreted as body elements.

This is a generic, titled admonition.  The title may be anything the
author desires.

The author-supplied title is also used as a "class" attribute value
after being converted into a valid identifier form (down-cased;
non-alphanumeric characters converted to single hyphens; "admonition-"
prefixed).  For example, this admonition::

    .. admonition:: And, by the way...

       You can make up your own admonition too.

becomes the following document tree (pseudo-XML)::

    <document source="test data">
        <admonition class="admonition-and-by-the-way">
            <title>
                And, by the way...
            <paragraph>
                You can make up your own admonition too.

The following option is recognized:

``class`` : text
    Override the computed "class" attribute value.  See the class_
    directive below.

--------
 Images
--------

There are two image directives: "image" and "figure".

Image
=====

:Directive Type: "image"
:Doctree Element: image
:Directive Arguments: One, required (image URI).
:Directive Options: Possible.
:Directive Content: None.

An "image" is a simple picture::

    .. image:: picture.png

The URI for the image source file is specified in the directive
argument.  As with hyperlink targets, the image URI may begin on the
same line as the explicit markup start and target name, or it may
begin in an indented text block immediately following, with no
intervening blank lines.  If there are multiple lines in the link
block, they are stripped of leading and trailing whitespace and joined
together.

Optionally, the image link block may contain a flat field list, the
_`image options`.  For example::

    .. image:: picture.jpeg
       :height: 100
       :width: 200
       :scale: 50
       :alt: alternate text
       :align: right

The following options are recognized:

``alt`` : text
    Alternate text: a short description of the image, displayed by
    applications that cannot display images, or spoken by applications
    for visually impaired users.

``height`` : integer
    The desired height of the image in pixels, used to reserve space
    or scale the image vertically.  When the "scale" option is also
    specified, they are combined.  For example, a height of 200 and a
    scale of 50 is equivalent to a height of 100 with no scale.

``width`` : integer
    The width of the image in pixels, used to reserve space or scale
    the image horizontally.  As with "height" above, when the "scale"
    option is also specified, they are combined.

``scale`` : integer
    The uniform scaling factor of the image, a percentage (but no "%"
    symbol is required or allowed).  "100" means full-size, and is
    equivalent to omitting a "scale" option.

    If no "height" or "width" options are specified, PIL [#PIL]_ may
    be used to determine them, if PIL is installed and the image file
    is available.

``align`` : "top", "middle", "bottom", "left", "center", or "right"
    The alignment of the image, equivalent to the HTML ``<img>`` tag's
    "align" attribute.  The values "top", "middle", and "bottom"
    control an image's vertical alignment (relative to the text
    baseline); they are only useful for inline images (substitutions).
    The values "left", "center", and "right" control an image's
    horizontal alignment, allowing the image to float and have the
    text flow around it.  The specific behavior depends upon the
    browser or rendering software used.

``target`` : text (URI or reference name)
    Makes the image into a hyperlink reference ("clickable").  The
    option argument may be a URI (relative or absolute), or a
    reference name with underscore suffix (e.g. ``name_``).

``class`` : text
    Set a "class" attribute value on the image element.  See the
    class_ directive below.

Figure
======

:Directive Type: "figure"
:Doctree Elements: figure, image, caption, legend
:Directive Arguments: One, required (image URI).
:Directive Options: Possible.
:Directive Content: Interpreted as the figure caption and an optional
                    legend.

A "figure" consists of image_ data (including `image options`_), an
optional caption (a single paragraph), and an optional legend
(arbitrary body elements)::

    .. figure:: picture.png
       :scale: 50
       :alt: map to buried treasure

       This is the caption of the figure (a simple paragraph).

       The legend consists of all elements after the caption.  In this
       case, the legend consists of this paragraph and the following
       table:

       +-----------------------+-----------------------+
       | Symbol                | Meaning               |
       +=======================+=======================+
       | .. image:: tent.png   | Campground            |
       +-----------------------+-----------------------+
       | .. image:: waves.png  | Lake                  |
       +-----------------------+-----------------------+
       | .. image:: peak.png   | Mountain              |
       +-----------------------+-----------------------+

There must be blank lines before the caption paragraph and before the
legend.  To specify a legend without a caption, use an empty comment
("..") in place of the caption.

The "figure" directive supports all of the options of the "image"
directive (see `image options`_ above).  In addition, the following
options are recognized:

``figwidth`` : integer or "image"
    The width of the figure in pixels, to limit the horizontal space
    used.  A special value of "image" is allowed, in which case the
    included image's actual width is used (requires PIL [#PIL]_).  If
    the image file is not found or the required software is
    unavailable, this option is ignored.

    Sets the "width" attribute of the "figure" doctree element.

    This option does not scale the included image; use the "width"
    `image`_ option for that. ::

        +---------------------------+
        |        figure             |
        |                           |
        |<------ figwidth --------->|
        |                           |
        |  +---------------------+  |
        |  |     image           |  |
        |  |                     |  |
        |  |<--- width --------->|  |
        |  +---------------------+  |
        |                           |
        |The figure's caption should|
        |wrap at this width.        |
        +---------------------------+

``figclass`` : text
    Set a "class" attribute value on the figure element.  See the
    class_ directive below.

.. [#PIL] `Python Imaging Library`_.

.. _Python Imaging Library: http://www.pythonware.com/products/pil/

---------------
 Body Elements
---------------

Topic
=====

:Directive Type: "topic"
:Doctree Element: topic
:Directive Arguments: 1, required (topic title).
:Directive Options: Possible.
:Directive Content: Interpreted as the topic body.

A topic is like a block quote with a title, or a self-contained
section with no subsections.  Use the "topic" directive to indicate a
self-contained idea that is separate from the flow of the document.
Topics may occur anywhere a section or transition may occur.  Body
elements (including topics) may not contain nested topics.

The directive's sole argument is interpreted as the topic title; the
next line must be blank.  All subsequent lines make up the topic body,
interpreted as body elements.  For example::

    .. topic:: Topic Title

        Subsequent indented lines comprise
        the body of the topic, and are
        interpreted as body elements.

The following option is recognized:

``class`` : text
    Set a "class" attribute value on the topic element.  See the
    class_ directive below.

Sidebar
=======

:Directive Type: "sidebar"
:Doctree Element: sidebar
:Directive Arguments: One, required (sidebar title).
:Directive Options: Possible.
:Directive Content: Interpreted as the sidebar body.

Sidebars are like miniature, parallel documents that occur inside
other documents, providing related or reference material.  A sidebar
is typically offset by a border and "floats" to the side of the page;
the document's main text may flow around it.  Sidebars can also be
likened to super-footnotes; their content is outside of the flow of
the document's main text.

Sidebars may occur anywhere a section or transition may occur.  Body
elements (including sidebars) may not contain nested sidebars.

The directive's sole argument is interpreted as the sidebar title,
which may be followed by a subtitle option (see below); the next line
must be blank.  All subsequent lines make up the sidebar body,
interpreted as body elements.  For example::

    .. sidebar:: Sidebar Title
       :subtitle: Optional Sidebar Subtitle

       Subsequent indented lines comprise
       the body of the sidebar, and are
       interpreted as body elements.

The following options are recognized:

``subtitle`` : text
    The sidebar's subtitle.

``class`` : text
    Set a "class" attribute value on the sidebar element.  See the
    class_ directive below.

Line Block
==========

:Directive Type: "line-block"
:Doctree Element: line_block
:Directive Arguments: None.
:Directive Options: Possible.
:Directive Content: Becomes the body of the line block.

The "line-block" directive constructs an element where whitespace
(including linebreaks) is significant and inline markup is supported.
It is equivalent to a `parsed literal block`_ with different
rendering: typically in an ordinary serif typeface instead of a
typewriter/monospaced face, and not automatically indented.  (Have the
line-block directive begin a block quote to get an indented line
block.)  Line blocks are useful for address blocks and verse (poetry,
song lyrics), where the structure of lines is significant.  For
example, here's a classic::

    "To Ma Own Beloved Lassie: A Poem on her 17th Birthday", by
    Ewan McTeagle (for Lassie O'Shea):

        .. line-block::

            Lend us a couple of bob till Thursday.
            I'm absolutely skint.
            But I'm expecting a postal order and I can pay you back
                as soon as it comes.
            Love, Ewan.

The following option is recognized:

``class`` : text
    Set a "class" attribute value on the line_block element.  See the
    class_ directive below.

.. _parsed-literal:

Parsed Literal Block
====================

:Directive Type: "parsed-literal"
:Doctree Element: literal_block
:Directive Arguments: None.
:Directive Options: Possible.
:Directive Content: Becomes the body of the literal block.

Unlike an ordinary literal block, the "parsed-literal" directive
constructs a literal block where the text is parsed for inline markup.
It is equivalent to a `line block`_ with different rendering:
typically in a typewriter/monospaced typeface, like an ordinary
literal block.  Parsed literal blocks are useful for adding hyperlinks
to code examples.

However, care must be taken with the text, because inline markup is
recognized and there is no protection from parsing.  Backslash-escapes
may be necessary to prevent unintended parsing.  And because the
markup characters are removed by the parser, care must also be taken
with vertical alignment.  Parsed "ASCII art" is tricky, and extra
whitespace may be necessary.

For example, all the element names in this content model are links::

    .. parsed-literal::

        ((title_,
          subtitle_?)?,
         docinfo_?,
         decoration_?,
         `%structure.model;`_)

The following option is recognized:

``class`` : text
    Set a "class" attribute value on the literal_block element.  See
    the class_ directive below.

Rubric
======

:Directive Type: "rubric"
:Doctree Element: rubric
:Directive Arguments: 1, required (rubric text).
:Directive Options: Possible.
:Directive Content: None.

     rubric n. 1. a title, heading, or the like, in a manuscript,
     book, statute, etc., written or printed in red or otherwise
     distinguished from the rest of the text. ...

     -- Random House Webster's College Dictionary, 1991

The "rubric" directive inserts a "rubric" element into the document
tree.  A rubric is like an informal heading that doesn't correspond to
the document's structure.

The following option is recognized:

``class`` : text
    Set a "class" attribute value on the rubric element.  See the
    class_ directive below.

Epigraph
========

:Directive Type: "epigraph"
:Doctree Element: block_quote
:Directive Arguments: None.
:Directive Options: None.
:Directive Content: Interpreted as the body of the block quote.

Produces an epigraph-class block quote.  For example, this input::

     .. epigraph::

        No matter where you go, there you are.

        -- Buckaroo Banzai

becomes this document tree fragment::

    <block_quote class="epigraph">
        <paragraph>
            No matter where you go, there you are.
        <attribution>
            Buckaroo Banzai

Highlights
==========

:Directive Type: "highlights"
:Doctree Element: block_quote
:Directive Arguments: None.
:Directive Options: None.
:Directive Content: Interpreted as the body of the block quote.

Produces a highlights-class block quote.  See Epigraph_ above.

Pull-Quote
==========

:Directive Type: "pull-quote"
:Doctree Element: block_quote
:Directive Arguments: None.
:Directive Options: None.
:Directive Content: Interpreted as the body of the block quote.

Produces a pull-quote-class block quote.  See Epigraph_ above.

--------
 Tables
--------

Formal tables need more structure than the reStructuredText syntax
supplies.  Tables may be given titles with the table_ directive.
Sometimes reStructuredText tables are inconvenient to write, or table
data in a standard format is readily available.  The csv-table_
directive supports CSV data.

Table
=====

:Directive Type: "table"
:Doctree Element: table
:Directive Arguments: 1, optional (table title).
:Directive Options: Possible.
:Directive Content: A normal reStructuredText table.

(New in Docutils 0.3.1)

The "table" directive is used to create a titled table, to associate a
title with a table::

    .. table:: Truth table for "not"

       =====  =====
         A    not A
       =====  =====
       False  True
       True   False
       =====  =====

The following option is recognized:

``class`` : text
    Set a "class" attribute value on the table element.  See the
    class_ directive below.

.. _csv-table:

CSV Table
=========

:Directive Type: "csv-table"
:Doctree Element: table
:Directive Arguments: 1, optional (table title).
:Directive Options: Possible.
:Directive Content: A CSV (comma-separated values) table.

.. Warning::

   The "csv-table" directive requires the ``csv.py`` module of the
   Python standard library, which was added in Python 2.3.  It will
   not work with earlier versions of Python.  Using the "csv-table"
   directive in a document will make the document **incompatible**
   with systems using Python 2.1 or 2.2.

(New in Docutils 0.3.4)

The "csv-table" directive is used to create a table from CSV
(comma-separated values) data.  CSV is a common data format generated
by spreadsheet applications and commercial databases.  The data may be
internal (an integral part of the document) or external (a separate
file).

Example::

    .. csv-table:: Frozen Delights!
       :headers: "Treat", "Quantity", "Description"
       :widths: 15, 10, 30

       "Albatross", 2.99, "On a stick!"
       "Crunchy Frog", 1.49, "If we took the bones out, it wouldn't be
       crunchy, now would it?"
       "Gannet Ripple", 1.99, "On a stick!"

Block markup and inline markup within cells is supported.  Line ends
are recognized within cells.

Working limitations:

* Whitespace delimiters are supported only for external CSV files.

* There is no support for checking that the number of columns in each
  row is the same.  However, this directive supports CSV generators
  that do not insert "empty" entries at the end of short rows, by
  automatically adding empty entries.

  .. Add "strict" option to verify input?

* Due to limitations of the CSV parser, this directive is not Unicode
  compatible.  It may also have problems with ASCII NUL characters.
  Accordingly, CSV tables should be ASCII-printable safe.

  .. Test with Unicode; see if that's really so.  "encoding" option?

The following options are recognized:

``class`` : text
    Set a "class" attribute value on the table element.  See the
    class_ directive below.

``widths`` : integer [, integer...]
    A comma-separated list of relative column widths.  The default is
    equal-width columns (100%/#columns).

``header-rows`` : integer
    The number of rows of CSV data to use in the table header.
    Defaults to 0.

``header`` : CSV data
    Supplemental data for the table header, added independently of and
    before any ``header-rows`` from the main CSV data.  Must use the
    same CSV format as the main CSV data.

``file`` | ``url`` : path
    Path or URL to CSV file.

``delim`` : char | "tab" | "space"
    A one-character string used to separate fields.  Defaults to ``,``
    (comma).  May be specified as a Unicode code point; see the
    unicode_ directive for syntax details.

``quote`` : char
    A one-character string used to quote elements containing the
    delimiter or which start with the quote character.  Defaults to
    ``"`` (quote).  May be specified as a Unicode code point; see the
    unicode_ directive for syntax details.

``keepspace`` : flag
    Treat whitespace immediately following the delimiter as
    significant.  The default is to ignore such whitespace.

``escape`` : char
    A one-character string used to escape the delimiter or quote
    characters.  May be specified as a Unicode code point; see the
    unicode_ directive for syntax details.  Used when the delimiter is
    used in an unquoted field, or when quote characters are used
    within a field.  The default is to double-up the character,
    e.g. "He said, ""Hi!"""

    .. Add another possible value, "double", to explicitly indicate
       the default case?

----------------
 Document Parts
----------------

.. _contents:

Table of Contents
=================

:Directive Type: "contents"
:Doctree Elements: pending, topic
:Directive Arguments: One, optional: title.
:Directive Options: Possible.
:Directive Content: None.

The "contents" directive inserts a table of contents (TOC) in two
passes: initial parse and transform.  During the initial parse, a
"pending" element is generated which acts as a placeholder, storing
the TOC title and any options internally.  At a later stage in the
processing, the "pending" element is replaced by a "topic" element, a
title and the table of contents proper.

The directive in its simplest form::

    .. contents::

Language-dependent boilerplate text will be used for the title.  The
English default title text is "Contents".

An explicit title, may be specified::

    .. contents:: Table of Contents

The title may span lines, although it is not recommended::

    .. contents:: Here's a very long Table of
       Contents title

Options may be specified for the directive, using a field list::

    .. contents:: Table of Contents
       :depth: 2

If the default title is to be used, the options field list may begin
on the same line as the directive marker::

    .. contents:: :depth: 2

The following options are recognized:

``depth`` : integer
    The number of section levels that are collected in the table of
    contents.  The default is unlimited depth.

``local`` : flag (empty)
    Generate a local table of contents.  Entries will only include
    subsections of the section in which the directive is given.  If no
    explicit title is given, the table of contents will not be titled.

``backlinks`` : "entry" or "top" or "none"
    Generate links from section headers back to the table of contents
    entries, the table of contents itself, or generate no backlinks.

``class`` : text
    Set a "class" attribute value on the topic element.  See the
    class_ directive below.

.. _sectnum:
.. _section-autonumbering:

Automatic Section Numbering
===========================

:Directive Type: "sectnum" or "section-autonumbering" (synonyms)
:Doctree Elements: pending, generated
:Directive Arguments: None.
:Directive Options: Possible.
:Directive Content: None.

The "sectnum" (or "section-autonumbering") directive automatically
numbers sections and subsections in a document.  Section numbers are
of the "multiple enumeration" form, where each level has a number,
separated by periods.  For example, the title of section 1, subsection
2, subsubsection 3 would have "1.2.3" prefixed.

The "sectnum" directive does its work in two passes: the initial parse
and a transform.  During the initial parse, a "pending" element is
generated which acts as a placeholder, storing any options internally.
At a later stage in the processing, the "pending" element triggers a
transform, which adds section numbers to titles.  Section numbers are
enclosed in a "generated" element, and titles have their "auto"
attribute set to "1".

The following options are recognized:

``depth`` : integer
    The number of section levels that are numbered by this directive.
    The default is unlimited depth.

------------
 References
------------

.. _target-notes:

Target Footnotes
================

:Directive Type: "target-notes"
:Doctree Elements: pending, footnote, footnote_reference
:Directive Arguments: None.
:Directive Options: None.
:Directive Content: None.

The "target-notes" directive creates a footnote for each external
target in the text, and corresponding footnote references after each
reference.  For every explicit target (of the form, ``.. _target name:
URL``) in the text, a footnote will be generated containing the
visible URL as content.

Footnotes
=========

**NOT IMPLEMENTED YET**

:Directive Type: "footnotes"
:Doctree Elements: pending, topic
:Directive Arguments: None?
:Directive Options: Possible?
:Directive Content: None.

@@@

Citations
=========

**NOT IMPLEMENTED YET**

:Directive Type: "citations"
:Doctree Elements: pending, topic
:Directive Arguments: None?
:Directive Options: Possible?
:Directive Content: None.

@@@

---------------
 HTML-Specific
---------------

Meta
====

:Directive Type: "meta"
:Doctree Element: meta (non-standard)
:Directive Arguments: None.
:Directive Options: None.
:Directive Content: Must contain a flat field list.

The "meta" directive is used to specify HTML metadata stored in HTML
META tags.  "Metadata" is data about data, in this case data about web
pages.  Metadata is used to describe and classify web pages in the
World Wide Web, in a form that is easy for search engines to extract
and collate.

Within the directive block, a flat field list provides the syntax for
metadata.  The field name becomes the contents of the "name" attribute
of the META tag, and the field body (interpreted as a single string
without inline markup) becomes the contents of the "content"
attribute.  For example::

    .. meta::
       :description: The reStructuredText plaintext markup language
       :keywords: plaintext, markup language

This would be converted to the following HTML::

    <meta name="description"
        content="The reStructuredText plaintext markup language">
    <meta name="keywords" content="plaintext, markup language">

Support for other META attributes ("http-equiv", "scheme", "lang",
"dir") are provided through field arguments, which must be of the form
"attr=value"::

    .. meta::
       :description lang=en: An amusing story
       :description lang=fr: Un histoire amusant

And their HTML equivalents::

    <meta name="description" lang="en" content="An amusing story">
    <meta name="description" lang="fr" content="Un histoire amusant">

Some META tags use an "http-equiv" attribute instead of the "name"
attribute.  To specify "http-equiv" META tags, simply omit the name::

    .. meta::
       :http-equiv=Content-Type: text/html; charset=ISO-8859-1

HTML equivalent::

    <meta http-equiv="Content-Type"
         content="text/html; charset=ISO-8859-1">

Imagemap
========

**NOT IMPLEMENTED YET**

Non-standard element: imagemap.

---------------
 Miscellaneous
---------------

.. _include:

Including an External Document Fragment
=======================================

:Directive Type: "include"
:Doctree Elements: depend on data being included
:Directive Arguments: One, required.
:Directive Options: Possible.
:Directive Content: None.

The "include" directive reads a reStructuredText-formatted text file
and parses it in the current document's context at the point of the
directive.  The directive argument is the path to the file to be
included, relative to the document containing the directive.  For
example::

    This first example will be parsed at the document level, and can
    thus contain any construct, including section headers.

    .. include:: inclusion.txt

    Back in the main document.

        This second example will be parsed in a block quote context.
        Therefore it may only contain body elements.  It may not
        contain section headers.

        .. include:: inclusion.txt

If an included document fragment contains section structure, the title
adornments must match those of the master document.

The text encoding of the master input source is used for included
files.

The following options are recognized:

``literal`` : flag (empty)
    The entire included text is inserted into the document as a single
    literal block (useful for program listings).

.. _raw:

Raw Data Pass-Through
=====================

:Directive Type: "raw"
:Doctree Element: raw
:Directive Arguments: One, required (output format type).
:Directive Options: Possible.
:Directive Content: Stored verbatim, uninterpreted.  None (empty) if a
                    "file" or "url" option given.

The "raw" directive indicates non-reStructuredText data that is to be
passed untouched to the Writer.  The name of the output format is
given in the first argument.  The interpretation of the raw data is up
to the Writer.  A Writer may ignore any raw output not matching its
format.

For example, the following input would be passed untouched by an HTML
Writer::

    .. raw:: html

       <hr width=50 size=10>

A LaTeX Writer could insert the following raw content into its
output stream::

    .. raw:: latex

       \setlength{\parindent}{0pt}

Raw data can also be read from an external file, specified in a
directive option.  In this case, the content block must be empty.  For
example::

    .. raw:: html
       :file: inclusion.html

The following options are recognized:

``file`` : string
    The local filesystem path of a raw data file to be included.
``url`` : string
    An Internet URL reference to a raw data file to be included.

.. _replace:

Replacement Text
================

:Directive Type: "replace"
:Doctree Element: Text & inline elements
:Directive Arguments: None.
:Directive Options: None.
:Directive Content: A single paragraph; may contain inline markup.

The "replace" directive is used to indicate replacement text for a
substitution reference.  It may be used within substitution
definitions only.  For example, this directive can be used to expand
abbreviations::

    .. |reST| replace:: reStructuredText

    Yes, |reST| is a long word, so I can't blame anyone for wanting to
    abbreviate it.

As reStructuredText doesn't support nested inline markup, the only way
to create a reference with styled text is to use substitutions with
the "replace" directive::

    I recommend you try |Python|_.

    .. |Python| replace:: Python, *the* best language around
    .. _Python: http://www.python.org/

.. _unicode:

Unicode Character Codes
=======================

:Directive Type: "unicode"
:Doctree Element: Text
:Directive Arguments: One or more, required (Unicode character codes,
                      optional text, and comments).
:Directive Options: None.
:Directive Content: None.

The "unicode" directive converts Unicode character codes (numerical
values) to characters, and may be used in substitution definitions
only.  Codes may be decimal numbers, hexadecimal numbers (prefixed by
``0x``, ``x``, ``\x``, ``U+``, ``u``, or ``\u``), or XML-style numeric
character entities (e.g. ``&#x1a2b;``).  Codes are case-insensitive.

Text following " .. " is a comment and is ignored.  Spaces are
ignored, and any other text remains as-is.

Example::

    Copyright |copy| 2003, |BogusMegaCorp (TM)|.

    .. |copy| unicode:: 0xA9 .. copyright sign
    .. |BogusMegaCorp (TM)| unicode:: BogusMegaCorp U+2122
       .. with trademark sign

Class
=====

:Directive Type: "class"
:Doctree Element: pending
:Directive Arguments: One, required (class name / attribute value).
:Directive Options: None.
:Directive Content: None.

The "class" directive sets a "class" attribute value on the first
immediately following non-comment element [#]_.  For details of the
"class" attribute, see `its entry`__ in `The Docutils Document Tree`_.
The "class" attribute value is converted to lowercase and all
non-alphanumeric characters are converted to hyphens in order to match
this regular expression: ``[a-z](-?[a-z0-9]+)*``.  See the docstring
from ``docutils.nodes.make_id`` for the rationale.

Examples::

    .. class:: special

    This is a "special" paragraph.

    .. class:: exceptional

    An Exceptional Section
    ======================

    This is an ordinary paragraph.

The text above is parsed and transformed into this doctree fragment::

    <paragraph class="special">
        This is a "special" paragraph.
    <section class="exceptional">
        <title>
            An Exceptional Section
        <paragraph>
            This is an ordinary paragraph.

.. [#] To set a "class" attribute value on a block quote, the "class"
   directive must be followed by an empty comment::

       .. class:: highlights
       ..

           Block quote text.

   The directive doesn't allow content, therefore an empty comment is
   required to terminate the directive.  Without the empty comment,
   the block quote text would be interpreted as the "class"
   directive's content, and the parser would complain.

__ ../doctree.html#class

.. _role:

Custom Interpreted Text Roles
=============================

:Directive Type: "role"
:Doctree Element: None; affects subsequent parsing.
:Directive Arguments: Two; one required (role name), one optional
                      (base role name, in parentheses).
:Directive Options: Possible (depends on base role).
:Directive Content: depends on base role.

(New in Docutils 0.3.2)

The "role" directive dynamically creates a custom interpreted text
role and registers it with the parser.  This means that after
declaring a role like this::

    .. role:: custom

the document may use the new "custom" role::

    An example of using :custom:`interpreted text`

This will be parsed into the following document tree fragment::

    <paragraph>
        An example of using
        <inline class="custom">
            interpreted text

The role must be declared in a document before it can be used.

The new role may be based on an existing role, specified as a second
argument in parentheses (whitespace optional)::

    .. role:: custom(emphasis)

    :custom:`text`

The parsed result is as follows::

    <paragraph>
        <emphasis class="custom">
            text

If no base role is explicitly specified, a generic custom role is
automatically used.  Subsequent interpreted text will produce an
"inline" element with a "class" attribute, as in the first example
above.

With most roles, the ":class:" option can be used to set a "class"
attribute that is different from the role name.  For example::

    .. role:: custom
       :class: special

    :custom:`interpreted text`

This is the parsed result::

    <paragraph>
        <inline class="special">
            interpreted text

.. _role class:

The following option is recognized by the "role" directive for most
base roles:

``class`` : text
    Set a "class" attribute value on the element produced (``inline``,
    or element associated with a base class) when the custom
    interpreted text role is used.  If no directive options are
    specified, a "class" option with the directive argument (role
    name) as the value is implied.  See the class_ directive above.

Specific roles may support other options and/or directive content.
See the `reStructuredText Interpreted Text Roles`_ document for
details.

.. _reStructuredText Interpreted Text Roles: roles.html

Restructuredtext-Test-Directive
===============================

:Directive Type: "restructuredtext-test-directive"
:Doctree Element: system_warning
:Directive Arguments: None.
:Directive Options: None.
:Directive Content: Interpreted as a literal block.

This directive is provided for test purposes only.  (Nobody is
expected to type in a name *that* long!)  It is converted into a
level-1 (info) system message showing the directive data, possibly
followed by a literal block containing the rest of the directive
block.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


=====================================
 An Introduction to reStructuredText
=====================================
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 1.8 $
:Date: $Date: 2004/06/03 17:47:43 $
:Copyright: This document has been placed in the public domain.

reStructuredText_ is an easy-to-read, what-you-see-is-what-you-get
plaintext markup syntax and parser system.  It is useful for inline
program documentation (such as Python docstrings), for quickly
creating simple web pages, and for standalone documents.
reStructuredText_ is a proposed revision and reinterpretation of the
StructuredText_ and Setext_ lightweight markup systems.

reStructuredText is designed for extensibility for specific
application domains.  Its parser is a component of Docutils_.

This document defines the goals_ of reStructuredText and provides a
history_ of the project.  It is written using the reStructuredText
markup, and therefore serves as an example of its use.  For a gentle
introduction to using reStructuredText, please read `A
ReStructuredText Primer`_.  The `Quick reStructuredText`_ user
reference is also useful.  The `reStructuredText Markup
Specification`_ is the definitive reference.  There is also an
analysis of the `Problems With StructuredText`_.

ReStructuredText's web page is
http://docutils.sourceforge.net/rst.html.

.. _reStructuredText: http://docutils.sourceforge.net/rst.html
.. _StructuredText:
    http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage
.. _Setext: http://docutils.sourceforge.net/mirror/setext.html
.. _Docutils: http://docutils.sourceforge.net/
.. _A ReStructuredText Primer: ../../user/rst/quickstart.html
.. _Quick reStructuredText: ../../user/rst/quickref.html
.. _reStructuredText Markup Specification: restructuredtext.html
.. _Problems with StructuredText: ../../dev/rst/problems.html

Goals
=====

The primary goal of reStructuredText_ is to define a markup syntax for
use in Python docstrings and other documentation domains, that is
readable and simple, yet powerful enough for non-trivial use.  The
intended purpose of the reStructuredText markup is twofold:

- the establishment of a set of standard conventions allowing the
  expression of structure within plaintext, and

- the conversion of such documents into useful structured data
  formats.

The secondary goal of reStructuredText is to be accepted by the Python
community (by way of being blessed by PythonLabs and the BDFL [#]_) as
a standard for Python inline documentation (possibly one of several
standards, to account for taste).

.. [#] Python's creator and "Benevolent Dictator For Life",
   Guido van Rossum.

To clarify the primary goal, here are specific design goals, in order,
beginning with the most important:

1. Readable.  The marked-up text must be easy to read without any
   prior knowledge of the markup language.  It should be as easily
   read in raw form as in processed form.

2. Unobtrusive.  The markup that is used should be as simple and
   unobtrusive as possible.  The simplicity of markup constructs
   should be roughly proportional to their frequency of use.  The most
   common constructs, with natural and obvious markup, should be the
   simplest and most unobtrusive.  Less common constructs, for which
   there is no natural or obvious markup, should be distinctive.

3. Unambiguous.  The rules for markup must not be open for
   interpretation.  For any given input, there should be one and only
   one possible output (including error output).

4. Unsurprising.  Markup constructs should not cause unexpected output
   upon processing.  As a fallback, there must be a way to prevent
   unwanted markup processing when a markup construct is used in a
   non-markup context (for example, when documenting the markup syntax
   itself).

5. Intuitive.  Markup should be as obvious and easily remembered as
   possible, for the author as well as for the reader.  Constructs
   should take their cues from such naturally occurring sources as
   plaintext email messages, newsgroup postings, and text
   documentation such as README.txt files.

6. Easy.  It should be easy to mark up text using any ordinary text
   editor.

7. Scalable.  The markup should be applicable regardless of the length
   of the text.

8. Powerful.  The markup should provide enough constructs to produce a
   reasonably rich structured document.

9. Language-neutral.  The markup should apply to multiple natural (as
   well as artificial) languages, not only English.

10. Extensible.  The markup should provide a simple syntax and
    interface for adding more complex general markup, and custom
    markup.

11. Output-format-neutral.  The markup will be appropriate for
    processing to multiple output formats, and will not be biased
    toward any particular format.

The design goals above were used as criteria for accepting or
rejecting syntax, or selecting between alternatives.

It is emphatically *not* the goal of reStructuredText to define
docstring semantics, such as docstring contents or docstring length.
These issues are orthogonal to the markup syntax and beyond the scope
of this specification.

Also, it is not the goal of reStructuredText to maintain compatibility
with StructuredText_ or Setext_.  reStructuredText shamelessly steals
their great ideas and ignores the not-so-great.

Author's note:

    Due to the nature of the problem we're trying to solve (or,
    perhaps, due to the nature of the proposed solution), the above
    goals unavoidably conflict.  I have tried to extract and distill
    the wisdom accumulated over the years in the Python Doc-SIG_
    mailing list and elsewhere, to come up with a coherent and
    consistent set of syntax rules, and the above goals by which to
    measure them.

    There will inevitably be people who disagree with my particular
    choices.  Some desire finer control over their markup, others
    prefer less.  Some are concerned with very short docstrings,
    others with full-length documents.  This specification is an
    effort to provide a reasonably rich set of markup constructs in a
    reasonably simple form, that should satisfy a reasonably large
    group of reasonable people.

    David Goodger (goodger@users.sourceforge.net), 2001-04-20

.. _Doc-SIG: http://www.python.org/sigs/doc-sig/

History
=======

reStructuredText_, the specification, is based on StructuredText_ and
Setext_.  StructuredText was developed by Jim Fulton of `Zope
Corporation`_ (formerly Digital Creations) and first released in 1996.
It is now released as a part of the open-source "Z Object Publishing
Environment" (ZOPE_).  Ian Feldman's and Tony Sanders' earlier Setext_
specification was either an influence on StructuredText or, by their
similarities, at least evidence of the correctness of this approach.

I discovered StructuredText_ in late 1999 while searching for a way to
document the Python modules in one of my projects.  Version 1.1 of
StructuredText was included in Daniel Larsson's pythondoc_.  Although
I was not able to get pythondoc to work for me, I found StructuredText
to be almost ideal for my needs.  I joined the Python Doc-SIG_
(Documentation Special Interest Group) mailing list and found an
ongoing discussion of the shortcomings of the StructuredText
"standard".  This discussion has been going on since the inception of
the mailing list in 1996, and possibly predates it.

I decided to modify the original module with my own extensions and
some suggested by the Doc-SIG members.  I soon realized that the
module was not written with extension in mind, so I embarked upon a
general reworking, including adapting it to the "re" regular
expression module (the original inspiration for the name of this
project).  Soon after I completed the modifications, I discovered that
StructuredText.py was up to version 1.23 in the ZOPE distribution.
Implementing the new syntax extensions from version 1.23 proved to be
an exercise in frustration, as the complexity of the module had become
overwhelming.

In 2000, development on StructuredTextNG_ ("Next Generation") began at
`Zope Corporation`_ (then Digital Creations).  It seems to have many
improvements, but still suffers from many of the problems of classic
StructuredText.

I decided that a complete rewrite was in order, and even started a
`reStructuredText SourceForge project`_ (now inactive).  My
motivations (the "itches" I aim to "scratch") are as follows:

- I need a standard format for inline documentation of the programs I
  write.  This inline documentation has to be convertible to other
  useful formats, such as HTML.  I believe many others have the same
  need.

- I believe in the Setext/StructuredText idea and want to help
  formalize the standard.  However, I feel the current specifications
  and implementations have flaws that desperately need fixing.

- reStructuredText could form part of the foundation for a
  documentation extraction and processing system, greatly benefitting
  Python.  But it is only a part, not the whole.  reStructuredText is
  a markup language specification and a reference parser
  implementation, but it does not aspire to be the entire system.  I
  don't want reStructuredText or a hypothetical Python documentation
  processor to die stillborn because of over-ambition.

- Most of all, I want to help ease the documentation chore, the bane
  of many a programmer.

Unfortunately I was sidetracked and stopped working on this project.
In November 2000 I made the time to enumerate the problems of
StructuredText and possible solutions, and complete the first draft of
a specification.  This first draft was posted to the Doc-SIG in three
parts:

- `A Plan for Structured Text`__
- `Problems With StructuredText`__
- `reStructuredText: Revised Structured Text Specification`__

__ http://mail.python.org/pipermail/doc-sig/2000-November/001239.html
__ http://mail.python.org/pipermail/doc-sig/2000-November/001240.html
__ http://mail.python.org/pipermail/doc-sig/2000-November/001241.html

In March 2001 a flurry of activity on the Doc-SIG spurred me to
further revise and refine my specification, the result of which you
are now reading.  An offshoot of the reStructuredText project has been
the realization that a single markup scheme, no matter how well
thought out, may not be enough.  In order to tame the endless debates
on Doc-SIG, a flexible `Docstring Processing System framework`_ needed
to be constructed.  This framework has become the more important of
the two projects; reStructuredText_ has found its place as one
possible choice for a single component of the larger framework.

The project web site and the first project release were rolled out in
June 2001, including posting the second draft of the spec [#spec-2]_
and the first draft of PEPs 256, 257, and 258 [#peps-1]_ to the
Doc-SIG.  These documents and the project implementation proceeded to
evolve at a rapid pace.  Implementation history details can be found
in the `project history file`_.

In November 2001, the reStructuredText parser was nearing completion.
Development of the parser continued with the addition of small
convenience features, improvements to the syntax, the filling in of
gaps, and bug fixes.  After a long holiday break, in early 2002 most
development moved over to the other Docutils components, the
"Readers", "Writers", and "Transforms".  A "standalone" reader
(processes standalone text file documents) was completed in February,
and a basic HTML writer (producing HTML 4.01, using CSS-1) was
completed in early March.

`PEP 287`_, "reStructuredText Standard Docstring Format", was created
to formally propose reStructuredText as a standard format for Python
docstrings, PEPs, and other files.  It was first posted to
comp.lang.python_ and the Python-dev_ mailing list on 2002-04-02.

Version 0.4 of the reStructuredText__ and `Docstring Processing
System`_ projects were released in April 2002.  The two projects were
immediately merged, renamed to "Docutils_", and a 0.1 release soon
followed.

.. __: `reStructuredText SourceForge project`_

.. [#spec-2] The second draft of the spec:

   - `An Introduction to reStructuredText`__
   - `Problems With StructuredText`__
   - `reStructuredText Markup Specification`__
   - `Python Extensions to the reStructuredText Markup
     Specification`__

   __ http://mail.python.org/pipermail/doc-sig/2001-June/001858.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001859.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001860.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001861.html

.. [#peps-1] First drafts of the PEPs:

   - `PEP 256: Docstring Processing System Framework`__
   - `PEP 258: DPS Generic Implementation Details`__
   - `PEP 257: Docstring Conventions`__

   Current working versions of the PEPs can be found in
   http://docutils.sourceforge.net/docs/peps/, and official versions
   can be found in the `master PEP repository`_.

   __ http://mail.python.org/pipermail/doc-sig/2001-June/001855.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001856.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001857.html

.. _Zope Corporation: http://www.zope.com
.. _ZOPE: http://www.zope.org
.. _reStructuredText SourceForge project:
   http://structuredtext.sourceforge.net/
.. _pythondoc: http://starship.python.net/crew/danilo/pythondoc/
.. _StructuredTextNG:
   http://dev.zope.org/Members/jim/StructuredTextWiki/StructuredTextNG
.. _project history file: ../../../HISTORY.html
.. _PEP 287: ../../peps/pep-0287.html
.. _Docstring Processing System framework: ../../peps/pep-0256.html
.. _comp.lang.python: news:comp.lang.python
.. _Python-dev: http://mail.python.org/pipermail/python-dev/
.. _Docstring Processing System: http://docstring.sourceforge.net/
.. _master PEP repository: http://www.python.org/peps/

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


=======================================
 reStructuredText Markup Specification
=======================================
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 1.63 $
:Date: $Date: 2004/06/28 19:24:55 $
:Copyright: This document has been placed in the public domain.

.. Note::

   This document is a detailed technical specification; it is not a
   tutorial or a primer.  If this is your first exposure to
   reStructuredText, please read `A ReStructuredText Primer`_ and the
   `Quick reStructuredText`_ user reference first.

.. _A ReStructuredText Primer: ../../user/rst/quickstart.html
.. _Quick reStructuredText: ../../user/rst/quickref.html

reStructuredText_ is plaintext that uses simple and intuitive
constructs to indicate the structure of a document.  These constructs
are equally easy to read in raw and processed forms.  This document is
itself an example of reStructuredText (raw, if you are reading the
text file, or processed, if you are reading an HTML document, for
example).  The reStructuredText parser is a component of Docutils_.

Simple, implicit markup is used to indicate special constructs, such
as section headings, bullet lists, and emphasis.  The markup used is
as minimal and unobtrusive as possible.  Less often-used constructs
and extensions to the basic reStructuredText syntax may have more
elaborate or explicit markup.

reStructuredText is applicable to documents of any length, from the
very small (such as inline program documentation fragments, e.g.
Python docstrings) to the quite large (this document).

The first section gives a quick overview of the syntax of the
reStructuredText markup by example.  A complete specification is given
in the `Syntax Details`_ section.

`Literal blocks`_ (in which no markup processing is done) are used for
examples throughout this document, to illustrate the plaintext markup.

.. contents::

-----------------------
 Quick Syntax Overview
-----------------------

A reStructuredText document is made up of body or block-level
elements, and may be structured into sections.  Sections_ are
indicated through title style (underlines & optional overlines).
Sections contain body elements and/or subsections.  Some body elements
contain further elements, such as lists containing list items, which
in turn may contain paragraphs and other body elements.  Others, such
as paragraphs, contain text and `inline markup`_ elements.

Here are examples of `body elements`_:

- Paragraphs_ (and `inline markup`_)::

      Paragraphs contain text and may contain inline markup:
      *emphasis*, **strong emphasis**, `interpreted text`, ``inline
      literals``, standalone hyperlinks (http://www.python.org),
      external hyperlinks (Python_), internal cross-references
      (example_), footnote references ([1]_), citation references
      ([CIT2002]_), substitution references (|example|), and _`inline
      internal targets`.

      Paragraphs are separated by blank lines and are left-aligned.

- Five types of lists:

  1. `Bullet lists`_::

         - This is a bullet list.

         - Bullets can be "-", "*", or "+".

  2. `Enumerated lists`_::

         1. This is an enumerated list.

         2. Enumerators may be arabic numbers, letters, or roman
            numerals.

  3. `Definition lists`_::

         what
             Definition lists associate a term with a definition.

         how
             The term is a one-line phrase, and the definition is one
             or more paragraphs or body elements, indented relative to
             the term.

  4. `Field lists`_::

         :what: Field lists map field names to field bodies, like
                database records.  They are often part of an extension
                syntax.

         :how: The field marker is a colon, the field name, and a
               colon.

               The field body may contain one or more body elements,
               indented relative to the field marker.

  5. `Option lists`_, for listing command-line options::

         -a            command-line option "a"
         -b file       options can have arguments
                       and long descriptions
         --long        options can be long also
         --input=file  long options can also have
                       arguments
         /V            DOS/VMS-style options too

     There must be at least two spaces between the option and the
     description.

- `Literal blocks`_::

      Literal blocks are either indented or line-prefix-quoted blocks,
      and indicated with a double-colon ("::") at the end of the
      preceding paragraph (right here -->)::

          if literal_block:
              text = 'is left as-is'
              spaces_and_linebreaks = 'are preserved'
              markup_processing = None

- `Block quotes`_::

      Block quotes consist of indented body elements:

          This theory, that is mine, is mine.

          -- Anne Elk (Miss)

- `Doctest blocks`_::

      >>> print 'Python-specific usage examples; begun with ">>>"'
      Python-specific usage examples; begun with ">>>"
      >>> print '(cut and pasted from interactive Python sessions)'
      (cut and pasted from interactive Python sessions)

- Two syntaxes for tables_:

  1. `Grid tables`_; complete, but complex and verbose::

         +------------------------+------------+----------+
         | Header row, column 1   | Header 2   | Header 3 |
         +========================+============+==========+
         | body row 1, column 1   | column 2   | column 3 |
         +------------------------+------------+----------+
         | body row 2             | Cells may span        |
         +------------------------+-----------------------+

  2. `Simple tables`_; easy and compact, but limited::

         ====================  ==========  ==========
         Header row, column 1  Header 2    Header 3
         ====================  ==========  ==========
         body row 1, column 1  column 2    column 3
         body row 2            Cells may span columns
         ====================  ======================

- `Explicit markup blocks`_ all begin with an explicit block marker,
  two periods and a space:

  - Footnotes_::

        .. [1] A footnote contains body elements, consistently
           indented by at least 3 spaces.

  - Citations_::

        .. [CIT2002] Just like a footnote, except the label is
           textual.

  - `Hyperlink targets`_::

        .. _Python: http://www.python.org

        .. _example:

        The "_example" target above points to this paragraph.

  - Directives_::

        .. image:: mylogo.png

  - `Substitution definitions`_::

        .. |symbol here| image:: symbol.png

  - Comments_::

        .. Comments begin with two dots and a space.  Anything may
           follow, except for the syntax of footnotes/citations,
           hyperlink targets, directives, or substitution definitions.

----------------
 Syntax Details
----------------

Descriptions below list "doctree elements" (document tree element
names; XML DTD generic identifiers) corresponding to syntax
constructs.  For details on the hierarchy of elements, please see `The
Docutils Document Tree`_ and the `Docutils Generic DTD`_ XML document
type definition.

Whitespace
==========

Spaces are recommended for indentation_, but tabs may also be used.
Tabs will be converted to spaces.  Tab stops are at every 8th column.

Other whitespace characters (form feeds [chr(12)] and vertical tabs
[chr(11)]) are converted to single spaces before processing.

Blank Lines
-----------

Blank lines are used to separate paragraphs and other elements.
Multiple successive blank lines are equivalent to a single blank line,
except within literal blocks (where all whitespace is preserved).
Blank lines may be omitted when the markup makes element separation
unambiguous, in conjunction with indentation.  The first line of a
document is treated as if it is preceded by a blank line, and the last
line of a document is treated as if it is followed by a blank line.

Indentation
-----------

Indentation is used to indicate, and is only significant in
indicating:

- multi-line contents of list items,
- multiple body elements within a list item (including nested lists),
- the definition part of a definition list item,
- block quotes,
- the extent of literal blocks, and
- the extent of explicit markup blocks.

Any text whose indentation is less than that of the current level
(i.e., unindented text or "dedents") ends the current level of
indentation.

Since all indentation is significant, the level of indentation must be
consistent.  For example, indentation is the sole markup indicator for
`block quotes`_::

    This is a top-level paragraph.

        This paragraph belongs to a first-level block quote.

        Paragraph 2 of the first-level block quote.

Multiple levels of indentation within a block quote will result in
more complex structures::

    This is a top-level paragraph.

        This paragraph belongs to a first-level block quote.

            This paragraph belongs to a second-level block quote.

    Another top-level paragraph.

            This paragraph belongs to a second-level block quote.

        This paragraph belongs to a first-level block quote.  The
        second-level block quote above is inside this first-level
        block quote.

When a paragraph or other construct consists of more than one line of
text, the lines must be left-aligned::

    This is a paragraph.  The lines of
    this paragraph are aligned at the left.

        This paragraph has problems.  The
    lines are not left-aligned.  In addition
      to potential misinterpretation, warning
        and/or error messages will be generated
      by the parser.

Several constructs begin with a marker, and the body of the construct
must be indented relative to the marker.  For constructs using simple
markers (`bullet lists`_, `enumerated lists`_, footnotes_, citations_,
`hyperlink targets`_, directives_, and comments_), the level of
indentation of the body is determined by the position of the first
line of text, which begins on the same line as the marker.  For
example, bullet list bodies must be indented by at least two columns
relative to the left edge of the bullet::

    - This is the first line of a bullet list
      item's paragraph.  All lines must align
      relative to the first line.  [1]_

          This indented paragraph is interpreted
          as a block quote.

    Because it is not sufficiently indented,
    this paragraph does not belong to the list
    item.

    .. [1] Here's a footnote.  The second line is aligned
       with the beginning of the footnote label.  The ".."
       marker is what determines the indentation.

For constructs using complex markers (`field lists`_ and `option
lists`_), where the marker may contain arbitrary text, the indentation
of the first line *after* the marker determines the left edge of the
body.  For example, field lists may have very long markers (containing
the field names)::

    :Hello: This field has a short field name, so aligning the field
            body with the first line is feasible.

    :Number-of-African-swallows-required-to-carry-a-coconut: It would
        be very difficult to align the field body with the left edge
        of the first line.  It may even be preferable not to begin the
        body on the same line as the marker.

Escaping Mechanism
==================

The character set universally available to plaintext documents, 7-bit
ASCII, is limited.  No matter what characters are used for markup,
they will already have multiple meanings in written text.  Therefore
markup characters *will* sometimes appear in text **without being
intended as markup**.  Any serious markup system requires an escaping
mechanism to override the default meaning of the characters used for
the markup.  In reStructuredText we use the backslash, commonly used
as an escaping character in other domains.

A backslash followed by any character (except whitespace characters)
escapes that character.  The escaped character represents the
character itself, and is prevented from playing a role in any markup
interpretation.  The backslash is removed from the output.  A literal
backslash is represented by two backslashes in a row (the first
backslash "escapes" the second, preventing it being interpreted in an
"escaping" role).

Backslash-escaped whitespace characters are removed from the document.
This allows for character-level `inline markup`_.

There are two contexts in which backslashes have no special meaning:
literal blocks and inline literals.  In these contexts, a single
backslash represents a literal backslash, without having to double up.

Please note that the reStructuredText specification and parser do not
address the issue of the representation or extraction of text input
(how and in what form the text actually *reaches* the parser).
Backslashes and other characters may serve a character-escaping
purpose in certain contexts and must be dealt with appropriately.  For
example, Python uses backslashes in strings to escape certain
characters, but not others.  The simplest solution when backslashes
appear in Python docstrings is to use raw docstrings::

    r"""This is a raw docstring.  Backslashes (\) are not touched."""

Reference Names
===============

Simple reference names are single words consisting of alphanumerics
plus isolated (no two adjacent) internal hyphens, underscores, and
periods; no whitespace or other characters are allowed.  Footnote
labels (Footnotes_ & `Footnote References`_), citation labels
(Citations_ & `Citation References`_), `interpreted text`_ roles, and
some `hyperlink references`_ use the simple reference name syntax.

Reference names using punctuation or whose names are phrases (two or
more space-separated words) are called "phrase-references".
Phrase-references are expressed by enclosing the phrase in backquotes
and treating the backquoted text as a reference name::

    Want to learn about `my favorite programming language`_?

    .. _my favorite programming language: http://www.python.org

Simple reference names may also optionally use backquotes.

Reference names are whitespace-neutral and case-insensitive.  When
resolving reference names internally:

- whitespace is normalized (one or more spaces, horizontal or vertical
  tabs, newlines, carriage returns, or form feeds, are interpreted as
  a single space), and

- case is normalized (all alphabetic characters are converted to
  lowercase).

For example, the following `hyperlink references`_ are equivalent::

    - `A HYPERLINK`_
    - `a    hyperlink`_
    - `A
      Hyperlink`_

Hyperlinks_, footnotes_, and citations_ all share the same namespace
for reference names.  The labels of citations (simple reference names)
and manually-numbered footnotes (numbers) are entered into the same
database as other hyperlink names.  This means that a footnote
(defined as "``.. [1]``") which can be referred to by a footnote
reference (``[1]_``), can also be referred to by a plain hyperlink
reference (1_).  Of course, each type of reference (hyperlink,
footnote, citation) may be processed and rendered differently.  Some
care should be taken to avoid reference name conflicts.

Document Structure
==================

Document
--------

Doctree element: document.

The top-level element of a parsed reStructuredText document is the
"document" element.  After initial parsing, the document element is a
simple container for a document fragment, consisting of `body
elements`_, transitions_, and sections_, but lacking a document title
or other bibliographic elements.  The code that calls the parser may
choose to run one or more optional post-parse transforms_,
rearranging the document fragment into a complete document with a
title and possibly other metadata elements (author, date, etc.; see
`Bibliographic Fields`_).

Specifically, there is no way to indicate a document title and
subtitle explicitly in reStructuredText.  Instead, a lone top-level
section title (see Sections_ below) can be treated as the document
title.  Similarly, a lone second-level section title immediately after
the "document title" can become the document subtitle.  The rest of
the sections are then lifted up a level or two.  See the `DocTitle
transform`_ for details.

Sections
--------

Doctree elements: section, title.

Sections are identified through their titles, which are marked up with
adornment: "underlines" below the title text, or underlines and
matching "overlines" above the title.  An underline/overline is a
single repeated punctuation character that begins in column 1 and
forms a line extending at least as far as the right edge of the title
text.  Specifically, an underline/overline character may be any
non-alphanumeric printable 7-bit ASCII character [#]_.  When an
overline is used, the length and character used must match the
underline.  Underline-only adornment styles are distinct from
overline-and-underline styles that use the same character.  There may
be any number of levels of section titles, although some output
formats may have limits (HTML has 6 levels).

.. [#] The following are all valid section title adornment
   characters::

       ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~

   Some characters are more suitable than others.  The following are
   recommended::

       = - ` : . ' " ~ ^ _ * + #

Rather than imposing a fixed number and order of section title
adornment styles, the order enforced will be the order as encountered.
The first style encountered will be an outermost title (like HTML H1),
the second style will be a subtitle, the third will be a subsubtitle,
and so on.

Below are examples of section title styles::

    ===============
     Section Title
    ===============

    ---------------
     Section Title
    ---------------

    Section Title
    =============

    Section Title
    -------------

    Section Title
    `````````````

    Section Title
    '''''''''''''

    Section Title
    .............

    Section Title
    ~~~~~~~~~~~~~

    Section Title
    *************

    Section Title
    +++++++++++++

    Section Title
    ^^^^^^^^^^^^^

When a title has both an underline and an overline, the title text may
be inset, as in the first two examples above.  This is merely
aesthetic and not significant.  Underline-only title text may *not* be
inset.

A blank line after a title is optional.  All text blocks up to the
next title of the same or higher level are included in a section (or
subsection, etc.).

All section title styles need not be used, nor need any specific
section title style be used.  However, a document must be consistent
in its use of section titles: once a hierarchy of title styles is
established, sections must use that hierarchy.

Each section title automatically generates a hyperlink target pointing
to the section.  The text of the hyperlink target (the "reference
name") is the same as that of the section title.  See `Implicit
Hyperlink Targets`_ for a complete description.

Sections may contain `body elements`_, transitions_, and nested
sections.

Transitions
-----------

Doctree element: transition.

    Instead of subheads, extra space or a type ornament between
    paragraphs may be used to mark text divisions or to signal
    changes in subject or emphasis.

    (The Chicago Manual of Style, 14th edition, section 1.80)

Transitions are commonly seen in novels and short fiction, as a gap
spanning one or more lines, with or without a type ornament such as a
row of asterisks.  Transitions separate other body elements.  A
transition should not begin or end a section or document, nor should
two transitions be immediately adjacent.

The syntax for a transition marker is a horizontal line of 4 or more
repeated punctuation characters.  The syntax is the same as section
title underlines without title text.  Transition markers require blank
lines before and after::

    Para.

    ----------

    Para.

Unlike section title underlines, no hierarchy of transition markers is
enforced, nor do differences in transition markers accomplish
anything.  It is recommended that a single consistent style be used.

The processing system is free to render transitions in output in any
way it likes.  For example, horizontal rules (``<hr>``) in HTML output
would be an obvious choice.

Body Elements
=============

Paragraphs
----------

Doctree element: paragraph.

Paragraphs consist of blocks of left-aligned text with no markup
indicating any other body element.  Blank lines separate paragraphs
from each other and from other body elements.  Paragraphs may contain
`inline markup`_.

Syntax diagram::

    +------------------------------+
    | paragraph                    |
    |                              |
    +------------------------------+

    +------------------------------+
    | paragraph                    |
    |                              |
    +------------------------------+

Bullet Lists
------------

Doctree elements: bullet_list, list_item.

A text block which begins with a "-", "*", or "+", followed by
whitespace, is a bullet list item (a.k.a. "unordered" list item).
List item bodies must be left-aligned and indented relative to the
bullet; the text immediately after the bullet determines the
indentation.  For example::

    - This is the first bullet list item.  The blank line above the
      first list item is required; blank lines between list items
      (such as below this paragraph) are optional.

    - This is the first paragraph in the second item in the list.

      This is the second paragraph in the second item in the list.
      The blank line above this paragraph is required.  The left edge
      of this paragraph lines up with the paragraph above, both
      indented relative to the bullet.

      - This is a sublist.  The bullet lines up with the left edge of
        the text blocks above.  A sublist is a new list so requires a
        blank line above and below.

    - This is the third item of the main list.

    This paragraph is not part of the list.

Here are examples of **incorrectly** formatted bullet lists::

    - This first line is fine.
    A blank line is required between list items and paragraphs.
    (Warning)

    - The following line appears to be a new sublist, but it is not:
      - This is a paragraph continuation, not a sublist (since there's
        no blank line).  This line is also incorrectly indented.
      - Warnings may be issued by the implementation.

Syntax diagram::

    +------+-----------------------+
    | "- " | list item             |
    +------| (body elements)+      |
           +-----------------------+

Enumerated Lists
----------------

Doctree elements: enumerated_list, list_item.

Enumerated lists (a.k.a. "ordered" lists) are similar to bullet lists,
but use enumerators instead of bullets.  An enumerator consists of an
enumeration sequence member and formatting, followed by whitespace.
The following enumeration sequences are recognized:

- arabic numerals: 1, 2, 3, ... (no upper limit).
- uppercase alphabet characters: A, B, C, ..., Z.
- lower-case alphabet characters: a, b, c, ..., z.
- uppercase Roman numerals: I, II, III, IV, ..., MMMMCMXCIX (4999).
- lowercase Roman numerals: i, ii, iii, iv, ..., mmmmcmxcix (4999).

The following formatting types are recognized:

- suffixed with a period: "1.", "A.", "a.", "I.", "i.".
- surrounded by parentheses: "(1)", "(A)", "(a)", "(I)", "(i)".
- suffixed with a right-parenthesis: "1)", "A)", "a)", "I)", "i)".

While parsing an enumerated list, a new list will be started whenever:

- An enumerator is encountered which does not have the same format and
  sequence type as the current list (e.g. "1.", "(a)" produces two
  separate lists).

- The enumerators are not in sequence (e.g., "1.", "3." produces two
  separate lists).

It is recommended that the enumerator of the first list item be
ordinal-1 ("1", "A", "a", "I", or "i").  Although other start-values
will be recognized, they may not be supported by the output format.  A
level-1 [info] system message will be generated for any list beginning
with a non-ordinal-1 enumerator.

Lists using Roman numerals must begin with "I"/"i" or a
multi-character value, such as "II" or "XV".  Any other
single-character Roman numeral ("V", "X", "L", "C", "D", "M") will be
interpreted as a letter of the alphabet, not as a Roman numeral.
Likewise, lists using letters of the alphabet may not begin with
"I"/"i", since these are recognized as Roman numeral 1.

The second line of each enumerated list item is checked for validity.
This is to prevent ordinary paragraphs from being mistakenly
interpreted as list items, when they happen to begin with text
identical to enumerators.  For example, this text is parsed as an
ordinary paragraph::

    A. Einstein was a really
    smart dude.

However, ambiguity cannot be avoided if the paragraph consists of only
one line.  This text is parsed as an enumerated list item::

    A. Einstein was a really smart dude.

If a single-line paragraph begins with text identical to an enumerator
("A.", "1.", "(b)", "I)", etc.), the first character will have to be
escaped in order to have the line parsed as an ordinary paragraph::

    \A. Einstein was a really smart dude.

Nested enumerated lists must be created with indentation.  For
example::

    1. Item 1.

       a) Item 1a.
       b) Item 1b.

Example syntax diagram::

    +-------+----------------------+
    | "1. " | list item            |
    +-------| (body elements)+     |
            +----------------------+

Definition Lists
----------------

Doctree elements: definition_list, definition_list_item, term,
classifier, definition.

Each definition list item contains a term, an optional classifier, and
a definition.  A term is a simple one-line word or phrase.  An
optional classifier may follow the term on the same line, after an
inline " : " (space, colon, space).  A definition is a block indented
relative to the term, and may contain multiple paragraphs and other
body elements.  There may be no blank line between a term line and a
definition block (this distinguishes definition lists from `block
quotes`_).  Blank lines are required before the first and after the
last definition list item, but are optional in-between.  For example::

    term 1
        Definition 1.

    term 2
        Definition 2, paragraph 1.

        Definition 2, paragraph 2.

    term 3 : classifier
        Definition 3.

Inline markup is parsed in the term line before the term/classifier
delimiter (" : ") is recognized.  The delimiter will only be
recognized if it appears outside of any inline markup.

A definition list may be used in various ways, including:

- As a dictionary or glossary.  The term is the word itself, a
  classifier may be used to indicate the usage of the term (noun,
  verb, etc.), and the definition follows.

- To describe program variables.  The term is the variable name, a
  classifier may be used to indicate the type of the variable (string,
  integer, etc.), and the definition describes the variable's use in
  the program.  This usage of definition lists supports the classifier
  syntax of Grouch_, a system for describing and enforcing a Python
  object schema.

Syntax diagram::

    +---------------------------+
    | term [ " : " classifier ] |
    +--+------------------------+--+
       | definition                |
       | (body elements)+          |
       +---------------------------+

Field Lists
-----------

Doctree elements: field_list, field, field_name, field_body.

Field lists are used as part of an extension syntax, such as options
for directives_, or database-like records meant for further
processing.  They may also be used for two-column table-like
structures resembling database records (label & data pairs).
Applications of reStructuredText may recognize field names and
transform fields or field bodies in certain contexts.  For examples,
see `Bibliographic Fields`_ below, or the "image" and "meta"
directives in `reStructuredText Directives`_.

Field lists are mappings from field names to field bodies, modeled on
RFC822_ headers.  A field name is made up of one or more letters,
numbers, whitespace, and punctuation, except colons (":").  Inline
markup is parsed in field names.  Field names are case-insensitive
when further processed or transformed.  The field name, along with a
single colon prefix and suffix, together form the field marker.  The
field marker is followed by whitespace and the field body.  The field
body may contain multiple body elements, indented relative to the
field marker.  The first line after the field name marker determines
the indentation of the field body.  For example::

    :Date: 2001-08-16
    :Version: 1
    :Authors: - Me
              - Myself
              - I
    :Indentation: Since the field marker may be quite long, the second
       and subsequent lines of the field body do not have to line up
       with the first line, but they must be indented relative to the
       field name marker, and they must line up with each other.
    :Parameter i: integer

The interpretation of individual words in a multi-word field name is
up to the application.  The application may specify a syntax for the
field name.  For example, second and subsequent words may be treated
as "arguments", quoted phrases may be treated as a single argument,
and direct support for the "name=value" syntax may be added.

Standard RFC822_ headers cannot be used for this construct because
they are ambiguous.  A word followed by a colon at the beginning of a
line is common in written text.  However, in well-defined contexts
such as when a field list invariably occurs at the beginning of a
document (PEPs and email messages), standard RFC822 headers could be
used.

Syntax diagram (simplified)::

    +--------------------+----------------------+
    | ":" field name ":" | field body           |
    +-------+------------+                      |
            | (body elements)+                  |
            +-----------------------------------+

Bibliographic Fields
````````````````````

Doctree elements: docinfo, author, authors, organization, contact,
version, status, date, copyright, field, topic.

When a field list is the first non-comment element in a document
(after the document title, if there is one), it may have its fields
transformed to document bibliographic data.  This bibliographic data
corresponds to the front matter of a book, such as the title page and
copyright page.

Certain registered field names (listed below) are recognized and
transformed to the corresponding doctree elements, most becoming child
elements of the "docinfo" element.  No ordering is required of these
fields, although they may be rearranged to fit the document structure,
as noted.  Unless otherwise indicated below, each of the bibliographic
elements' field bodies may contain a single paragraph only.  Field
bodies may be checked for `RCS keywords`_ and cleaned up.  Any
unrecognized fields will remain as generic fields in the docinfo
element.

The registered bibliographic field names and their corresponding
doctree elements are as follows:

- Field name "Author": author element.
- "Authors": authors.
- "Organization": organization.
- "Contact": contact.
- "Address": address.
- "Version": version.
- "Status": status.
- "Date": date.
- "Copyright": copyright.
- "Dedication": topic.
- "Abstract": topic.

The "Authors" field may contain either: a single paragraph consisting
of a list of authors, separated by ";" or ","; or a bullet list whose
elements each contain a single paragraph per author.  ";" is checked
first, so "Doe, Jane; Doe, John" will work.  In some languages
(e.g. Swedish), there is no singular/plural distinction between
"Author" and "Authors", so only an "Authors" field is provided, and a
single name is interpreted as an "Author".  If a single name contains
a comma, end it with a semicolon to disambiguate: ":Authors: Doe,
Jane;".

The "Address" field is for a multi-line surface mailing address.  A
specialized form of line block`_ (see `reStructuredText Directives`_),
newlines and whitespace will be preserved.

The "Dedication" and "Abstract" fields may contain arbitrary body
elements.  Only one of each is allowed.  They become topic elements
with "Dedication" or "Abstract" titles (or language equivalents)
immediately following the docinfo element.

This field-name-to-element mapping can be replaced for other
languages.  See the `DocInfo transform`_ implementation documentation
for details.

Unregistered/generic fields may contain one or more paragraphs or
arbitrary body elements.

RCS Keywords
````````````

`Bibliographic fields`_ recognized by the parser are normally checked
for RCS [#]_ keywords and cleaned up [#]_.  RCS keywords may be
entered into source files as "$keyword$", and once stored under RCS or
CVS [#]_, they are expanded to "$keyword: expansion text $".  For
example, a "Status" field will be transformed to a "status" element::

    :Status: $keyword: expansion text $

.. [#] Revision Control System.
.. [#] RCS keyword processing can be turned off (unimplemented).
.. [#] Concurrent Versions System.  CVS uses the same keywords as RCS.

Processed, the "status" element's text will become simply "expansion
text".  The dollar sign delimiters and leading RCS keyword name are
removed.

The RCS keyword processing only kicks in when both of these conditions
hold:

1. The field list is in bibliographic context (first non-comment
   construct in the document, after a document title if there is
   one).

2. The field name is a recognized bibliographic field name.

Option Lists
------------

Doctree elements: option_list, option_list_item, option_group, option,
option_string, option_argument, description.

Option lists are two-column lists of command-line options and
descriptions, documenting a program's options.  For example::

    -a         Output all.
    -b         Output both (this description is
               quite long).
    -c arg     Output just arg.
    --long     Output all day long.

    -p         This option has two paragraphs in the description.
               This is the first.

               This is the second.  Blank lines may be omitted between
               options (as above) or left in (as here and below).

    --very-long-option  A VMS-style option.  Note the adjustment for
                        the required two spaces.

    --an-even-longer-option
               The description can also start on the next line.

    -2, --two  This option has two variants.

    -f FILE, --file=FILE  These two options are synonyms; both have
                          arguments.

    /V         A VMS/DOS-style option.

There are several types of options recognized by reStructuredText:

- Short POSIX options consist of one dash and an option letter.
- Long POSIX options consist of two dashes and an option word; some
  systems use a single dash.
- Old GNU-style "plus" options consist of one plus and an option
  letter ("plus" options are deprecated now, their use discouraged).
- DOS/VMS options consist of a slash and an option letter or word.

Please note that both POSIX-style and DOS/VMS-style options may be
used by DOS or Windows software.  These and other variations are
sometimes used mixed together.  The names above have been chosen for
convenience only.

The syntax for short and long POSIX options is based on the syntax
supported by Python's getopt.py_ module, which implements an option
parser similar to the `GNU libc getopt_long()`_ function but with some
restrictions.  There are many variant option systems, and
reStructuredText option lists do not support all of them.

Although long POSIX and DOS/VMS option words may be allowed to be
truncated by the operating system or the application when used on the
command line, reStructuredText option lists do not show or support
this with any special syntax.  The complete option word should be
given, supported by notes about truncation if and when applicable.

Options may be followed by an argument placeholder, whose role and
syntax should be explained in the description text.  Either a space or
an equals sign may be used as a delimiter between options and option
argument placeholders; short options ("-" or "+" prefix only) may omit
the delimiter.  Option arguments may take one of two forms:

- Begins with a letter (``[a-zA-Z]``) and subsequently consists of
  letters, numbers, underscores and hyphens (``[a-zA-Z0-9_-]``).
- Begins with an open-angle-bracket (``<``) and ends with a
  close-angle-bracket (``>``); any characters except angle brackets
  are allowed internally.

Multiple option "synonyms" may be listed, sharing a single
description.  They must be separated by comma-space.

There must be at least two spaces between the option(s) and the
description.  The description may contain multiple body elements.  The
first line after the option marker determines the indentation of the
description.  As with other types of lists, blank lines are required
before the first option list item and after the last, but are optional
between option entries.

Syntax diagram (simplified)::

    +----------------------------+-------------+
    | option [" " argument] "  " | description |
    +-------+--------------------+             |
            | (body elements)+                 |
            +----------------------------------+

Literal Blocks
--------------

Doctree element: literal_block.

A paragraph consisting of two colons ("::") signifies that the
following text block(s) comprise a literal block.  The literal block
must either be indented or quoted (see below).  No markup processing
is done within a literal block.  It is left as-is, and is typically
rendered in a monospaced typeface::

    This is a typical paragraph.  An indented literal block follows.

    ::

        for a in [5,4,3,2,1]:   # this is program code, shown as-is
            print a
        print "it's..."
        # a literal block continues until the indentation ends

    This text has returned to the indentation of the first paragraph,
    is outside of the literal block, and is therefore treated as an
    ordinary paragraph.

The paragraph containing only "::" will be completely removed from the
output; no empty paragraph will remain.

As a convenience, the "::" is recognized at the end of any paragraph.
If immediately preceded by whitespace, both colons will be removed
from the output (this is the "partially minimized" form).  When text
immediately precedes the "::", *one* colon will be removed from the
output, leaving only one colon visible (i.e., "::" will be replaced by
":"; this is the "fully minimized" form).

In other words, these are all equivalent (please pay attention to the
colons after "Paragraph"):

1. Expanded form::

      Paragraph:

      ::

          Literal block

2. Partially minimized form::

      Paragraph: ::

          Literal block

3. Fully minimized form::

      Paragraph::

          Literal block

All whitespace (including line breaks, but excluding minimum
indentation for indented literal blocks) is preserved.  Blank lines
are required before and after a literal block, but these blank lines
are not included as part of the literal block.

Indented Literal Blocks
```````````````````````

Indented literal blocks are indicated by indentation relative to the
surrounding text (leading whitespace on each line).  The minimum
indentation will be removed from each line of an indented literal
block.  The literal block need not be contiguous; blank lines are
allowed between sections of indented text.  The literal block ends
with the end of the indentation.

Syntax diagram::

    +------------------------------+
    | paragraph                    |
    | (ends with "::")             |
    +------------------------------+
       +---------------------------+
       | indented literal block    |
       +---------------------------+

Quoted Literal Blocks
`````````````````````

Quoted literal blocks are unindented contiguous blocks of text where
each line begins with the same non-alphanumeric printable 7-bit ASCII
character [#]_.  A blank line ends a quoted literal block.  The
quoting characters are preserved in the processed document.

.. [#]
   The following are all valid quoting characters::

       ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~

   Note that these are the same characters as are valid for title
   adornment of sections_.

Possible uses include literate programming in Haskell and email
quoting::

    John Doe wrote::

    >> Great idea!
    >
    > Why didn't I think of that?

    You just did!  ;-)

Syntax diagram::

    +------------------------------+
    | paragraph                    |
    | (ends with "::")             |
    +------------------------------+
    +------------------------------+
    | ">" per-line-quoted          |
    | ">" contiguous literal block |
    +------------------------------+

Block Quotes
------------

Doctree element: block_quote, attribution.

A text block that is indented relative to the preceding text, without
markup indicating it to be a literal block, is a block quote.  All
markup processing (for body elements and inline markup) continues
within the block quote::

    This is an ordinary paragraph, introducing a block quote.

        "It is my business to know things.  That is my trade."

        -- Sherlock Holmes

If the final block of a block quote begins with "--", "---", or a true
em-dash (flush left within the block quote), it is interpreted as an
attribution.  If the attribution consists of multiple lines, the left
edges of the second and subsequent lines must align.

Blank lines are required before and after a block quote, but these
blank lines are not included as part of the block quote.

Syntax diagram::

    +------------------------------+
    | (current level of            |
    | indentation)                 |
    +------------------------------+
       +---------------------------+
       | block quote               |
       | (body elements)+          |
       |                           |
       | -- attribution text       |
       |    (optional)             |
       +---------------------------+

Doctest Blocks
--------------

Doctree element: doctest_block.

Doctest blocks are interactive Python sessions cut-and-pasted into
docstrings.  They are meant to illustrate usage by example, and
provide an elegant and powerful testing environment via the `doctest
module`_ in the Python standard library.

Doctest blocks are text blocks which begin with ``">>> "``, the Python
interactive interpreter main prompt, and end with a blank line.
Doctest blocks are treated as a special case of literal blocks,
without requiring the literal block syntax.  If both are present, the
literal block syntax takes priority over Doctest block syntax::

    This is an ordinary paragraph.

    >>> print 'this is a Doctest block'
    this is a Doctest block

    The following is a literal block::

        >>> This is not recognized as a doctest block by
        reStructuredText.  It *will* be recognized by the doctest
        module, though!

Indentation is not required for doctest blocks.

Tables
------

Doctree elements: table, tgroup, colspec, thead, tbody, row, entry.

ReStructuredText provides two syntaxes for delineating table cells:
`Grid Tables`_ and `Simple Tables`_.

As with other body elements, blank lines are required before and after
tables.  Tables' left edges should align with the left edge of
preceding text blocks; if indented, the table is considered to be part
of a block quote.

Once isolated, each table cell is treated as a miniature document; the
top and bottom cell boundaries act as delimiting blank lines.  Each
cell contains zero or more body elements.  Cell contents may include
left and/or right margins, which are removed before processing.

Grid Tables
```````````

Grid tables provide a complete table representation via grid-like
"ASCII art".  Grid tables allow arbitrary cell contents (body
elements), and both row and column spans.  However, grid tables can be
cumbersome to produce, especially for simple data sets.  The `Emacs
table mode`_ is a tool that allows easy editing of grid tables, in
Emacs.  See `Simple Tables`_ for a simpler (but limited)
representation.

Grid tables are described with a visual grid made up of the characters
"-", "=", "|", and "+".  The hyphen ("-") is used for horizontal lines
(row separators).  The equals sign ("=") may be used to separate
optional header rows from the table body (not supported by the `Emacs
table mode`_).  The vertical bar ("|") is used for vertical lines
(column separators).  The plus sign ("+") is used for intersections of
horizontal and vertical lines.  Example::

    +------------------------+------------+----------+----------+
    | Header row, column 1   | Header 2   | Header 3 | Header 4 |
    | (header rows optional) |            |          |          |
    +========================+============+==========+==========+
    | body row 1, column 1   | column 2   | column 3 | column 4 |
    +------------------------+------------+----------+----------+
    | body row 2             | Cells may span columns.          |
    +------------------------+------------+---------------------+
    | body row 3             | Cells may  | - Table cells       |
    +------------------------+ span rows. | - contain           |
    | body row 4             |            | - body elements.    |
    +------------------------+------------+---------------------+

Some care must be taken with grid tables to avoid undesired
interactions with cell text in rare cases.  For example, the following
table contains a cell in row 2 spanning from column 2 to column 4::

    +--------------+----------+-----------+-----------+
    | row 1, col 1 | column 2 | column 3  | column 4  |
    +--------------+----------+-----------+-----------+
    | row 2        |                                  |
    +--------------+----------+-----------+-----------+
    | row 3        |          |           |           |
    +--------------+----------+-----------+-----------+

If a vertical bar is used in the text of that cell, it could have
unintended effects if accidentally aligned with column boundaries::

    +--------------+----------+-----------+-----------+
    | row 1, col 1 | column 2 | column 3  | column 4  |
    +--------------+----------+-----------+-----------+
    | row 2        | Use the command ``ls | more``.   |
    +--------------+----------+-----------+-----------+
    | row 3        |          |           |           |
    +--------------+----------+-----------+-----------+

Several solutions are possible.  All that is needed is to break the
continuity of the cell outline rectangle.  One possibility is to shift
the text by adding an extra space before::

    +--------------+----------+-----------+-----------+
    | row 1, col 1 | column 2 | column 3  | column 4  |
    +--------------+----------+-----------+-----------+
    | row 2        |  Use the command ``ls | more``.  |
    +--------------+----------+-----------+-----------+
    | row 3        |          |           |           |
    +--------------+----------+-----------+-----------+

Another possibility is to add an extra line to row 2::

    +--------------+----------+-----------+-----------+
    | row 1, col 1 | column 2 | column 3  | column 4  |
    +--------------+----------+-----------+-----------+
    | row 2        | Use the command ``ls | more``.   |
    |              |                                  |
    +--------------+----------+-----------+-----------+
    | row 3        |          |           |           |
    +--------------+----------+-----------+-----------+

Simple Tables
`````````````

Simple tables provide a compact and easy to type but limited
row-oriented table representation for simple data sets.  Cell contents
are typically single paragraphs, although arbitrary body elements may
be represented in most cells.  Simple tables allow multi-line rows (in
all but the first column) and column spans, but not row spans.  See
`Grid Tables`_ above for a complete table representation.

Simple tables are described with horizontal borders made up of "=" and
"-" characters.  The equals sign ("=") is used for top and bottom
table borders, and to separate optional header rows from the table
body.  The hyphen ("-") is used to indicate column spans in a single
row by underlining the joined columns.

A simple table begins with a top border of equals signs with one or
more spaces at each column boundary (two or more spaces recommended).
Regardless of spans, the top border *must* fully describe all table
columns.  There must be at least two columns in the table (to
differentiate it from section headers).  The last of the optional
header rows is underlined with '=', again with spaces at column
boundaries.  There may not be a blank line below the header row
separator; it would be interpreted as the bottom border of the table.
The bottom boundary of the table consists of '=' underlines, also with
spaces at column boundaries.  For example, here is a truth table, a
three-column table with one header row and four body rows::

    =====  =====  =======
      A      B    A and B
    =====  =====  =======
    False  False  False
    True   False  False
    False  True   False
    True   True   True
    =====  =====  =======

Underlines of '-' may be used to indicate column spans by "filling in"
column margins to join adjacent columns.  Column span underlines must
be complete (they must cover all columns) and align with established
column boundaries.  Text lines containing column span underlines may
not contain any other text.  A column span underline applies only to
one row immediately above it.  For example, here is a table with a
column span in the header::

    =====  =====  ======
       Inputs     Output
    ------------  ------
      A      B    A or B
    =====  =====  ======
    False  False  False
    True   False  True
    False  True   True
    True   True   True
    =====  =====  ======

Each line of text must contain spaces at column boundaries, except
where cells have been joined by column spans.  Each line of text
starts a new row, except when there is a blank cell in the first
column.  In that case, that line of text is parsed as a continuation
line.  For this reason, cells in the first column of new rows (*not*
continuation lines) *must* contain some text; blank cells would lead
to a misinterpretation.  An empty comment ("..") is sufficient and
will be omitted from the processed output (see Comments_ below).
Also, this mechanism limits cells in the first column to only one line
of text.  Use `grid tables`_ if this limitation is unacceptable.

Underlines of '-' may also be used to visually separate rows, even if
there are no column spans.  This is especially useful in long tables,
where rows are many lines long.

Blank lines are permitted within simple tables.  Their interpretation
depends on the context.  Blank lines *between* rows are ignored.
Blank lines *within* multi-line rows may separate paragraphs or other
body elements within cells.

The rightmost column is unbounded; text may continue past the edge of
the table (as indicated by the table borders).  However, it is
recommended that borders be made long enough to contain the entire
text.

The following example illustrates continuation lines (row 2 consists
of two lines of text, and four lines for row 3), a blank line
separating paragraphs (row 3, column 2), and text extending past the
right edge of the table::

    =====  =====
    col 1  col 2
    =====  =====
    1      Second column of row 1.
    2      Second column of row 2.
           Second line of paragraph.
    3      - Second column of row 3.

           - Second item in bullet
             list (row 3, column 2).
    =====  =====

Explicit Markup Blocks
----------------------

An explicit markup block is a text block:

- whose first line begins with ".." followed by whitespace (the
  "explicit markup start"),
- whose second and subsequent lines (if any) are indented relative to
  the first, and
- which ends before an unindented line.

Explicit markup blocks are analogous to bullet list items, with ".."
as the bullet.  The text immediately after the explicit markup start
determines the indentation of the block body.  Blank lines are
required between explicit markup blocks and other elements, but are
optional between explicit markup blocks where unambiguous.

The explicit markup syntax is used for footnotes, citations, hyperlink
targets, directives, substitution definitions, and comments.

Footnotes
`````````

Doctree elements: footnote, label.

Each footnote consists of an explicit markup start (".. "), a left
square bracket, the footnote label, a right square bracket, and
whitespace, followed by indented body elements.  A footnote label can
be:

- a whole decimal number consisting of one or more digits,

- a single "#" (denoting `auto-numbered footnotes`_),

- a "#" followed by a simple reference name (an `autonumber label`_),
  or

- a single "*" (denoting `auto-symbol footnotes`_).

If the first body element within a footnote is a simple paragraph, it
may begin on the same line as the footnote label.  Other elements must
begin on a new line, consistently indented (by at least 3 spaces) and
left-aligned.

Footnotes may occur anywhere in the document, not only at the end.
Where or how they appear in the processed output depends on the
processing system.

Here is a manually numbered footnote::

    .. [1] Body elements go here.

Each footnote automatically generates a hyperlink target pointing to
itself.  The text of the hyperlink target name is the same as that of
the footnote label.  `Auto-numbered footnotes`_ generate a number as
their footnote label and reference name.  See `Implicit Hyperlink
Targets`_ for a complete description of the mechanism.

Syntax diagram::

    +-------+-------------------------+
    | ".. " | "[" label "]" footnote  |
    +-------+                         |
            | (body elements)+        |
            +-------------------------+

Auto-Numbered Footnotes
.......................

A number sign ("#") may be used as the first character of a footnote
label to request automatic numbering of the footnote or footnote
reference.

The first footnote to request automatic numbering is assigned the
label "1", the second is assigned the label "2", and so on (assuming
there are no manually numbered footnotes present; see `Mixed Manual
and Auto-Numbered Footnotes`_ below).  A footnote which has
automatically received a label "1" generates an implicit hyperlink
target with name "1", just as if the label was explicitly specified.

.. _autonumber label: `autonumber labels`_

A footnote may specify a label explicitly while at the same time
requesting automatic numbering: ``[#label]``.  These labels are called
_`autonumber labels`.  Autonumber labels do two things:

- On the footnote itself, they generate a hyperlink target whose name
  is the autonumber label (doesn't include the "#").

- They allow an automatically numbered footnote to be referred to more
  than once, as a footnote reference or hyperlink reference.  For
  example::

      If [#note]_ is the first footnote reference, it will show up as
      "[1]".  We can refer to it again as [#note]_ and again see
      "[1]".  We can also refer to it as note_ (an ordinary internal
      hyperlink reference).

      .. [#note] This is the footnote labeled "note".

The numbering is determined by the order of the footnotes, not by the
order of the references.  For footnote references without autonumber
labels (``[#]_``), the footnotes and footnote references must be in
the same relative order but need not alternate in lock-step.  For
example::

    [#]_ is a reference to footnote 1, and [#]_ is a reference to
    footnote 2.

    .. [#] This is footnote 1.
    .. [#] This is footnote 2.
    .. [#] This is footnote 3.

    [#]_ is a reference to footnote 3.

Special care must be taken if footnotes themselves contain
auto-numbered footnote references, or if multiple references are made
in close proximity.  Footnotes and references are noted in the order
they are encountered in the document, which is not necessarily the
same as the order in which a person would read them.

Auto-Symbol Footnotes
.....................

An asterisk ("*") may be used for footnote labels to request automatic
symbol generation for footnotes and footnote references.  The asterisk
may be the only character in the label.  For example::

    Here is a symbolic footnote reference: [*]_.

    .. [*] This is the footnote.

A transform will insert symbols as labels into corresponding footnotes
and footnote references.  The number of references must be equal to
the number of footnotes.  One symbol footnote cannot have multiple
references.

The standard Docutils system uses the following symbols for footnote
marks [#]_:

- asterisk/star ("*")
- dagger (HTML character entity "&dagger;", Unicode U+02020)
- double dagger ("&Dagger;"/U+02021)
- section mark ("&sect;"/U+000A7)
- pilcrow or paragraph mark ("&para;"/U+000B6)
- number sign ("#")
- spade suit ("&spades;"/U+02660)
- heart suit ("&hearts;"/U+02665)
- diamond suit ("&diams;"/U+02666)
- club suit ("&clubs;"/U+02663)

.. [#] This list was inspired by the list of symbols for "Note
   Reference Marks" in The Chicago Manual of Style, 14th edition,
   section 12.51.  "Parallels" ("||") were given in CMoS instead of
   the pilcrow.  The last four symbols (the card suits) were added
   arbitrarily.

If more than ten symbols are required, the same sequence will be
reused, doubled and then tripled, and so on ("**" etc.).

.. Note:: When using auto-symbol footnotes, the choice of output
   encoding is important.  Many of the symbols used are not encodable
   in certain common text encodings such as Latin-1 (ISO 8859-1).  The
   use of UTF-8 for the output encoding is recommended.  An
   alternative for HTML and XML output is to use the
   "xmlcharrefreplace" `output encoding error handler`__ (available in
   Python 2.3 & later).

__ ../../user/config.html#output-encoding-error-handler

Mixed Manual and Auto-Numbered Footnotes
........................................

Manual and automatic footnote numbering may both be used within a
single document, although the results may not be expected.  Manual
numbering takes priority.  Only unused footnote numbers are assigned
to auto-numbered footnotes.  The following example should be
illustrative::

    [2]_ will be "2" (manually numbered),
    [#]_ will be "3" (anonymous auto-numbered), and
    [#label]_ will be "1" (labeled auto-numbered).

    .. [2] This footnote is labeled manually, so its number is fixed.

    .. [#label] This autonumber-labeled footnote will be labeled "1".
       It is the first auto-numbered footnote and no other footnote
       with label "1" exists.  The order of the footnotes is used to
       determine numbering, not the order of the footnote references.

    .. [#] This footnote will be labeled "3".  It is the second
       auto-numbered footnote, but footnote label "2" is already used.

Citations
`````````

Citations are identical to footnotes except that they use only
non-numeric labels such as ``[note]`` or ``[GVR2001]``.  Citation
labels are simple `reference names`_ (case-insensitive single words
consisting of alphanumerics plus internal hyphens, underscores, and
periods; no whitespace).  Citations may be rendered separately and
differently from footnotes.  For example::

    Here is a citation reference: [CIT2002]_.

    .. [CIT2002] This is the citation.  It's just like a footnote,
       except the label is textual.

.. _hyperlinks:

Hyperlink Targets
`````````````````

Doctree element: target.

These are also called _`explicit hyperlink targets`, to differentiate
them from `implicit hyperlink targets`_ defined below.

Hyperlink targets identify a location within or outside of a document,
which may be linked to by `hyperlink references`_.

Hyperlink targets may be named or anonymous.  Named hyperlink targets
consist of an explicit markup start (".. "), an underscore, the
reference name (no trailing underscore), a colon, whitespace, and a
link block::

    .. _hyperlink-name: link-block

Reference names are whitespace-neutral and case-insensitive.  See
`Reference Names`_ for details and examples.

Anonymous hyperlink targets consist of an explicit markup start
(".. "), two underscores, a colon, whitespace, and a link block; there
is no reference name::

    .. __: anonymous-hyperlink-target-link-block

An alternate syntax for anonymous hyperlinks consists of two
underscores, a space, and a link block::

    __ anonymous-hyperlink-target-link-block

See `Anonymous Hyperlinks`_ below.

There are three types of hyperlink targets: internal, external, and
indirect.

1. _`Internal hyperlink targets` have empty link blocks.  They provide
   an end point allowing a hyperlink to connect one place to another
   within a document.  An internal hyperlink target points to the
   element following the target.  For example::

       Clicking on this internal hyperlink will take us to the target_
       below.

       .. _target:

       The hyperlink target above points to this paragraph.

   Internal hyperlink targets may be "chained".  Multiple adjacent
   internal hyperlink targets all point to the same element::

       .. _target1:
       .. _target2:

       The targets "target1" and "target2" are synonyms; they both
       point to this paragraph.

   If the element "pointed to" is an external hyperlink target (with a
   URI in its link block; see #2 below) the URI from the external
   hyperlink target is propagated to the internal hyperlink targets;
   they will all "point to" the same URI.  There is no need to
   duplicate a URI.  For example, all three of the following hyperlink
   targets refer to the same URI::

       .. _Python DOC-SIG mailing list archive:
       .. _archive:
       .. _Doc-SIG: http://mail.python.org/pipermail/doc-sig/

   An inline form of internal hyperlink target is available; see
   `Inline Internal Targets`_.

2. _`External hyperlink targets` have an absolute or relative URI or
   email address in their link blocks.  For example, take the
   following input::

       See the Python_ home page for info.

       `Write to me`_ with your questions.

       .. _Python: http://www.python.org
       .. _Write to me: jdoe@example.com

   After processing into HTML, the hyperlinks might be expressed as::

       See the <a href="http://www.python.org">Python</a> home page
       for info.

       <a href="mailto:jdoe@example.com">Write to me</a> with your
       questions.

   An external hyperlink's URI may begin on the same line as the
   explicit markup start and target name, or it may begin in an
   indented text block immediately following, with no intervening
   blank lines.  If there are multiple lines in the link block, they
   are stripped of leading and trailing whitespace and concatenated.
   The following external hyperlink targets are equivalent::

       .. _one-liner: http://docutils.sourceforge.net/rst.html

       .. _starts-on-this-line: http://
          docutils.sourceforge.net/rst.html

       .. _entirely-below:
          http://docutils.
          sourceforge.net/rst.html

   If an external hyperlink target's URI contains an underscore as its
   last character, it must be escaped to avoid being mistaken for an
   indirect hyperlink target::

       This link_ refers to a file called ``underscore_``.

       .. _link: underscore\_

   It is possible (although not generally recommended) to include URIs
   directly within hyperlink references.  See `Embedded URIs`_ below.

3. _`Indirect hyperlink targets` have a hyperlink reference in their
   link blocks.  In the following example, target "one" indirectly
   references whatever target "two" references, and target "two"
   references target "three", an internal hyperlink target.  In
   effect, all three reference the same thing::

       .. _one: two_
       .. _two: three_
       .. _three:

   Just as with `hyperlink references`_ anywhere else in a document,
   if a phrase-reference is used in the link block it must be enclosed
   in backquotes.  As with `external hyperlink targets`_, the link
   block of an indirect hyperlink target may begin on the same line as
   the explicit markup start or the next line.  It may also be split
   over multiple lines, in which case the lines are joined with
   whitespace before being normalized.

   For example, the following indirect hyperlink targets are
   equivalent::

       .. _one-liner: `A HYPERLINK`_
       .. _entirely-below:
          `a    hyperlink`_
       .. _split: `A
          Hyperlink`_

If a reference name contains a colon followed by whitespace, either:

- the phrase must be enclosed in backquotes::

      .. _`FAQTS: Computers: Programming: Languages: Python`:
         http://python.faqts.com/

- or the colon(s) must be backslash-escaped in the link target::

      .. _Chapter One\: "Tadpole Days":

      It's not easy being green...

See `Implicit Hyperlink Targets`_ below for the resolution of
duplicate reference names.

Syntax diagram::

    +-------+----------------------+
    | ".. " | "_" name ":" link    |
    +-------+ block                |
            |                      |
            +----------------------+

Anonymous Hyperlinks
....................

The `World Wide Web Consortium`_ recommends in its `HTML Techniques
for Web Content Accessibility Guidelines`_ that authors should
"clearly identify the target of each link."  Hyperlink references
should be as verbose as possible, but duplicating a verbose hyperlink
name in the target is onerous and error-prone.  Anonymous hyperlinks
are designed to allow convenient verbose hyperlink references, and are
analogous to `Auto-Numbered Footnotes`_.  They are particularly useful
in short or one-off documents.  However, this feature is easily abused
and can result in unreadable plaintext and/or unmaintainable
documents.  Caution is advised.

Anonymous `hyperlink references`_ are specified with two underscores
instead of one::

    See `the web site of my favorite programming language`__.

Anonymous targets begin with ".. __:"; no reference name is required
or allowed::

    .. __: http://www.python.org

As a convenient alternative, anonymous targets may begin with "__"
only::

    __ http://www.python.org

The reference name of the reference is not used to match the reference
to its target.  Instead, the order of anonymous hyperlink references
and targets within the document is significant: the first anonymous
reference will link to the first anonymous target.  The number of
anonymous hyperlink references in a document must match the number of
anonymous targets.  For readability, it is recommended that targets be
kept close to references.  Take care when editing text containing
anonymous references; adding, removing, and rearranging references
require attention to the order of corresponding targets.

Directives
``````````

Doctree elements: depend on the directive.

Directives are an extension mechanism for reStructuredText, a way of
adding support for new constructs without adding new primary syntax
(directives may support additional syntax locally).  All standard
directives (those implemented and registered in the reference
reStructuredText parser) are described in the `reStructuredText
Directives`_ document, and are always available.  Any other directives
are domain-specific, and may require special action to make them
available when processing the document.

For example, here's how an image may be placed::

    .. image:: mylogo.jpeg

A figure (a graphic with a caption) may placed like this::

    .. figure:: larch.png

       The larch.

An admonition (note, caution, etc.) contains other body elements::

    .. note:: This is a paragraph

       - Here is a bullet list.

Directives are indicated by an explicit markup start (".. ") followed
by the directive type, two colons, and whitespace (together called the
"directive marker").  Directive types are case-insensitive single
words (alphanumerics plus internal hyphens, underscores, and periods;
no whitespace).  Two colons are used after the directive type for
these reasons:

- Two colons are distinctive, and unlikely to be used in common text.

- Two colons avoids clashes with common comment text like::

      .. Danger: modify at your own risk!

- If an implementation of reStructuredText does not recognize a
  directive (i.e., the directive-handler is not installed), a level-3
  (error) system message is generated, and the entire directive block
  (including the directive itself) will be included as a literal
  block.  Thus "::" is a natural choice.

The directive block is consists of any text on the first line of the
directive after the directive marker, and any subsequent indented
text.  The interpretation of the directive block is up to the
directive code.  There are three logical parts to the directive block:

1. Directive arguments.
2. Directive options.
3. Directive content.

Individual directives can employ any combination of these parts.
Directive arguments can be filesystem paths, URLs, title text, etc.
Directive options are indicated using `field lists`_; the field names
and contents are directive-specific.  Arguments and options must form
a contiguous block beginning on the first or second line of the
directive; a blank line indicates the beginning of the directive
content block.  If either arguments and/or options are employed by the
directive, a blank line must separate them from the directive content.
The "figure" directive employs all three parts::

    .. figure:: larch.png
       :scale: 50

       The larch.

Simple directives may not require any content.  If a directive that
does not employ a content block is followed by indented text anyway,
it is an error.  If a block quote should immediately follow a
directive, use an empty comment in-between (see Comments_ below).

Actions taken in response to directives and the interpretation of text
in the directive content block or subsequent text block(s) are
directive-dependent.  See `reStructuredText Directives`_ for details.

Directives are meant for the arbitrary processing of their contents,
which can be transformed into something possibly unrelated to the
original text.  It may also be possible for directives to be used as
pragmas, to modify the behavior of the parser, such as to experiment
with alternate syntax.  There is no parser support for this
functionality at present; if a reasonable need for pragma directives
is found, they may be supported.

Directives do not generate "directive" elements; they are a *parser
construct* only, and have no intrinsic meaning outside of
reStructuredText.  Instead, the parser will transform recognized
directives into (possibly specialized) document elements.  Unknown
directives will trigger level-3 (error) system messages.

Syntax diagram::

    +-------+-------------------------------+
    | ".. " | directive type "::" directive |
    +-------+ block                         |
            |                               |
            +-------------------------------+

Substitution Definitions
````````````````````````

Doctree element: substitution_definition.

Substitution definitions are indicated by an explicit markup start
(".. ") followed by a vertical bar, the substitution text, another
vertical bar, whitespace, and the definition block.  Substitution text
may not begin or end with whitespace.  A substitution definition block
contains an embedded inline-compatible directive (without the leading
".. "), such as an image.  For example::

    The |biohazard| symbol must be used on containers used to
    dispose of medical waste.

    .. |biohazard| image:: biohazard.png

It is an error for a substitution definition block to directly or
indirectly contain a circular substitution reference.

`Substitution references`_ are replaced in-line by the processed
contents of the corresponding definition (linked by matching
substitution text).  Matches are case-sensitive but forgiving; if no
exact match is found, a case-insensitive comparison is attempted.

Substitution definitions allow the power and flexibility of
block-level directives_ to be shared by inline text.  They are a way
to include arbitrarily complex inline structures within text, while
keeping the details out of the flow of text.  They are the equivalent
of SGML/XML's named entities or programming language macros.

Without the substitution mechanism, every time someone wants an
application-specific new inline structure, they would have to petition
for a syntax change.  In combination with existing directive syntax,
any inline structure can be coded without new syntax (except possibly
a new directive).

Syntax diagram::

    +-------+-----------------------------------------------------+
    | ".. " | "|" substitution text "| " directive type "::" data |
    +-------+ directive block                                     |
            |                                                     |
            +-----------------------------------------------------+

Following are some use cases for the substitution mechanism.  Please
note that most of the embedded directives shown are examples only and
have not been implemented.

Objects
    Substitution references may be used to associate ambiguous text
    with a unique object identifier.

    For example, many sites may wish to implement an inline "user"
    directive::

        |Michael| and |Jon| are our widget-wranglers.

        .. |Michael| user:: mjones
        .. |Jon|     user:: jhl

    Depending on the needs of the site, this may be used to index the
    document for later searching, to hyperlink the inline text in
    various ways (mailto, homepage, mouseover Javascript with profile
    and contact information, etc.), or to customize presentation of
    the text (include username in the inline text, include an icon
    image with a link next to the text, make the text bold or a
    different color, etc.).

    The same approach can be used in documents which frequently refer
    to a particular type of objects with unique identifiers but
    ambiguous common names.  Movies, albums, books, photos, court
    cases, and laws are possible.  For example::

        |The Transparent Society| offers a fascinating alternate view
        on privacy issues.

        .. |The Transparent Society| book:: isbn=0738201448

    Classes or functions, in contexts where the module or class names
    are unclear and/or interpreted text cannot be used, are another
    possibility::

        4XSLT has the convenience method |runString|, so you don't
        have to mess with DOM objects if all you want is the
        transformed output.

        .. |runString| function:: module=xml.xslt class=Processor

Images
    Images are a common use for substitution references::

        West led the |H| 3, covered by dummy's |H| Q, East's |H| K,
        and trumped in hand with the |S| 2.

        .. |H| image:: /images/heart.png
           :height: 11
           :width: 11
        .. |S| image:: /images/spade.png
           :height: 11
           :width: 11

        * |Red light| means stop.
        * |Green light| means go.
        * |Yellow light| means go really fast.

        .. |Red light|    image:: red_light.png
        .. |Green light|  image:: green_light.png
        .. |Yellow light| image:: yellow_light.png

        |-><-| is the official symbol of POEE_.

        .. |-><-| image:: discord.png
        .. _POEE: http://www.poee.org/

    The "image" directive has been implemented.

Styles [#]_
    Substitution references may be used to associate inline text with
    an externally defined presentation style::

        Even |the text in Texas| is big.

        .. |the text in Texas| style:: big

    The style name may be meaningful in the context of some particular
    output format (CSS class name for HTML output, LaTeX style name
    for LaTeX, etc), or may be ignored for other output formats (such
    as plaintext).

    .. @@@ This needs to be rethought & rewritten or removed:

       Interpreted text is unsuitable for this purpose because the set
       of style names cannot be predefined - it is the domain of the
       content author, not the author of the parser and output
       formatter - and there is no way to associate a style name
       argument with an interpreted text style role.  Also, it may be
       desirable to use the same mechanism for styling blocks::

           .. style:: motto
              At Bob's Underwear Shop, we'll do anything to get in
              your pants.

           .. style:: disclaimer
              All rights reversed.  Reprint what you like.

    .. [#] There may be sufficient need for a "style" mechanism to
       warrant simpler syntax such as an extension to the interpreted
       text role syntax.  The substitution mechanism is cumbersome for
       simple text styling.

Templates
    Inline markup may be used for later processing by a template
    engine.  For example, a Zope_ author might write::

        Welcome back, |name|!

        .. |name| tal:: replace user/getUserName

    After processing, this ZPT output would result::

        Welcome back,
        <span tal:replace="user/getUserName">name</span>!

    Zope would then transform this to something like "Welcome back,
    David!" during a session with an actual user.

Replacement text
    The substitution mechanism may be used for simple macro
    substitution.  This may be appropriate when the replacement text
    is repeated many times throughout one or more documents,
    especially if it may need to change later.  A short example is
    unavoidably contrived::

        |RST| is a little annoying to type over and over, especially
        when writing about |RST| itself, and spelling out the
        bicapitalized word |RST| every time isn't really necessary for
        |RST| source readability.

        .. |RST| replace:: reStructuredText_
        .. _reStructuredText: http://docutils.sourceforge.net/rst.html

    Substitution is also appropriate when the replacement text cannot
    be represented using other inline constructs, or is obtrusively
    long::

        But still, that's nothing compared to a name like
        |j2ee-cas|__.

        .. |j2ee-cas| replace::
           the Java `TM`:super: 2 Platform, Enterprise Edition Client
           Access Services
        __ http://developer.java.sun.com/developer/earlyAccess/
           j2eecas/

Comments
````````

Doctree element: comment.

Arbitrary indented text may follow the explicit markup start and will
be processed as a comment element.  No further processing is done on
the comment block text; a comment contains a single "text blob".
Depending on the output formatter, comments may be removed from the
processed output.  The only restriction on comments is that they not
use the same syntax as any of the other explicit markup constructs:
substitution definitions, directives, footnotes, citations, or
hyperlink targets.  To ensure that none of the other explicit markup
constructs is recognized, leave the ".." on a line by itself::

    .. This is a comment
    ..
       _so: is this!
    ..
       [and] this!
    ..
       this:: too!
    ..
       |even| this:: !

A explicit markup start followed by a blank line and nothing else
(apart from whitespace) is an "empty comment".  It serves to terminate
a preceding construct, and does **not** consume any indented text
following.  To have a block quote follow a list or any indented
construct, insert an unindented empty comment in-between.

Syntax diagram::

    +-------+----------------------+
    | ".. " | comment              |
    +-------+ block                |
            |                      |
            +----------------------+

Implicit Hyperlink Targets
==========================

Implicit hyperlink targets are generated by section titles, footnotes,
and citations, and may also be generated by extension constructs.
Implicit hyperlink targets otherwise behave identically to explicit
`hyperlink targets`_.

Problems of ambiguity due to conflicting duplicate implicit and
explicit reference names are avoided by following this procedure:

1. `Explicit hyperlink targets`_ override any implicit targets having
   the same reference name.  The implicit hyperlink targets are
   removed, and level-1 (info) system messages are inserted.

2. Duplicate implicit hyperlink targets are removed, and level-1
   (info) system messages inserted.  For example, if two or more
   sections have the same title (such as "Introduction" subsections of
   a rigidly-structured document), there will be duplicate implicit
   hyperlink targets.

3. Duplicate explicit hyperlink targets are removed, and level-2
   (warning) system messages are inserted.  Exception: duplicate
   `external hyperlink targets`_ (identical hyperlink names and
   referenced URIs) do not conflict, and are not removed.

System messages are inserted where target links have been removed.
See "Error Handling" in `PEP 258`_.

The parser must return a set of *unique* hyperlink targets.  The
calling software (such as the Docutils_) can warn of unresolvable
links, giving reasons for the messages.

Inline Markup
=============

In reStructuredText, inline markup applies to words or phrases within
a text block.  The same whitespace and punctuation that serves to
delimit words in written text is used to delimit the inline markup
syntax constructs.  The text within inline markup may not begin or end
with whitespace.  Arbitrary `character-level inline markup`_ is
supported although not encouraged.  Inline markup cannot be nested.

There are nine inline markup constructs.  Five of the constructs use
identical start-strings and end-strings to indicate the markup:

- emphasis_: "*"
- `strong emphasis`_: "**"
- `interpreted text`_: "`"
- `inline literals`_: "``"
- `substitution references`_: "|"

Three constructs use different start-strings and end-strings:

- `inline internal targets`_: "_`" and "`"
- `footnote references`_: "[" and "]_"
- `hyperlink references`_: "`" and "\`_" (phrases), or just a
  trailing "_" (single words)

`Standalone hyperlinks`_ are recognized implicitly, and use no extra
markup.

The inline markup start-string and end-string recognition rules are as
follows.  If any of the conditions are not met, the start-string or
end-string will not be recognized or processed.

1. Inline markup start-strings must start a text block or be
   immediately preceded by whitespace or one of::

       ' " ( [ { < - / :

2. Inline markup start-strings must be immediately followed by
   non-whitespace.

3. Inline markup end-strings must be immediately preceded by
   non-whitespace.

4. Inline markup end-strings must end a text block or be immediately
   followed by whitespace or one of::

       ' " ) ] } > - / : . , ; ! ? \

5. If an inline markup start-string is immediately preceded by a
   single or double quote, "(", "[", "{", or "<", it must not be
   immediately followed by the corresponding single or double quote,
   ")", "]", "}", or ">".

6. An inline markup end-string must be separated by at least one
   character from the start-string.

7. An unescaped backslash preceding a start-string or end-string will
   disable markup recognition, except for the end-string of `inline
   literals`_.  See `Escaping Mechanism`_ above for details.

For example, none of the following are recognized as containing inline
markup start-strings:

- asterisks: * "*" '*' (*) (* [*] {*} 1*x BOM32_*
- double asterisks: **  a**b O(N**2) etc.
- backquotes: ` `` etc.
- underscores: _ __ __init__ __init__() etc.
- vertical bars: | || etc.

It may be desirable to use inline literals for some of these anyhow,
especially if they represent code snippets.  It's a judgment call.

These cases *do* require either literal-quoting or escaping to avoid
misinterpretation::

    *4, class_, *args, **kwargs, `TeX-quoted', *ML, *.txt

The inline markup recognition rules were devised intentionally to
allow 90% of non-markup uses of "*", "`", "_", and "|" *without*
resorting to backslashes.  For 9 of the remaining 10%, use inline
literals or literal blocks::

    "``\*``" -> "\*" (possibly in another font or quoted)

Only those who understand the escaping and inline markup rules should
attempt the remaining 1%.  ;-)

Inline markup delimiter characters are used for multiple constructs,
so to avoid ambiguity there must be a specific recognition order for
each character.  The inline markup recognition order is as follows:

- Asterisks: `Strong emphasis`_ ("**") is recognized before emphasis_
  ("*").

- Backquotes: `Inline literals`_ ("``"), `inline internal targets`_
  (leading "_`", trailing "`"), are mutually independent, and are
  recognized before phrase `hyperlink references`_ (leading "`",
  trailing "\`_") and `interpreted text`_ ("`").

- Trailing underscores: Footnote references ("[" + label + "]_") and
  simple `hyperlink references`_ (name + trailing "_") are mutually
  independent.

- Vertical bars: `Substitution references`_ ("|") are independently
  recognized.

- `Standalone hyperlinks`_ are the last to be recognized.

Character-Level Inline Markup
-----------------------------

It is possible to mark up individual characters within a word with
backslash escapes (see `Escaping Mechanism`_ above).  Backslash
escapes can be used to allow arbitrary text to immediately follow
inline markup::

    Python ``list``\s use square bracket syntax.

The backslash will disappear from the processed document.  The word
"list" will appear as inline literal text, and the letter "s" will
immediately follow it as normal text, with no space in-between.

Arbitrary text may immediately precede inline markup using
backslash-escaped whitespace::

    Possible in *re*\ ``Structured``\ *Text*, though not encouraged.

The backslashes and spaces separating "re", "Structured", and "Text"
above will disappear from the processed document.

.. CAUTION::

   The use of backslash-escapes for character-level inline markup is
   not encouraged.  Such use is ugly and detrimental to the
   unprocessed document's readability.  Please use this feature
   sparingly and only where absolutely necessary.

Emphasis
--------

Doctree element: emphasis.

Start-string = end-string = "*".

Text enclosed by single asterisk characters is emphasized::

    This is *emphasized text*.

Emphasized text is typically displayed in italics.

Strong Emphasis
---------------

Doctree element: strong.

Start-string = end-string = "**".

Text enclosed by double-asterisks is emphasized strongly::

    This is **strong text**.

Strongly emphasized text is typically displayed in boldface.

Interpreted Text
----------------

Doctree element: depends on the explicit or implicit role and
processing.

Start-string = end-string = "`".

Interpreted text is text that is meant to be related, indexed, linked,
summarized, or otherwise processed, but the text itself is typically
left alone.  Interpreted text is enclosed by single backquote
characters::

    This is `interpreted text`.

The "role" of the interpreted text determines how the text is
interpreted.  The role may be inferred implicitly (as above; the
"default role" is used) or indicated explicitly, using a role marker.
A role marker consists of a colon, the role name, and another colon.
A role name is a single word consisting of alphanumerics plus internal
hyphens, underscores, and periods; no whitespace or other characters
are allowed.  A role marker is either a prefix or a suffix to the
interpreted text, whichever reads better; it's up to the author::

    :role:`interpreted text`

    `interpreted text`:role:

Interpreted text allows extensions to the available inline descriptive
markup constructs.  To emphasis_, `strong emphasis`_, `inline
literals`_, and `hyperlink references`_, we can add "title reference",
"index entry", "acronym", "class", "red", "blinking" or anything else
we want.  Only pre-determined roles are recognized; unknown roles will
generate errors.  A core set of standard roles is implemented in the
reference parser; see `reStructuredText Interpreted Text Roles`_ for
individual descriptions.  In addition, applications may support
specialized roles.

Inline Literals
---------------

Doctree element: literal.

Start-string = end-string = "``".

Text enclosed by double-backquotes is treated as inline literals::

    This text is an example of ``inline literals``.

Inline literals may contain any characters except two adjacent
backquotes in an end-string context (according to the recognition
rules above).  No markup interpretation (including backslash-escape
interpretation) is done within inline literals.

Line breaks are *not* preserved in inline literals.  Although a
reStructuredText parser will preserve runs of spaces in its output,
the final representation of the processed document is dependent on the
output formatter, thus the preservation of whitespace cannot be
guaranteed.  If the preservation of line breaks and/or other
whitespace is important, `literal blocks`_ should be used.

Inline literals are useful for short code snippets.  For example::

    The regular expression ``[+-]?(\d+(\.\d*)?|\.\d+)`` matches
    floating-point numbers (without exponents).

Hyperlink References
--------------------

Doctree element: reference.

- Named hyperlink references:

  - Start-string = "" (empty string), end-string = "_".
  - Start-string = "`", end-string = "\`_".  (Phrase references.)

- Anonymous hyperlink references:

  - Start-string = "" (empty string), end-string = "__".
  - Start-string = "`", end-string = "\`__".  (Phrase references.)

Hyperlink references are indicated by a trailing underscore, "_",
except for `standalone hyperlinks`_ which are recognized
independently.  The underscore can be thought of as a right-pointing
arrow.  The trailing underscores point away from hyperlink references,
and the leading underscores point toward `hyperlink targets`_.

Hyperlinks consist of two parts.  In the text body, there is a source
link, a reference name with a trailing underscore (or two underscores
for `anonymous hyperlinks`_)::

    See the Python_ home page for info.

A target link with a matching reference name must exist somewhere else
in the document.  See `Hyperlink Targets`_ for a full description).

`Anonymous hyperlinks`_ (which see) do not use reference names to
match references to targets, but otherwise behave similarly to named
hyperlinks.

Embedded URIs
`````````````

A hyperlink reference may directly embed a target URI inline, within
angle brackets ("<...>") as follows::

    See the `Python home page <http://www.python.org>`_ for info.

This is exactly equivalent to::

    See the `Python home page`_ for info.

    .. _Python home page: http://www.python.org

The bracketed URI must be preceded by whitespace and be the last text
before the end string.  With a single trailing underscore, the
reference is named and the same target URI may be referred to again.

With two trailing underscores, the reference and target are both
anonymous, and the target cannot be referred to again.  These are
"one-off" hyperlinks.  For example::

    `RFC 2396 <http://www.rfc-editor.org/rfc/rfc2396.txt>`__ and `RFC
    2732 <http://www.rfc-editor.org/rfc/rfc2732.txt>`__ together
    define the syntax of URIs.

Equivalent to::

    `RFC 2396`__ and `RFC 2732`__ together define the syntax of URIs.

    __ http://www.rfc-editor.org/rfc/rfc2396.txt
    __ http://www.rfc-editor.org/rfc/rfc2732.txt

If reference text happens to end with angle-bracketed text that is
*not* a URI, the open-angle-bracket needs to be backslash-escaped.
For example, here is a reference to a title describing a tag::

    See `HTML Element: \<a>`_ below.

The reference text may also be omitted, in which case the URI will be
duplicated for use as the reference text.  This is useful for relative
URIs where the address or file name is also the desired reference
text::

    See `<a_named_relative_link>`_ or `<an_anonymous_relative_link>`__
    for details.

.. CAUTION::

   This construct offers easy authoring and maintenance of hyperlinks
   at the expense of general readability.  Inline URIs, especially
   long ones, inevitably interrupt the natural flow of text.  For
   documents meant to be read in source form, the use of independent
   block-level `hyperlink targets`_ is **strongly recommended**.  The
   embedded URI construct is most suited to documents intended *only*
   to be read in processed form.

Inline Internal Targets
------------------------

Doctree element: target.

Start-string = "_`", end-string = "`".

Inline internal targets are the equivalent of explicit `internal
hyperlink targets`_, but may appear within running text.  The syntax
begins with an underscore and a backquote, is followed by a hyperlink
name or phrase, and ends with a backquote.  Inline internal targets
may not be anonymous.

For example, the following paragraph contains a hyperlink target named
"Norwegian Blue"::

    Oh yes, the _`Norwegian Blue`.  What's, um, what's wrong with it?

See `Implicit Hyperlink Targets`_ for the resolution of duplicate
reference names.

Footnote References
-------------------

Doctree element: footnote_reference.

Start-string = "[", end-string = "]_".

Each footnote reference consists of a square-bracketed label followed
by a trailing underscore.  Footnote labels are one of:

- one or more digits (i.e., a number),

- a single "#" (denoting `auto-numbered footnotes`_),

- a "#" followed by a simple reference name (an `autonumber label`_),
  or

- a single "*" (denoting `auto-symbol footnotes`_).

For example::

    Please RTFM [1]_.

    .. [1] Read The Fine Manual

Citation References
-------------------

Doctree element: citation_reference.

Start-string = "[", end-string = "]_".

Each citation reference consists of a square-bracketed label followed
by a trailing underscore.  Citation labels are simple `reference
names`_ (case-insensitive single words, consisting of alphanumerics
plus internal hyphens, underscores, and periods; no whitespace).

For example::

    Here is a citation reference: [CIT2002]_.

See Citations_ for the citation itself.

Substitution References
-----------------------

Doctree element: substitution_reference, reference.

Start-string = "|", end-string = "|" (optionally followed by "_" or
"__").

Vertical bars are used to bracket the substitution reference text.  A
substitution reference may also be a hyperlink reference by appending
a "_" (named) or "__" (anonymous) suffix; the substitution text is
used for the reference text in the named case.

The processing system replaces substitution references with the
processed contents of the corresponding `substitution definitions`_
(which see for the definition of "correspond").  Substitution
definitions produce inline-compatible elements.

Examples::

    This is a simple |substitution reference|.  It will be replaced by
    the processing system.

    This is a combination |substitution and hyperlink reference|_.  In
    addition to being replaced, the replacement text or element will
    refer to the "substitution and hyperlink reference" target.

Standalone Hyperlinks
---------------------

Doctree element: reference.

Start-string = end-string = "" (empty string).

A URI (absolute URI [#URI]_ or standalone email address) within a text
block is treated as a general external hyperlink with the URI itself
as the link's text.  For example::

    See http://www.python.org for info.

would be marked up in HTML as::

    See <a href="http://www.python.org">http://www.python.org</a> for
    info.

Two forms of URI are recognized:

1. Absolute URIs.  These consist of a scheme, a colon (":"), and a
   scheme-specific part whose interpretation depends on the scheme.

   The scheme is the name of the protocol, such as "http", "ftp",
   "mailto", or "telnet".  The scheme consists of an initial letter,
   followed by letters, numbers, and/or "+", "-", ".".  Recognition is
   limited to known schemes, per the `Official IANA Registry of URI
   Schemes`_ and the W3C's `Retired Index of WWW Addressing Schemes`_.

   The scheme-specific part of the resource identifier may be either
   hierarchical or opaque:

   - Hierarchical identifiers begin with one or two slashes and may
     use slashes to separate hierarchical components of the path.
     Examples are web pages and FTP sites::

         http://www.python.org

         ftp://ftp.python.org/pub/python

   - Opaque identifiers do not begin with slashes.  Examples are
     email addresses and newsgroups::

         mailto:someone@somewhere.com

         news:comp.lang.python

   With queries, fragments, and %-escape sequences, URIs can become
   quite complicated.  A reStructuredText parser must be able to
   recognize any absolute URI, as defined in RFC2396_ and RFC2732_.

2. Standalone email addresses, which are treated as if they were
   absolute URIs with a "mailto:" scheme.  Example::

       someone@somewhere.com

Punctuation at the end of a URI is not considered part of the URI,
unless the URI is terminated by a closing angle bracket (">").
Backslashes may be used in URIs to escape markup characters,
specifically asterisks ("*") and underscores ("_") which are vaid URI
characters (see `Escaping Mechanism`_ above).

.. [#URI] Uniform Resource Identifier.  URIs are a general form of
   URLs (Uniform Resource Locators).  For the syntax of URIs see
   RFC2396_ and RFC2732_.

----------------
 Error Handling
----------------

Doctree element: system_message, problematic.

Markup errors are handled according to the specification in `PEP
258`_.

.. _reStructuredText: http://docutils.sourceforge.net/rst.html
.. _Docutils: http://docutils.sourceforge.net/
.. _The Docutils Document Tree: ../doctree.html
.. _Docutils Generic DTD: ../docutils.dtd
.. _transforms:
   http://docutils.sourceforge.net/docutils/transforms/
.. _Grouch: http://www.mems-exchange.org/software/grouch/
.. _RFC822: http://www.rfc-editor.org/rfc/rfc822.txt
.. _DocTitle transform:
.. _DocInfo transform:
   http://docutils.sourceforge.net/docutils/transforms/frontmatter.py
.. _getopt.py:
   http://www.python.org/doc/current/lib/module-getopt.html
.. _GNU libc getopt_long():
   http://www.gnu.org/software/libc/manual/html_node/Getopt-Long-Options.html
.. _doctest module:
   http://www.python.org/doc/current/lib/module-doctest.html
.. _Emacs table mode: http://table.sourceforge.net/
.. _Official IANA Registry of URI Schemes:
   http://www.iana.org/assignments/uri-schemes
.. _Retired Index of WWW Addressing Schemes:
   http://www.w3.org/Addressing/schemes.html
.. _World Wide Web Consortium: http://www.w3.org/
.. _HTML Techniques for Web Content Accessibility Guidelines:
   http://www.w3.org/TR/WCAG10-HTML-TECHS/#link-text
.. _reStructuredText Directives: directives.html
.. _reStructuredText Interpreted Text Roles: roles.html
.. _RFC2396: http://www.rfc-editor.org/rfc/rfc2396.txt
.. _RFC2732: http://www.rfc-editor.org/rfc/rfc2732.txt
.. _Zope: http://www.zope.com/
.. _PEP 258: ../../peps/pep-0258.html

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


=========================================
 reStructuredText Interpreted Text Roles
=========================================
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 1.14 $
:Date: $Date: 2004/06/03 17:49:02 $
:Copyright: This document has been placed in the public domain.

This document describes the interpreted text roles implemented in the
reference reStructuredText parser.

Interpreted text uses backquotes (`) around the text.  An explicit
role marker may optionally appear before or after the text, delimited
with colons.  For example::

    This is `interpreted text` using the default role.

    This is :title:`interpreted text` using an explicit role.

A default role may be defined by applications of reStructuredText; it
is used if no explicit ``:role:`` prefix or suffix is given.  The
"default default role" is `:title-reference:`_.

See the `Interpreted Text`_ section in the `reStructuredText Markup
Specification`_ for syntax details.  For details on the hierarchy of
elements, please see `The Docutils Document Tree`_ and the `Docutils
Generic DTD`_ XML document type definition.  For interpreted text role
implementation details, see `Creating reStructuredText Interpreted
Text Roles`_.

.. _"role" directive: directives.html#role
.. _Interpreted Text: restructuredtext.html#interpreted-text
.. _reStructuredText Markup Specification: restructuredtext.html
.. _The Docutils Document Tree: ../doctree.html
.. _Docutils Generic DTD: ../docutils.dtd
.. _Creating reStructuredText Interpreted Text Roles:
   ../../howto/rst-roles.html

.. contents::

---------------
 Customization
---------------

Custom interpreted text roles may be defined in a document with the
`"role" directive`_.  Customization details are listed with each role.

.. _class:

The ``class`` option is recognized by the "role" directive for most
interpreted text roles.  A description__ is provided by `"role"
directive`_ documentation.

__ directives.html#role-class

---------------------------------
 Standard Interpreted Text Roles
---------------------------------

``:emphasis:``
==============

:Aliases: None
:DTD Element: emphasis
:Customization:
    :Options: class_.
    :Content: None.

Implements emphasis.  These are equivalent::

    *text*
    :emphasis:`text`

``:literal:``
==============

:Aliases: None
:DTD Element: literal
:Customization:
    :Options: class_.
    :Content: None.

Implements inline literal text.  These are equivalent::

    ``text``
    :literal:`text`

Care must be taken with backslash-escapes though.  These are *not*
equivalent::

    ``text \ and \ backslashes``
    :literal:`text \ and \ backslashes`

The backslashes in the first line are preserved (and do nothing),
whereas the backslashes in the second line escape the following
spaces.

``:pep-reference:``
===================

:Aliases: ``:PEP:``
:DTD Element: reference
:Customization:
    :Options: class_.
    :Content: None.

The ``:pep-reference:`` role is used to create an HTTP reference to a
PEP (Python Enhancement Proposal).  The ``:PEP:`` alias is usually
used.  For example::

    See :PEP:`287` for more information about reStructuredText.

This is equivalent to::

    See `PEP 287`__ for more information about reStructuredText.

    __ http://www.python.org/peps/pep-0287.html

``:rfc-reference:``
===================

:Aliases: ``:RFC:``
:DTD Element: reference
:Customization:
    :Options: class_.
    :Content: None.

The ``:rfc-reference:`` role is used to create an HTTP reference to an
RFC (Internet Request for Comments).  The ``:RFC:`` alias is usually
used.  For example::

    See :RFC:`2822` for information about email headers.

This is equivalent to::

    See `RFC 2822`__ for information about email headers.

    __ http://www.faqs.org/rfcs/rfc2822.html

``:strong:``
============

:Aliases: None
:DTD Element: strong
:Customization:
    :Options: class_.
    :Content: None.

Implements strong emphasis.  These are equivalent::

    **text**
    :strong:`text`

``:subscript:``
===============

:Aliases: ``:sub:``
:DTD Element: subscript
:Customization:
    :Options: class_.
    :Content: None.

Implements subscripts.

.. Tip::

   Whitespace or punctuation is required around interpreted text, but
   often not desired with subscripts & superscripts.
   Backslash-escaped whitespace can be used; the whitespace will be
   removed from the processed document::

       H\ :sub:`2`\ O
       E = mc\ :sup:`2`

   In such cases, readability of the plain text can be greatly
   improved with substitutions::

       The chemical formula for pure water is |H2O|.

       .. |H2O| replace:: H\ :sub:`2`\ O

   See `the reStructuredText spec`__ for further information on
   `character-level markup`__ and `the substitution mechanism`__.

   __ restructuredtext.html
   __ restructuredtext.html#character-level-inline-markup
   __ restructuredtext.html#substitution-references

``:superscript:``
=================

:Aliases: ``:sup:``
:DTD Element: superscript
:Customization:
    :Options: class_.
    :Content: None.

Implements superscripts.  See the tip in `:subscript:`_ above.

``:title-reference:``
=====================

:Aliases: ``:title:``, ``:t:``.
:DTD Element: title_reference
:Customization:
    :Options: class_.
    :Content: None.

The ``:title-reference:`` role is used to describe the titles of
books, periodicals, and other materials.  It is the equivalent of the
HTML "cite" element, and it is expected that HTML writers will
typically render "title_reference" elements using "cite".

Since title references are typically rendered with italics, they are
often marked up using ``*emphasis*``, which is misleading and vague.
The "title_reference" element provides accurate and unambiguous
descriptive markup.

Let's assume ``:title-reference:`` is the default interpreted text
role (see below) for this example::

    `Design Patterns` [GoF95]_ is an excellent read.

The following document fragment (pseudo-XML_) will result from
processing::

    <paragraph>
        <title_reference>
            Design Patterns

        <citation_reference refname="gof95">
            GoF95
         is an excellent read.

``:title-reference:`` is the default interpreted text role in the
standard reStructuredText parser.  This means that no explicit role is
required.  Applications of reStructuredText may designate a different
default role, in which case the explicit ``:title-reference:`` role
must be used to obtain a ``title_reference`` element.

.. _pseudo-XML: ../doctree.html#pseudo-xml


==============================
 Docutils Configuration Files
==============================

:Author: David Goodger
:Contact: goodger@python.org
:Revision: $Revision: 1.19 $
:Date: $Date: 2004/06/16 17:21:56 $
:Copyright: This document has been placed in the public domain.

.. contents::

.. Cross-reference command-line options with configuration file
   settings?  Make alphabetical indexes of both.

Configuration files are used for persistent customization; they can be
set once and take effect every time you use a front-end tool.
Configuration file settings override the built-in defaults, and
command-line options override all.

By default, Docutils checks the following places for configuration
files, in the following order:

1. ``/etc/docutils.conf``: This is a system-wide configuration file,
   applicable to all Docutils processing on the system.

2. ``./docutils.conf``: This is a project-specific configuration file,
   located in the current directory.  The Docutils front end has to be
   executed from the directory containing this configuration file for
   it to take effect (note that this may have nothing to do with the
   location of the source files).  Settings in the project-specific
   configuration file will override corresponding settings in the
   system-wide file.

3. ``~/.docutils``: This is a user-specific configuration file,
   located in the user's home directory.  Settings in this file will
   override corresponding settings in both the system-wide and
   project-specific configuration files.

If more than one configuration file is found, all will be read but
later entries will override earlier ones.  For example, a "stylesheet"
entry in a user-specific configuration file will override a
"stylesheet" entry in the system-wide file.

The default implicit config file paths can be overridden by the
``DOCUTILSCONFIG`` environment variable.  ``DOCUTILSCONFIG`` should
contain a colon-separated (semicolon-separated on Windows) sequence of
config file paths to search for; leave it empty to disable implicit
config files altogether.  Tilde-expansion is performed on paths.
Paths are interpreted relative to the current working directory.
Empty path items are ignored.

In addition, a configuration file may be explicitly specified with the
"--config" command-line option.  This configuration file is read after
the three implicit ones listed above (or the ones defined by the
``DOCUTILSCONFIG`` environment variable), and its entries will have
priority.

-------------------------
Configuration File Syntax
-------------------------

Configuration files use the standard ConfigParser.py_ Python_ module.
From its documentation:

    The configuration file consists of sections, lead by a "[section]"
    header and followed by "name: value" entries, with continuations
    in the style of `RFC 822`_; "name=value" is also accepted.  Note
    that leading whitespace is removed from values.  ...  Lines
    beginning with "#" or ";" are ignored and may be used to provide
    comments.

.. Note:: No format string interpolation is done.

Configuration file entry names correspond to internal runtime
settings.  Underscores ("_") and hyphens ("-") can be used
interchangably in entry names; hyphens are automatically converted to
underscores.

For on/off switch settings (booleans), the following values are
recognized:

* On: "true", "yes", "on", "1"
* Off: "false", "no", "off", "0", "" (no value)

-------------------------------------
Configuration File Sections & Entries
-------------------------------------

Below are the Docutils runtime settings, listed by config file
section.  Any setting may be specified in any section, but only
settings from active sections will be used.  Sections correspond to
Docutils components (module name or alias; section names are always in
lowercase letters).  Each `Docutils application`_ uses a specific set
of components; corresponding configuration file sections are applied
when the application is used.  Configuration sections are applied in
general-to-specific order, as follows:

1. `[general]`_

2. `[parsers]`_, parser dependencies, and the section specific to the
   Parser used ("[... parser]").  Currently, only `[restructuredtext
   parser]`_ is applicable.

3. `[readers]`_, reader dependencies, and the section specific to the
   Reader used ("[... reader]").  For example, `[pep reader]`_ depends
   on `[standalone reader]`_.

4. `[writers]`_, writer dependencies, and the section specific to the
   Writer used ("[... writer]").  For example, `[pep_html writer]`_
   depends on `[html4css1 writer]`_.

5. `[applications]`_, application dependencies, and the section
    specific to the Application (front-end tool) in use
    ("[... application]").

Since any setting may be specified in any section, this ordering
allows component- or application-specific overrides of earlier
settings.  For example, there may be Reader-specific overrides of
general settings; Writer-specific overrides of Parser settings;
Application-specific overrides of Writer settings; and so on.

If multiple configuration files are applicable, the process is
completed (all sections are applied in the order given) for each one
before going on to the next.  For example, a "[pep_html writer]
stylesheet" setting in an earlier configuration file would be
overridden by an "[html4css1 writer] stylesheet" setting in a later
file.

Some knowledge of Python_ is assumed for some attributes.

.. _ConfigParser.py:
   http://www.python.org/doc/current/lib/module-ConfigParser.html
.. _Python: http://www.python.org/
.. _RFC 822: http://www.rfc-editor.org/rfc/rfc822.txt
.. _Docutils application: tools.html

[general]
=========

Settings in the "[general]" section are always applied.

_`datestamp`
    Include a time/datestamp in the document footer.  Contains a
    format string for Python's ``time.strftime``.  See the `time
    module documentation`__.

    Default: None.  Options: ``--date, -d, --time, -t,
    --no-datestamp``.

    Configuration file entry examples::

        # Equivalent to --date command-line option, results in
        # ISO 8601 extended format datestamp, e.g. "2001-12-21":
        datestamp: %Y-%m-%d

        # Equivalent to --time command-line option, results in
        # date/timestamp like "2001-12-21 18:43 UTC":
        datestamp: %Y-%m-%d %H:%M UTC

        # Disables datestamp; equivalent to --no-datestamp:
        datestamp:

    __ http://www.python.org/doc/current/lib/module-time.html

_`debug`
    Report debug-level system messages.

    Default: don't (None).  Options: ``--debug, --no-debug``.

_`dump_internals`
    At the end of processing, write all internal attributes of the
    document (``document.__dict__``) to stderr.

    Default: don't (None).  Options: ``--dump-internals`` (hidden, for
    development use only).

_`dump_pseudo_xml`
    At the end of processing, write the pseudo-XML representation of
    the document to stderr.

    Default: don't (None).  Options: ``--dump-pseudo-xml`` (hidden,
    for development use only).

_`dump_settings`
    At the end of processing, write all Docutils settings to stderr.

    Default: don't (None).  Options: ``--dump-settings`` (hidden, for
    development use only).

_`dump_transforms`
    At the end of processing, write a list of all transforms applied
    to the document to stderr.

    Default: don't (None).  Options: ``--dump-transforms`` (hidden,
    for development use only).

_`error_encoding`
    The text encoding for error output.

    Default: "ascii".  Options: ``--error-encoding, -e``.

_`error_encoding_error_handler`
    The error handler for unencodable characters in error output.  See
    output_encoding_error_handler_ for acceptable values.

    Default: "backslashreplace" for Python 2.3 and later; "replace"
    otherwise.  Options: ``--error-encoding-error-handler,
    --error-encoding, -e``.

_`exit_status_level`
    A system message level threshold; non-halting system messages at
    or above this level will produce a non-zero exit status at normal
    exit.  Exit status is the maximum system message level plus 10 (11
    for INFO, etc.).

    Default: disabled (5).  Options: ``--exit-status``.

_`expose_internals`
    List of internal attribues to expose as external attributes (with
    "internal:" namespace prefix).  To specify multiple attributes in
    configuration files, use colons to separate names; on the command
    line, the option may be used more than once.

    Default: don't (None).  Options: ``--expose-internal-attribute``
    (hidden, for development use only).

_`footnote_backlinks`
    Enable or disable backlinks from footnotes and citations to their
    references.

    Default: enabled (1).  Options: ``--footnote-backlinks,
    --no-footnote-backlinks``.

_`generator`
    Include a "Generated by Docutils" credit and link in the document
    footer.

    Default: off (None).  Options: ``--generator, -g,
    --no-generator``.

_`halt_level`
    The threshold at or above which system messages are converted to
    exceptions, halting execution immediately.  If `traceback`_ is
    set, the exception will propagate; otherwise, Docutils will exit.

    Default: severe (4).  Options: ``--halt, --strict``.

_`input_encoding`
    The text encoding for input.

    Default: auto-detect (None).  Options: ``--input-encoding, -i``.

_`language_code`
    `ISO 639`_ 2-letter language code (3-letter codes used only if no
    2-letter code exists).

    Default: English ("en").  Options: ``--language, -l``.

_`output_encoding`
    The text encoding for output.

    Default: "UTF-8".  Options: ``--output-encoding, -o``.

_`output_encoding_error_handler`
    The error handler for unencodable characters in the output.
    Acceptable values include:

    strict
        Raise an exception in case of an encoding error.
    replace
        Replace malformed data with a suitable replacement marker,
        such as "?".
    ignore
        Ignore malformed data and continue without further notice.
    xmlcharrefreplace
        (Python 2.3+)  Replace with the appropriate XML character
        reference, such as "``&#8224;``".
    backslashreplace
        (Python 2.3+)  Replace with backslashed escape sequences, such
        as "``\u2020``".

    Acceptable values are the same as for the "error" parameter of
    Python's ``encode`` string method; other values may be defined in
    applications or in future versions of Python.

    Default: "strict".  Options: ``--output-encoding-error-handler,
    --output-encoding, -o``.

_`report_level`
    Verbosity threshold at or above which system messages are
    reported.

    Default: warning (2).  Options: ``--report, -r, --verbose, -v,
    --quiet, -q``.

_`sectnum_xform`
    Enable or disable the section numbering transform
    (docutils.transforms.parts.SectNum).

    Default: enabled (1).  Options: ``--no-section-numbering``.

_`source_link`
    Include a "View document source" link in the document footer.  URL
    will be relative to the destination.

    Default: don't (None).  Options: ``--source-link, -s,
    --no-source-link``.

_`source_url`
    An explicit URL for a "View document source" link, used verbatim.

    Default: compute if source_link (None).  Options: ``--source-url,
    --no-source-link``.

_`toc_backlinks`
    Enable backlinks from section titles to table of contents entries
    ("entry"), to the top of the TOC ("top"), or disable ("none").

    Default: "entry".  Options: ``--toc-entry-backlinks,
    --toc-top-backlinks, --no-toc-backlinks``.

_`traceback`
    Enable Python tracebacks when halt-level system messages and other
    exceptions occur.  Useful for debugging, and essential for issue
    reports.  Exceptions are allowed to propagate, instead of being
    caught and reported (in a user-friendly way) by Docutils.

    Default: disabled (None).  Options: ``--traceback,
    --no-traceback``.

_`warning_stream`
    Path to a file for the output of system messages (warnings)
    [#pwd]_.

    Default: stderr (None).  Options: ``--warnings``.

[parsers]
---------

Docutils currently supports only one parser, for reStructuredText.

[restructuredtext parser]
`````````````````````````

_`pep_references`
    Recognize and link to standalone PEP references (like "PEP 258").

    Default: disabled (None); enabled (1) in PEP Reader.  Options:
    ``--pep-references``.

_`pep_base_url`
    Base URL for PEP references.

    Default: "http://www.python.org/peps/".  Option:
    ``--pep-base-url``.

_`rfc_references`
    Recognize and link to standalone RFC references (like "RFC 822").

    Default: disabled (None); enabled (1) in PEP Reader.  Options:
    ``--rfc-references``.

_`rfc_base_url`
    Base URL for RFC references.

    Default: "http://www.faqs.org/rfcs/".  Option: ``--rfc-base-url``.

_`tab_width`
    Number of spaces for hard tab expansion.

    Default: 8.  Options: ``--tab-width``.

_`trim_footnote_reference_space`
    Remove spaces before footnote references.

    Default: don't (None).  Options:
    ``--trim-footnote-reference-space``.

[readers]
---------

[standalone reader]
```````````````````

_`docinfo_xform`
    Enable or disable the bibliographic field list transform
    (docutils.transforms.frontmatter.DocInfo).

    Default: enabled (1).  Options: ``--no-doc-info``.

_`doctitle_xform`
    Enable or disable the promotion of a lone top-level section title
    to document title (and subsequent section title to document
    subtitle promotion; docutils.transforms.frontmatter.DocTitle).

    Default: enabled (1).  Options: ``--no-doc-title``.

[pep reader]
````````````

The `pep_references`_ and `rfc_references`_ options
(`[restructuredtext parser]`_) are set on by default.

[python reader]
```````````````

Under construction.

[writers]
---------

[docutils_xml writer]
`````````````````````

_`doctype_declaration`
    Generate XML with a DOCTYPE declaration.

    Default: do (1).  Options: ``--no-doctype``.

_`indents`
    Generate XML with indents and newlines.

    Default: don't (None).  Options: ``--indents``.

_`newlines`
    Generate XML with newlines before and after tags.

    Default: don't (None).  Options: ``--newlines``.

.. _xml_declaration [docutils_xml writer]:

xml_declaration
    Generate XML with an XML declaration.  Also defined for the
    `HTML Writer`__.

    .. Caution:: The XML declaration carries text encoding
       information, without which standard tools may be unable to read
       the generated XML.

    Default: do (1).  Options: ``--no-xml-declaration``.

    __ `xml_declaration [html4css1 writer]`_

[html4css1 writer]
``````````````````

.. _attribution [html4css1 writer]:

attribution
    Format for block quote attributions: one of "dash" (em-dash
    prefix), "parentheses"/"parens", or "none".  Also defined for the
    `LaTeX Writer`__.

    Default: "dash".  Options: ``--attribution``.

    __ `attribution [latex2e writer]`_

_`compact_lists`
    Remove extra vertical whitespace between items of bullet lists and
    enumerated lists, when list items are "simple" (i.e., all items
    each contain one paragraph and/or one "simple" sublist only).

    Default: enabled (1).  Options: ``--compact-lists,
    --no-compact-lists``.

_`embed_stylesheet`
    Embed the stylesheet in the output HTML file.  The stylesheet file
    must be accessible during processing.  The stylesheet is embedded
    inside a comment, so it must not contain the text "``--``" (two
    hyphens).

    Default: link, don't embed (None).  Options: ``--embed-stylesheet,
    --link-stylesheet``.

.. _footnote_references [html4css1 writer]:

footnote_references
    Format for footnote references, one of "superscript" or
    "brackets".  Also defined for the `LaTeX Writer`__.

    Default: "superscript"; "brackets" in PEP/HTML Writer.  Options:
    ``--footnote-references``.

    __ `footnote_references [latex2e writer]`_

_`initial_header_level`
    The initial level for header elements.  This does not affect the
    document title & subtitle; see doctitle_xform_.

    Default: 1 (for "<h1>").  Option: ``--initial-header-level``.

.. _stylesheet [html4css1 writer]:

stylesheet
    CSS stylesheet URL, used verbatim.  Overridden by
    "stylesheet_path" URL option (``--stylesheet-path``).  Also
    defined for the `LaTeX Writer`__.

    Default: "default.css".  Options: ``--stylesheet``.

    __ `stylesheet [latex2e writer]`_

.. _stylesheet_path [html4css1 writer]:

stylesheet_path
    Path to CSS stylesheet [#pwd]_.  Overrides "stylesheet" URL option
    (``--stylesheet``).  Path is adjusted relative to the output HTML
    file.  Also defined for the `LaTeX Writer`__.

    Default: None.  Options: ``--stylesheet-path``.

    __ `stylesheet_path [latex2e writer]`_

.. _xml_declaration [html4css1 writer]:

xml_declaration
    Generate XML with an XML declaration.  Also defined for the
    `Docutils XML Writer`__.

    .. Caution:: The XML declaration carries text encoding
       information, without which standard tools may be unable to read
       the generated XML.

    Default: do (1).  Options: ``--no-xml-declaration``.

    __ `xml_declaration [docutils_xml writer]`_

[pep_html writer]
.................

The PEP/HTML Writer derives from the standard HTML Writer, and shares
all settings defined in the `[html4css1 writer]`_ section.  The
"[html4css1 writer]" section is processed before "[pep_html writer]".

_`no_random`
    Workaround for platforms which core-dump on "``import random``".

    Default: random enabled (None).  Options: ``--no-random``
    (hidden).

_`pep_home`
    Home URL prefix for PEPs.

    Default: current directory (".").  Options: ``--pep-home``.

_`template`
    Path to PEP template file [#pwd]_.

    Default: "pep-html-template" (in current directory).  Options:
    ``--template``.

_`python_home`
    Python's home URL.

    Default: parent directory ("..").  Options: ``--python-home``.

[latex2e writer]
````````````````

_`use_latex_toc`
    To get pagenumbers in the table of contents the table of contents
    must be generated by latex. Usually latex must be run twice to get
    numbers correct.

    *Note:* LaTeX will number the sections, which might be a bug in
    this case.

    Default: off.  Option: ``--use-latex-toc``.

.. XXX Missing: use_latex_docinfo

_`use_latex_footnotes`
    Use LaTeX-footnotes not a figure simulation. This might give no
    Hyperrefs on /to footnotes, but should be able to handle an
    unlimited number of footnotes.

    Default: off.  Option: ``--use-latex-footnotes``.

_`hyperlink_color`
    Color of any hyperlinks embedded in text. Use "0" to disable
    coloring of links.

    Default: "blue", use "0" to disable.  Option:
    ``--hyperlink-color``.

_`documentclass`
    Specify latex documentclass, *but* beaware that books have chapters
    articles not.

    Default: "article".  Option: ``--documentclass``.

_`documentoptions`
    Specify document options.  Multiple options can be given, separated by
    commas.

    Default is "10pt".  Option: ``--documentoptions``.

.. _stylesheet [latex2e writer]:

stylesheet
    Specify a stylesheet file. The file will be ``input`` by latex in
    the document header. If this is set to "" disables generation of
    input latex command.  Also defined for the `HTML Writer`__.

    Default: no stylesheet ("").  Option: ``--stylesheet``.

    __ `stylesheet [html4css1 writer]`_

.. _stylesheet_path [latex2e writer]:

stylesheet_path
    Path to stylesheet [#pwd]_.  Overrides "stylesheet" setting
    (``--stylesheet``).  XXX LaTeX semantics?  Also defined for the
    `HTML Writer`__.

    Default: None.  Option: ``--stylesheet-path``.

    __ `stylesheet_path [html4css1 writer]`_

.. XXX Missing: embed_stylesheet

.. _footnote_references [latex2e writer]:

footnote_references
    Format for footnote references: one of "superscript" or
    "brackets".  Also defined for the `HTML Writer`__.

    Default is "brackets".  Option: ``--footnote-references``.

    __ `footnote_references [html4css1 writer]`_

.. _attribution [latex2e writer]:

attribution
    Format for block quote attributions, the same as for the
    html-writer: one of "dash" (em-dash prefix),
    "parentheses"/"parens" or "none".  Also defined for the `HTML
    Writer`__.

    Default: "dash".  Option: ``--attribution``.

    __ `attribution [html4css1 writer]`_

_`compound_enumerators`
    Enable or disable compound enumerators for nested enumerated lists
    (e.g. "1.2.a.ii").

    Default: disabled (None).  Options: ``--compound-enumerators``,
    ``--no-compound-enumerators``.

_`section_prefix_for_enumerators`
    Enable or disable section ("." subsection ...) prefixes for
    compound enumerators.  This has no effect unless
    `compound_enumerators`_ are enabled.

    Default: disabled (None).  Options:
    ``--section-prefix-for-enumerators``,
    ``--no-section-prefix-for-enumerators``.

_`section_enumerator_separator`
    The separator between section number prefix and enumerator for
    compound enumerated lists (see `compound_enumerators`_).

    Generally it isn't recommended to use both sub-sections and nested
    enumerated lists with compound enumerators.  This setting avoids
    ambiguity in the situation where a section "1" has a list item
    enumerated "1.1", and subsection "1.1" has list item "1".  With a
    separator of ".", these both would translate into a final compound
    enumerator of "1.1.1".  With a separator of "-", we get the
    unambiguous "1-1.1" and "1.1-1".

    Default: "-".  Option: ``--section-enumerator-separator``.

table_style
    Specify the drawing of separation lines.

    - "standard" lines around and between cells.
    - "booktabs" a line above and below the table and one after the
      head.
    - "nolines".

[pseudoxml writer]
``````````````````

No settings are defined for this Writer.

[applications]
--------------

[buildhtml application]
```````````````````````

_`prune`
    List of directories not to process.  To specify multiple
    directories in configuration files, use colon-separated paths; on
    the command line, the option may be used more than once.

    Default: none ([]).  Options: ``--prune``.

_`recurse`
    Recursively scan subdirectories, or ignore subdirectories.

    Default: recurse (1).  Options: ``--recurse, --local``.

_`silent`
    Work silently (no progress messages).  Independent of
    "report_level".

    Default: show progress (None).  Options: ``--silent``.

[docfactory application]
````````````````````````

(To be completed.)

Other Settings
==============

These settings are only effective as command-line options, positional
arguments, or for internal use; setting them in configuration files
has no effect.

_`config`
    Path to a configuration file to read (if it exists) [#pwd]_.
    Settings may override defaults and earlier settings.  The config
    file is processed immediately.  Multiple ``--config`` options may
    be specified; each will be processed in turn.

    Filesystem path settings contained within the config file will be
    interpreted relative to the config file's location (*not* relative
    to the current working directory).

    Default: None.  Options: ``--config``.

_`_directories`
    (``buildhtml.py`` front end.)  List of paths to source
    directories, set from positional arguments.

    Default: current working directory (None).  No command-line
    options.

_`_disable_config`
    Prevent standard configuration files from being read.  For
    internal use only.

    Default: config files enabled (None).  No command-line options.

_`_destination`
    Path to output destination, set from positional arguments.

    Default: stdout (None).  No command-line options.

_`_source`
    Path to input source, set from positional arguments.

    Default: stdin (None).  No command-line options.

.. _ISO 639: http://lcweb.loc.gov/standards/iso639-2/englangn.html

.. [#pwd] Path relative to the working directory of the process at
   launch.

------------------------------
Old-Format Configuration Files
------------------------------

Formerly, Docutils configuration files contained a single "[options]"
section only.  This was found to be inflexible, and in August 2003
Docutils adopted the current component-based configuration file
sections as described above.  Docutils will still recognize the old
"[options]" section, but complains with a deprecation warning.

To convert existing config files, the easiest way is to change the
section title: change "[options]" to "[general]".  Most settings
haven't changed.  The only ones to watch out for are these:

=====================  =====================================
Old-Format Setting     New Section & Setting
=====================  =====================================
pep_stylesheet         [pep_html writer] stylesheet
pep_stylesheet_path    [pep_html writer] stylesheet_path
pep_template           [pep_html writer] template
=====================  =====================================


================================
 Generating LaTeX with Docutils
================================

:Author: Engelbert Gruber
:Contact: grubert@users.sourceforge.net
:Revision: $Revision: 1.49 $
:Date: $Date: 2004/06/25 19:09:25 $
:Copyright: This document has been placed in the public domain.

.. contents::

Introduction
============

Producing LaTeX code from reST input could be done in at least two ways:

a. Transform the internal markup into corresponding LaTeX markup e.g.
   a section title would be written as ```\section{this section ...}``.
b. Using LaTeX as a typesetting system to produce desired paperwork
   without caring about loosing document structure information.

The former might be preferable, but limits to LaTeXs capabilities, so
in reality it is a mix. The reality is that LaTeX has a titlepage with
title, author and date, by default only title is used. Author and date
are shown in the docutils docinfo table and set to blank for LaTeX.
To get author and date set by LaTeX specify option "use-LaTeX-docinfo".

Options
=======

Configuration can be done in two ways (again):

1. Options to the docutils tool: e.g. language selection.
2. Options to LaTeX via the stylesheet file.

The generated LaTeX documents should be kept processable by a standard
LaTeX installation (if such a thing exists), therefore the document
contains default settings. To allow *overwriting defaults* the stylesheet
is included at last.

Run ``rst2latex.py --help`` to see the command-line options, or have look in
config documentytion.

=====================  ================================================
Configuration Issue    Description
=====================  ================================================
papersize              Default: a4paper. Paper geometry can be changed
                       using ``\geometry{xxx}`` entries.

                       Some possibilities:

                       * a4paper, b3paper, letterpaper, executivepaper,
                         legalpaper
                       * landscape, portrait, twoside.

                       and a ton of other option setting margins.

                       An example::

                         \geometry{a5paper,landscape}
---------------------  ------------------------------------------------
paragraph indent       By default LaTeX indents the forst line in a
                       paragraph. The following lines set indentation
                       to zero but add a vertical space between
                       paragraphs.::

                         \setlength{\parindent}{0pt}
                         \setlength{\parskip}{6pt plus 2pt minus 1pt}
---------------------  ------------------------------------------------
admonitionwidth        The width for admonitions.
                       Default: 0.9*textwidth, this can be changed
                       e.g.::

                         \setlength{\admonitionwidth}{0.7\textwidth}
---------------------  ------------------------------------------------
docinfowidth           The width for the docinfo table.
                       Default: 0.9*textwidth, changed to e.g.::

                         \setlength{\docinfowidth}{0.7\textwidth}
---------------------  ------------------------------------------------
rubric style           The header contains the definition of a new
                       LaTeX command rubric. Inserting::

                         \renewcommand{\rubric}[1]{\subsection*{
                           ~\hfill {\color{red} #1} \hfill ~}}

                       sets rubric to subsection style in red.

                       Default: subsection style italic.
---------------------  ------------------------------------------------
line spacing           Is done with package *setspace*, which gives
                       singlespace, onehalfspace and doublespace
                       commands. To get documentwide double spacing,
                       add this to your stylesheet ::

                         \usepackage{setspace}
                         \doublespacing

                       Another way ::

                         \linespread{1.55}

                       And yet another, add ``doublesp`` to the
                       documentoptions and e.g. ::

                         \setstretch{1.7}

                       for bigger linespacing.
---------------------  ------------------------------------------------
font selection         see below
=====================  ================================================

Font selection
--------------

When generating pdf-files from LaTeX, use the pdflatex command, the files
are a lot smaller if postscript fonts are used. This *was* fixed by putting
``\usepackage{times}`` into the stylesheet.

It is said that the typewriter font in computer modern font, the default
LaTeX font package, is too heavy compared to the others. There is a package
or some commands too fix this, which i currently cannot find.

Some people diagnose a similar unbalance for the postscript fonts, the
package to fix this is ``\usepackage{pslatex}``.
pslatex in contrast to the standard LaTeX fonts has a bold typewriter font.

As ``times`` does not use the appropriate mathematical fonts and ``pslatex``
does not work with T1 encodings one should use::

  \usepackage{mathptmx}
  \usepackage[scaled=.90]{helvet}
  \usepackage{courier}

Unicode
-------

The generated LaTeX documents are in latin1 encoding per default, if unicode
characters are required one must set ``--output-encoding=utf-8`` install
`LaTeX unicode`_ support and add::

    \usepackage{ucs}
    \usepackage[utf8]{inputenc}

to the stylesheet.

.. _LaTeX unicode: http://www.unruh.de/DniQ/LaTeX/unicode/

Table of figures
----------------

A table of figures can be generated by a command directly to LaTeX::

  .. raw:: LaTeX

     \listoffigures

LaTeX also has a command ``\listoftables``.

Section numbering
-----------------

If section numbering and LaTeX table of contents is used LaTeX and
docutils will number sections. To switch off displaying of LaTeX's
numbers one has to add following lines to the stylesheet ::

  % no section number display
  \makeatletter
  \def\@seccntformat#1{}
  \makeatother
  % no numbers in toc
  \renewcommand{\numberline}[1]{}

images
------

Images are included in LaTeX by the graphicx package. The supported
image formats depend on the used driver (dvi, dvips, pdftex, ...).

pdf-image inclusion in pdf files fails, specify ``--graphicx-option=pdftex``
or ``--graphicx-option=auto``.

Commands directly to LaTeX
==========================

By means of the reST-raw directive one can give commands directly to
LaTeX, e.g. forcing a page break::

  .. raw:: LaTeX

     \newpage

Or setting formulas in LaTeX::

  .. raw:: LaTeX

     $$x^3 + 3x^2a + 3xa^2 + a^3,$$

Or making a colorbox: If someone wants to get a red background for a textblock,
she/he can put \definecolor{bg}{rgb}{.9,0,0} into style.tex and in
reStructuredText do something like this::

  |begincolorbox|
  Nobody expects the spanish inquisition.
  |endcolorbox|

  .. |begincolorbox| raw:: LaTeX

     \\begin{center}
     \\colorbox{bg}{
     \\parbox{0.985\\linewidth}{

  .. |endcolorbox| raw:: LaTeX

     }}
     \\end{center}

Custom title page
-----------------

Currently maketitle only shows the title and subtitle, date and author are shown
in the docinfo table.

To change the titlepage layout, e.g. see fancyhdr, one must redefine the
maketitle command in the stylesheet::

  \renewcommand{\maketitle}{
    \begin{titlepage}
      \begin{center}
      \textsf{TITLE \@title} \\
      Date: \today
      \end{center}
    \end{titlepage}
  }

``\@title`` contains the title.

Problems
========

Open to be fixed or open to discussion.

footnotes and citations
-----------------------

Initially both were implemented using figures, because hyperlinking back
and forth seemed to be impossible. Later images were put into figures.

This results in footnotes images and figures possibly being mixed at page
foot.

* Use LaTeX footnotes and citations for printing or more complex layout.
* Footnotes and citations done with figures might excell in hyperlink
  support.

If ``use-latex-citations`` is used a bibliography is inserted right at
the end of the document. *This should be customizable*.

Tilde in italian
----------------

Does not work, or only in verbatim or verb. Could be fixed by using T1, but
the hyphen in literal environments is different:

* Inline or in verbatim it is lower than it should.
  If package courier is used acroread 5 puts it too high, xpdf
  shows it at the same height as the horizontal line of ``+``.
* In a block with markup two hyphens become one (endashed ?).
  and ``<<``, ``>>`` become separate characters too.

Tables
------

:Tablewidth: reST-documents line length is assumed to be 80 characters. The
             tablewidth is set relative to this value. If someone produces
             documents with line length of 132 this will fail.

             Table width is tried to fit in page even if it is wider than
             the assumed linewidth, still assumed linewidth is a hook.

* In tools.txt the option tables right column, there should be some more spacing
  between the description and the next paragraph "Default:".

  Paragraph separation in tables is hairy.
  see http://www.tex.ac.uk/cgi-bin/texfaq2html?label=struttab

  - The strut solution did not work.
  - setting extrarowheight added ad top of row not between paragraphs in
    a cell. ALTHOUGH i set it to 2pt because, text is too close to the topline.
  - baselineskip/stretch does not help.
* Should there be two hlines after table head and on table end ?
* Table: multicol cells are always {l}.
* The contents of a rowspan cell do not influence table height.
  (Maybe if we put a tabular inside ?)
* Table heads and footer for longtable (firstpage lastpage ..).
* Table cells with multirow and multicolumn
* literal-blocks in table cells:

  - If verbatim or flushleft is used one gets vertical space above and below.
  - This is bad for the topmost paragraph in a cell, therefore the writer
    uses raggedright.
  - Ragged right fails on followup paragraphs as the vertical space would be
    missing.

Notes
~~~~~

* table-style booktabs: booktabs.sty 1.00 does not work with longtable.

Miscellaneous
-------------

* Selection of LaTeX fontsize configurable.
* Assumed reST linelength for table width setting configurable.
* literal-block indentation configurable.
* The underscore ``_`` does not work in literal-blocks, has different width.
  I tried

  - Underline a blank ``\underline{ }`` to low, and does not fit the font.
    But on many systems underscore is an underlined blank.
  - Escape it ``{\_}`` to narrow
  - Subscript a hyphen ``$_{\tt-}$`` too wide and thin
  - ``$_{-}$`` the same.

* recongize LaTeX and replace by ``\LaTeX``.
* Support embed-stylesheet.
* the ^-sign is problematic: using mathmode wedge is usually the wrong font.
* Sidebar handling.
* Maybe add end of line after term in definition list. see
    http://roundup.sf.net/doc-0.5/features.pdf
* Pdfbookmark level 4 (and greater) does not work (might be settable but OTOH).
* center subsection{Abstract} gives a LaTeX error here.
  ``! LaTeX Error: Something's wrong--perhaps a missing \item.``
  Committed a HACK: centering by hfill.
* Document errors are also too silent.
* Use optionlist for docinfo, the table does only work for single page.
* Consider peter funk's hooks for TeXpert:

  * Define his own document preamble (including the choice to
    choose his own documentclass.  That would make the ``--documentclass``
    option superfluous).  I suggest to call this option ``--preamble``
  * Use two additional hooks to put additional stuff just behind the
    ``\begin{document}`` and just before the ``\end{document}`` macros.
    Typical uses would be ``\tableofcontents``, ``\listoffigures`` and
    ``\appendix``, ``\makeindex``, ``\makeglossary`` and some such
    for larger documents.

* Hyphens: co-developers should be co--developers ?
* The indentional problematic error in test.txt is not referring anywhere.
* Footnotes are not all on the same page (as in tools/test.txt) and do not link back
  and forth.
* No link to system errors.
* Hyperlinks are not hyphenated this leads to bad spacing. see tools/test.txt
  2.14 directives directives
* Meta keywords into pdf ?
* Pagestyle headings does not work, when sections are starred.
* For additional docinfo items: the field_body is inserted as text, i.e. no
  markup is done.
* Multiple author entries in docinfo (same thing as in html).
* pslatex has not textbackslash in bold ?
* make use (but test against pslatex) ::

    \textbackslash      \backslash \
    \textbar            \mid       |
    \textless           <          <
    \textgreater        >          >
    \textasciicircum    \hat       ^
    \textasciitilde     \tilde     ~
    \textbullet         \bullet    gefüllter Kreis auf halber Höhe
    \textperiodcentered \cdot      ·
    \textvisiblespace   (n.v.)     sichtbares Leerzeichen
* keep literal-blocks together on a page, avoid pagebreaks.

  failed experiments up to now: samepage, minipage, pagebreak 1 to 4 before
  the block.


==========================
 Docutils Front-End Tools
==========================

:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 1.45 $
:Date: $Date: 2004/06/07 15:52:08 $
:Copyright: This document has been placed in the public domain.

.. contents::

Introduction
============

Once the Docutils package is unpacked, you will discover a "``tools``"
directory containing several front ends for common Docutils
processing.  Rather than a single all-purpose program, Docutils has
many small front ends, each specialized for a specific "Reader" (which
knows how to interpret a file in context), a "Parser" (which
understands the syntax of the text), and a "Writer" (which knows how
to generate a specific data format).

Most front ends have common options and the same command-line usage
pattern::

    toolname [options] [<source> [<destination]]

(The exceptions are buildhtml.py_ and pep2html.py_.)  See rst2html.py_
for concrete examples.  Each tool has a "``--help``" option which
lists the `command-line options`_ and arguments it supports.
Processing can also be customized with `configuration files`_.

The two arguments, "source" and "destination", are optional.  If only
one argument (source) is specified, the standard output (stdout) is
used for the destination.  If no arguments are specified, the standard
input (stdin) is used for the source as well.

Getting Help
------------

First, try the "``--help``" option each front-end tool has.

Users who have questions or need assistance with Docutils or
reStructuredText should `post a message`_ to the `Docutils-Users
mailing list`_.  The `Docutils project web site`_ has more
information.

.. _post a message: mailto:docutils-users@lists.sourceforge.net
.. _Docutils-Users mailing list:
   http://lists.sourceforge.net/lists/listinfo/docutils-users
.. _Docutils project web site: http://docutils.sourceforge.net/

The Tools
=========

buildhtml.py
------------

:Readers: Standalone, PEP
:Parser: reStructuredText
:Writers: HTML, PEP/HTML

Use ``buildhtml.py`` to generate .html from all the .txt files
(including PEPs) in each <directory> given, and their subdirectories
too.  (Use the ``--local`` option to skip subdirectories.)

Usage::

    buildhtml.py [options] [<directory> ...]

After unpacking the Docutils package, the following shell commands
will generate HTML for all included documentation::

    cd docutils/tools
    buildhtml.py ..

For official releases, the directory may be called "docutils-X.Y",
where "X.Y" is the release version.  Alternatively::

    cd docutils
    tools/buildhtml.py --config=tools/docutils.conf

The current directory (and all subdirectories) is chosen by default if
no directory is named.  Some files may generate system messages
(tools/test.txt contains intentional errors); use the ``--quiet``
option to suppress all warnings.  The ``--config`` option ensures that
the correct stylesheets, templates, and settings are in place (a
``docutils.conf`` configuration file in the current directory is
picked up automatically).  Command-line options may be used to
override config file settings or replace them altogether.

rst2html.py
-----------

:Reader: Standalone
:Parser: reStructuredText
:Writer: HTML

The ``rst2html.py`` front end reads standalone reStructuredText source
files and produces HTML 4 (XHTML 1) output compatible with modern
browsers.  For example, to process a reStructuredText file
"``test.txt``" into HTML::

    rst2html.py test.txt test.html

In fact, there *is* a "``test.txt``" file in the "``tools``"
directory.  It contains "at least one example of each reStructuredText
construct", including several intentional errors (to test the error
reporting system).  Use it to put the system through its paces and
compare input to output.

Now open the "``test.html``" file in your favorite browser to see the
results.  To get a footer with a link to the source file, date & time
of processing, and links to the Docutils projects, add some options::

    rst2html.py -stg test.txt test.html

Stylesheets
```````````

``rst2html.py`` inserts into the generated HTML a link to a cascading
stylesheet, defaulting to "``default.css``" (override with a
"``--stylesheet``" or "``--stylesheet-path``" command-line option or
with configuration file settings).  The
"``tools/stylesheets/default.css``" stylesheet is provided for basic
use.  To experiment with styles, rather than editing the default
stylesheet (which will be updated as the project evolves), it is
recommended to use an "``@import``" statement to create a "wrapper"
stylesheet.  For example, a "``my.css``" stylesheet could contain the
following::

    @import url(default.css);

    h1, h2, h3, h4, h5, h6, p.topic-title {
      font-family: sans-serif }

Generate HTML with the following command::

    rst2html.py -stg --stylesheet my.css test.txt test.html

When viewed in a browser, the new "wrapper" stylesheet will change the
typeface family of titles to "sans serif", typically Helvetica or
Arial.  Other styles will not be affected.  Styles in wrapper
stylesheets override styles in imported stylesheets, enabling
incremental experimentation.

pep.py
------

:Reader: PEP
:Parser: reStructuredText
:Writer: PEP/HTML

``pep.py`` reads a new-style PEP (marked up with reStructuredText) and
produces HTML.  It requires a template file and a stylesheet.  By
default, it makes use of a "``pep-html-template``" file and a
"``default.css``" stylesheet in the current directory, but these can
be overridden by command-line options or configuration files.  The
"``tools/stylesheets/pep.css``" stylesheet is intended specifically
for PEP use.

The "``docutils.conf``" `configuration file`_ in the "``tools``"
directory of Docutils contains a default setup for use in processing
the PEP files (``docs/peps/pep-*.txt``) into HTML.  It specifies a
default template (``tools/pep-html-template``) and a default
stylesheet (``tools/stylesheets/pep.css``).  See Stylesheets_ above
for more information.

``pep.py`` can be run from the ``tools`` directory or from the
``docs/peps/`` directory, by adjusting the settings.  These two sets
of commands are equivalent::

    cd <path-to-docutils>/tools
    # This will pick up the "docutils.conf" file automatically:
    pep.py ../docs/peps/pep-0287.txt ../docs/peps/pep-0287.html

    cd <path-to-docutils>/docs/peps
    # Must tell the tool where to find the config file:
    ../../tools/pep.py --config ../../tools/docutils.conf \
        pep-0287.txt pep-0287.html

pep2html.py
-----------

:Reader: PEP
:Parser: reStructuredText
:Writer: PEP/HTML

``pep2html.py`` is a modified version of the original script by
Fredrik Lundh, with support for Docutils added.  It reads the
beginning of a PEP text file to determine the format (old-style
indented or new-style reStructuredText) and processes accordingly.
Since it does not use the Docutils front end mechanism (the common
command-line options are not supported), it can only be configured
using `configuration files`_.  The template and stylesheet
requirements of ``pep2html.py`` are the same as those of `pep.py`_
above.

Arguments to ``pep2html.py`` may be a list of PEP numbers or .txt
files.  If no arguments are given, all files of the form
"``pep-*.txt``" are processed.

rst2latex.py
------------

:Reader: Standalone
:Parser: reStructuredText
:Writer: LaTeX2e

The ``rst2latex.py`` front end reads standalone reStructuredText
source files and produces LaTeX2e output. For example, to process a
reStructuredText file "``test.txt``" into LaTeX::

    rst2latex.py test.txt test.tex

The output file "``test.tex``" should then be processed with ``latex``
or ``pdflatex`` to get a typeset document.

Some limitations and difference apply:

- GIF, JPG and PNG images are not handled, when processed with
  ``latex``; use ``pdflatex`` instead.
- Only the Latin-1 output encoding has been tested up to now (Latin-1
  has been made the default output encoding for LaTeX).
- The optional stylesheet file allows the inclusion of special packages
  or overwriting default settings for LaTeX.
- Not all constructs are possible, see `Generating LaTeX with Docutils`_.

docutils-xml.py
---------------

:Reader: Standalone
:Parser: reStructuredText
:Writer: XML (Docutils native)

The ``docutils-xml.py`` front end produces Docutils-native XML output.
This can be transformed with standard XML tools such as XSLT
processors into arbitrary final forms.

publish.py
----------

:Reader: Standalone
:Parser: reStructuredText
:Writer: Pseudo-XML

``publish.py`` is used for debugging the Docutils "Reader to Transform
to Writer" pipeline.  It produces a compact pretty-printed
"pseudo-XML", where nesting is indicated by indentation (no end-tags).
External attributes for all elements are output, and internal
attributes for any leftover "pending" elements are also given.

quicktest.py
------------

:Reader: N/A
:Parser: reStructuredText
:Writer: N/A

The ``quicktest.py`` tool is used for testing the reStructuredText
parser.  It does not use a Docutils Reader or Writer or the standard
Docutils command-line options.  Rather, it does its own I/O and calls
the parser directly.  No transforms are applied to the parsed
document.  Various forms output are possible:

- Pretty-printed pseudo-XML (default)
- Test data (Python list of input and pseudo-XML output strings;
  useful for creating new test cases)
- Pretty-printed native XML
- Raw native XML (with or without a stylesheet reference)

Customization
=============

Command-Line Options
--------------------

Each front-end tool supports command-line options for one-off
customization.  For persistent customization, use `configuration
files`_.  Command-line options take priority over configuration file
settings.

Use the "--help" option on each of the front ends to list the
command-line options it supports.  Command-line options and their
corresponding configuration file entry names are listed in the
`Docutils Configuration Files`_ document.

.. _configuration file:

Configuration Files
-------------------

Configuration files are used for persistent customization; they can be
set once and take effect every time you use a front-end tool.

For details, see `Docutils Configuration Files`_.

.. _Docutils Configuration Files: config.html
.. _Generating LaTeX with Docutils: latex.html

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


A ReStructuredText Primer
=========================

:Author: Richard Jones
:Version: $Revision: 1.17 $
:Copyright: This document has been placed in the public domain.

.. contents::

The text below contains links that look like "(quickref__)".  These
are relative links that point to the `Quick reStructuredText`_ user
reference.  If these links don't work, please refer to the `master
quick reference`_ document.

__
.. _Quick reStructuredText: quickref.html
.. _master quick reference:
   http://docutils.sourceforge.net/docs/user/rst/quickref.html

Structure
---------

From the outset, let me say that "Structured Text" is probably a bit
of a misnomer.  It's more like "Relaxed Text" that uses certain
consistent patterns.  These patterns are interpreted by a HTML
converter to produce "Very Structured Text" that can be used by a web
browser.

The most basic pattern recognised is a **paragraph** (quickref__).
That's a chunk of text that is separated by blank lines (one is
enough).  Paragraphs must have the same indentation -- that is, line
up at their left edge.  Paragraphs that start indented will result in
indented quote paragraphs. For example::

  This is a paragraph.  It's quite
  short.

     This paragraph will result in an indented block of
     text, typically used for quoting other text.

  This is another one.

Results in:

  This is a paragraph.  It's quite
  short.

     This paragraph will result in an indented block of
     text, typically used for quoting other text.

  This is another one.

__ quickref.html#paragraphs

Text styles
-----------

(quickref__)

__ quickref.html#inline-markup

Inside paragraphs and other bodies of text, you may additionally mark
text for *italics* with "``*italics*``" or **bold** with
"``**bold**``".

If you want something to appear as a fixed-space literal, use
"````double back-quotes````".  Note that no further fiddling is done
inside the double back-quotes -- so asterisks "``*``" etc. are left
alone.

If you find that you want to use one of the "special" characters in
text, it will generally be OK -- reStructuredText is pretty smart.
For example, this * asterisk is handled just fine.  If you actually
want text \*surrounded by asterisks* to **not** be italicised, then
you need to indicate that the asterisk is not special.  You do this by
placing a backslash just before it, like so "``\*``" (quickref__), or
by enclosing it in double back-quotes (inline literals), like this::

    ``\*``

__ quickref.html#escaping

Lists
-----

Lists of items come in three main flavours: **enumerated**,
**bulleted** and **definitions**.  In all list cases, you may have as
many paragraphs, sublists, etc. as you want, as long as the left-hand
side of the paragraph or whatever aligns with the first line of text
in the list item.

Lists must always start a new paragraph -- that is, they must appear
after a blank line.

**enumerated** lists (numbers, letters or roman numerals; quickref__)
  __ quickref.html#enumerated-lists

  Start a line off with a number or letter followed by a period ".",
  right bracket ")" or surrounded by brackets "( )" -- whatever you're
  comfortable with.  All of the following forms are recognised::

    1. numbers

    A. upper-case letters
       and it goes over many lines

       with two paragraphs and all!

    a. lower-case letters

       3. with a sub-list starting at a different number
       4. make sure the numbers are in the correct sequence though!

    I. upper-case roman numerals

    i. lower-case roman numerals

    (1) numbers again

    1) and again

  Results in (note: the different enumerated list styles are not
  always supported by every web browser, so you may not get the full
  effect here):

  1. numbers

  A. upper-case letters
     and it goes over many lines

     with two paragraphs and all!

  a. lower-case letters

     3. with a sub-list starting at a different number
     4. make sure the numbers are in the correct sequence though!

  I. upper-case roman numerals

  i. lower-case roman numerals

  (1) numbers again

  1) and again

**bulleted** lists (quickref__)
  __ quickref.html#bullet-lists

  Just like enumerated lists, start the line off with a bullet point
  character - either "-", "+" or "*"::

    * a bullet point using "*"

      - a sub-list using "-"

        + yet another sub-list

      - another item

  Results in:

  * a bullet point using "*"

    - a sub-list using "-"

      + yet another sub-list

    - another item

**definition** lists (quickref__)
  __ quickref.html#definition-lists

  Unlike the other two, the definition lists consist of a term, and
  the definition of that term.  The format of a definition list is::

    what
      Definition lists associate a term with a definition.

    *how*
      The term is a one-line phrase, and the definition is one or more
      paragraphs or body elements, indented relative to the term.
      Blank lines are not allowed between term and definition.

  Results in:

  what
    Definition lists associate a term with a definition.

  *how*
    The term is a one-line phrase, and the definition is one or more
    paragraphs or body elements, indented relative to the term.
    Blank lines are not allowed between term and definition.

Preformatting (code samples)
----------------------------
(quickref__)

__ quickref.html#literal-blocks

To just include a chunk of preformatted, never-to-be-fiddled-with
text, finish the prior paragraph with "``::``".  The preformatted
block is finished when the text falls back to the same indentation
level as a paragraph prior to the preformatted block.  For example::

  An example::

      Whitespace, newlines, blank lines, and all kinds of markup
        (like *this* or \this) is preserved by literal blocks.
    Lookie here, I've dropped an indentation level
    (but not far enough)

  no more example

Results in:

  An example::

      Whitespace, newlines, blank lines, and all kinds of markup
        (like *this* or \this) is preserved by literal blocks.
    Lookie here, I've dropped an indentation level
    (but not far enough)

  no more example

Note that if a paragraph consists only of "``::``", then it's removed
from the output::

  ::

      This is preformatted text, and the
      last "::" paragraph is removed

Results in:

::

    This is preformatted text, and the
    last "::" paragraph is removed

Sections
--------

(quickref__)

__ quickref.html#section-structure

To break longer text up into sections, you use **section headers**.
These are a single line of text (one or more words) with adornment: an
underline alone, or an underline and an overline together, in dashes
"``-----``", equals "``======``", tildes "``~~~~~~``" or any of the
non-alphanumeric characters ``= - ` : ' " ~ ^ _ * + # < >`` that you
feel comfortable with.  An underline-only adornment is distinct from
an overline-and-underline adornment using the same character.  The
underline/overline must be at least as long as the title text.  Be
consistent, since all sections marked with the same adornment style
are deemed to be at the same level::

  Chapter 1 Title
  ===============

  Section 1.1 Title
  -----------------

  Subsection 1.1.1 Title
  ~~~~~~~~~~~~~~~~~~~~~~

  Section 1.2 Title
  -----------------

  Chapter 2 Title
  ===============

This results in the following structure, illustrated by simplified
pseudo-XML::

    <section>
        <title>
            Chapter 1 Title
        <section>
            <title>
                Section 1.1 Title
            <section>
                <title>
                    Subsection 1.1.1 Title
        <section>
            <title>
                Section 1.2 Title
    <section>
        <title>
            Chapter 2 Title

(Pseudo-XML uses indentation for nesting and has no end-tags.  It's
not possible to show actual processed output, as in the other
examples, because sections cannot exist inside block quotes.  For a
concrete example, compare the section structure of this document's
source text and processed output.)

Note that section headers are available as link targets, just using
their name.  To link to the Lists_ heading, I write "``Lists_``".  If
the heading has a space in it like `text styles`_, we need to quote
the heading "```text styles`_``".

Document Title / Subtitle
`````````````````````````

The title of the whole document is distinct from section titles and
may be formatted somewhat differently (e.g. the HTML writer by default
shows it as a centered heading).

To indicate the document title in reStructuredText, use a unique adornment
style at the beginning of the document.  To indicate the document subtitle,
use another unique adornment style immediately after the document title.  For
example::

    ================
     Document Title
    ================
    ----------
     Subtitle
    ----------

    Section Title
    =============

    ...

Note that "Document Title" and "Section Title" above both use equals
signs, but are distict and unrelated styles.  The text of
overline-and-underlined titles (but not underlined-only) may be inset
for aesthetics.

Images
------

(quickref__)

__ quickref.html#directives

To include an image in your document, you use the the ``image`` directive__.
For example::

  .. image:: images/biohazard.png

results in:

.. image:: images/biohazard.png

The ``images/biohazard.png`` part indicates the filename of the image
you wish to appear in the document. There's no restriction placed on
the image (format, size etc).  If the image is to appear in HTML and
you wish to supply additional information, you may::

  .. image:: images/biohazard.png
     :height: 100
     :width: 200
     :scale: 50
     :alt: alternate text

See the full `image directive documentation`__ for more info.

__ ../../ref/rst/directives.html
__ ../../ref/rst/directives.html#images

What Next?
----------

This primer introduces the most common features of reStructuredText,
but there are a lot more to explore.  The `Quick reStructuredText`_
user reference is a good place to go next.  For complete details, the
`reStructuredText Markup Specification`_ is the place to go [#]_.

Users who have questions or need assistance with Docutils or
reStructuredText should `post a message`_ to the `Docutils-Users
mailing list`_.  The `Docutils project web site`_ has more
information.

.. [#] If that relative link doesn't work, try the master document:
   http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html.

.. _reStructuredText Markup Specification:
   ../../ref/rst/restructuredtext.html
.. _post a message: mailto:docutils-users@lists.sourceforge.net
.. _Docutils-Users mailing list:
   http://lists.sourceforge.net/lists/listinfo/docutils-users
.. _Docutils project web site: http://docutils.sourceforge.net/


            GNU GENERAL PUBLIC LICENSE
               Version 2, June 1991

 Copyright (C) 1989, 1991 Free Software Foundation, Inc.
                       59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.

                Preamble

  The licenses for most software are designed to take away your
freedom to share and change it.  By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users.  This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it.  (Some other Free Software Foundation software is covered by
the GNU Library General Public License instead.)  You can apply it to
your programs, too.

  When we speak of free software, we are referring to freedom, not
price.  Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.

  To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.

  For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have.  You must make sure that they, too, receive or can get the
source code.  And you must show them these terms so they know their
rights.

  We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.

  Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software.  If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.

  Finally, any free program is threatened constantly by software
patents.  We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary.  To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.

  The precise terms and conditions for copying, distribution and
modification follow.

            GNU GENERAL PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License.  The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language.  (Hereinafter, translation is included without limitation in
the term "modification".)  Each licensee is addressed as "you".

Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope.  The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.

  1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.

You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.

  2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:

    a) You must cause the modified files to carry prominent notices
    stating that you changed the files and the date of any change.

    b) You must cause any work that you distribute or publish, that in
    whole or in part contains or is derived from the Program or any
    part thereof, to be licensed as a whole at no charge to all third
    parties under the terms of this License.

    c) If the modified program normally reads commands interactively
    when run, you must cause it, when started running for such
    interactive use in the most ordinary way, to print or display an
    announcement including an appropriate copyright notice and a
    notice that there is no warranty (or else, saying that you provide
    a warranty) and that users may redistribute the program under
    these conditions, and telling the user how to view a copy of this
    License.  (Exception: if the Program itself is interactive but
    does not normally print such an announcement, your work based on
    the Program is not required to print an announcement.)

These requirements apply to the modified work as a whole.  If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works.  But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.

Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.

In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.

  3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:

    a) Accompany it with the complete corresponding machine-readable
    source code, which must be distributed under the terms of Sections
    1 and 2 above on a medium customarily used for software interchange; or,

    b) Accompany it with a written offer, valid for at least three
    years, to give any third party, for a charge no more than your
    cost of physically performing source distribution, a complete
    machine-readable copy of the corresponding source code, to be
    distributed under the terms of Sections 1 and 2 above on a medium
    customarily used for software interchange; or,

    c) Accompany it with the information you received as to the offer
    to distribute corresponding source code.  (This alternative is
    allowed only for noncommercial distribution and only if you
    received the program in object code or executable form with such
    an offer, in accord with Subsection b above.)

The source code for a work means the preferred form of the work for
making modifications to it.  For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable.  However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.

If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.

  4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License.  Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.

  5. You are not required to accept this License, since you have not
signed it.  However, nothing else grants you permission to modify or
distribute the Program or its derivative works.  These actions are
prohibited by law if you do not accept this License.  Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.

  6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions.  You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.

  7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License.  If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all.  For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.

If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.

It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices.  Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.

This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.

  8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded.  In such case, this License incorporates
the limitation as if written in the body of this License.

  9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time.  Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.

Each version is given a distinguishing version number.  If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation.  If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.

  10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission.  For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this.  Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.

                NO WARRANTY

  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.

  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.

             END OF TERMS AND CONDITIONS

        How to Apply These Terms to Your New Programs

  If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.

  To do so, attach the following notices to the program.  It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.

    <one line to give the program's name and a brief idea of what it does.>
    Copyright (C) <year>  <name of author>

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

Also add information on how to contact you by electronic and paper mail.

If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:

    Gnomovision version 69, Copyright (C) year name of author
    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
    This is free software, and you are welcome to redistribute it
    under certain conditions; type `show c' for details.

The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License.  Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.

You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary.  Here is a sample; alter the names:

  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
  `Gnomovision' (which makes passes at compilers) written by James Hacker.

  <signature of Ty Coon>, 1 April 1989
  Ty Coon, President of Vice

This General Public License does not permit incorporating your program into
proprietary programs.  If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library.  If this is what you want to do, use the GNU Library General
Public License instead of this License.


A. HISTORY OF THE SOFTWARE
==========================

Python was created in the early 1990s by Guido van Rossum at Stichting
Mathematisch Centrum (CWI) in the Netherlands as a successor of a
language called ABC.  Guido is Python's principal author, although it
includes many contributions from others.  The last version released
from CWI was Python 1.2.  In 1995, Guido continued his work on Python
at the Corporation for National Research Initiatives (CNRI) in Reston,
Virginia where he released several versions of the software.  Python
1.6 was the last of the versions released by CNRI.  In 2000, Guido and
the Python core development team moved to BeOpen.com to form the
BeOpen PythonLabs team.  Python 2.0 was the first and only release
from BeOpen.com.

Following the release of Python 1.6, and after Guido van Rossum left
CNRI to work with commercial software developers, it became clear that
the ability to use Python with software available under the GNU Public
License (GPL) was very desirable.  CNRI and the Free Software
Foundation (FSF) interacted to develop enabling wording changes to the
Python license.  Python 1.6.1 is essentially the same as Python 1.6,
with a few minor bug fixes, and with a different license that enables
later versions to be GPL-compatible.  Python 2.1 is a derivative work
of Python 1.6.1, as well as of Python 2.0.

After Python 2.0 was released by BeOpen.com, Guido van Rossum and the
other PythonLabs developers joined Digital Creations.  All
intellectual property added from this point on, starting with Python
2.1 and its alpha and beta releases, is owned by the Python Software
Foundation (PSF), a non-profit modeled after the Apache Software
Foundation.  See http://www.python.org/psf/ for more information about
the PSF.

Thanks to the many outside volunteers who have worked under Guido's
direction to make these releases possible.

B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
===============================================================

PSF LICENSE AGREEMENT
---------------------

1. This LICENSE AGREEMENT is between the Python Software Foundation
("PSF"), and the Individual or Organization ("Licensee") accessing and
otherwise using Python 2.1.1 software in source or binary form and its
associated documentation.

2. Subject to the terms and conditions of this License Agreement, PSF
hereby grants Licensee a nonexclusive, royalty-free, world-wide
license to reproduce, analyze, test, perform and/or display publicly,
prepare derivative works, distribute, and otherwise use Python 2.1.1
alone or in any derivative version, provided, however, that PSF's
License Agreement and PSF's notice of copyright, i.e., "Copyright (c)
2001 Python Software Foundation; All Rights Reserved" are retained in
Python 2.1.1 alone or in any derivative version prepared by Licensee.

3. In the event Licensee prepares a derivative work that is based on
or incorporates Python 2.1.1 or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python 2.1.1.

4. PSF is making Python 2.1.1 available to Licensee on an "AS IS"
basis.  PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 2.1.1 WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
2.1.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 2.1.1,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

7. Nothing in this License Agreement shall be deemed to create any
relationship of agency, partnership, or joint venture between PSF and
Licensee.  This License Agreement does not grant permission to use PSF
trademarks or trade name in a trademark sense to endorse or promote
products or services of Licensee, or any third party.

8. By copying, installing or otherwise using Python 2.1.1, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.

BEOPEN.COM TERMS AND CONDITIONS FOR PYTHON 2.0
----------------------------------------------

BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1

1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
Individual or Organization ("Licensee") accessing and otherwise using
this software in source or binary form and its associated
documentation ("the Software").

2. Subject to the terms and conditions of this BeOpen Python License
Agreement, BeOpen hereby grants Licensee a non-exclusive,
royalty-free, world-wide license to reproduce, analyze, test, perform
and/or display publicly, prepare derivative works, distribute, and
otherwise use the Software alone or in any derivative version,
provided, however, that the BeOpen Python License is retained in the
Software, alone or in any derivative version prepared by Licensee.

3. BeOpen is making the Software available to Licensee on an "AS IS"
basis.  BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

5. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

6. This License Agreement shall be governed by and interpreted in all
respects by the law of the State of California, excluding conflict of
law provisions.  Nothing in this License Agreement shall be deemed to
create any relationship of agency, partnership, or joint venture
between BeOpen and Licensee.  This License Agreement does not grant
permission to use BeOpen trademarks or trade names in a trademark
sense to endorse or promote products or services of Licensee, or any
third party.  As an exception, the "BeOpen Python" logos available at
http://www.pythonlabs.com/logos.html may be used according to the
permissions granted on that web page.

7. By copying, installing or otherwise using the software, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.

CNRI OPEN SOURCE GPL-COMPATIBLE LICENSE AGREEMENT
-------------------------------------------------

1. This LICENSE AGREEMENT is between the Corporation for National
Research Initiatives, having an office at 1895 Preston White Drive,
Reston, VA 20191 ("CNRI"), and the Individual or Organization
("Licensee") accessing and otherwise using Python 1.6.1 software in
source or binary form and its associated documentation.

2. Subject to the terms and conditions of this License Agreement, CNRI
hereby grants Licensee a nonexclusive, royalty-free, world-wide
license to reproduce, analyze, test, perform and/or display publicly,
prepare derivative works, distribute, and otherwise use Python 1.6.1
alone or in any derivative version, provided, however, that CNRI's
License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
1995-2001 Corporation for National Research Initiatives; All Rights
Reserved" are retained in Python 1.6.1 alone or in any derivative
version prepared by Licensee.  Alternately, in lieu of CNRI's License
Agreement, Licensee may substitute the following text (omitting the
quotes): "Python 1.6.1 is made available subject to the terms and
conditions in CNRI's License Agreement.  This Agreement together with
Python 1.6.1 may be located on the Internet using the following
unique, persistent identifier (known as a handle): 1895.22/1013.  This
Agreement may also be obtained from a proxy server on the Internet
using the following URL: http://hdl.handle.net/1895.22/1013".

3. In the event Licensee prepares a derivative work that is based on
or incorporates Python 1.6.1 or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python 1.6.1.

4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
basis.  CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

7. This License Agreement shall be governed by the federal
intellectual property law of the United States, including without
limitation the federal copyright law, and, to the extent such
U.S. federal law does not apply, by the law of the Commonwealth of
Virginia, excluding Virginia's conflict of law provisions.
Notwithstanding the foregoing, with regard to derivative works based
on Python 1.6.1 that incorporate non-separable material that was
previously distributed under the GNU General Public License (GPL), the
law of the Commonwealth of Virginia shall govern this License
Agreement only as to issues arising under or with respect to
Paragraphs 4, 5, and 7 of this License Agreement.  Nothing in this
License Agreement shall be deemed to create any relationship of
agency, partnership, or joint venture between CNRI and Licensee.  This
License Agreement does not grant permission to use CNRI trademarks or
trade name in a trademark sense to endorse or promote products or
services of Licensee, or any third party.

8. By clicking on the "ACCEPT" button where indicated, or by copying,
installing or otherwise using Python 1.6.1, Licensee agrees to be
bound by the terms and conditions of this License Agreement.

        ACCEPT

CWI PERMISSIONS STATEMENT AND DISCLAIMER
----------------------------------------

Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
The Netherlands.  All rights reserved.

Permission to use, copy, modify, and distribute this software and its
documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation, and that the name of Stichting Mathematisch
Centrum or CWI not be used in advertising or publicity pertaining to
distribution of the software without specific, written prior
permission.

STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.


A. HISTORY OF THE SOFTWARE
==========================

Python was created in the early 1990s by Guido van Rossum at Stichting
Mathematisch Centrum (CWI, see http://www.cwi.nl) in the Netherlands
as a successor of a language called ABC.  Guido remains Python's
principal author, although it includes many contributions from others.

In 1995, Guido continued his work on Python at the Corporation for
National Research Initiatives (CNRI, see http://www.cnri.reston.va.us)
in Reston, Virginia where he released several versions of the
software.

In May 2000, Guido and the Python core development team moved to
BeOpen.com to form the BeOpen PythonLabs team.  In October of the same
year, the PythonLabs team moved to Digital Creations (see
http://www.digicool.com).  In 2001, the Python Software Foundation
(PSF, see http://www.python.org/psf/) was formed, a non-profit
organization created specifically to own Python-related Intellectual
Property.  Digital Creations is a sponsoring member of the PSF.

All Python releases are Open Source (see http://www.opensource.org for
the Open Source Definition).  Historically, most, but not all, Python
releases have also been GPL-compatible; the table below summarizes
the various releases.

    Release         Derived     Year        Owner       GPL-
            from                                compatible? (1)

    0.9.0 thru 1.2              1991-1995   CWI         yes
    1.3 thru 1.5.2  1.2         1995-1999   CNRI        yes
    1.6             1.5.2       2000        CNRI        no
    2.0             1.6         2000        BeOpen.com  no
    1.6.1           1.6         2001        CNRI        no
    2.1             2.0+1.6.1   2001        PSF         no
    2.0.1           2.0+1.6.1   2001        PSF         yes
    2.1.1           2.1+2.0.1   2001        PSF         yes
    2.2             2.1.1       2001        PSF         yes

Footnotes:

(1) GPL-compatible doesn't mean that we're distributing Python under
    the GPL.  All Python licenses, unlike the GPL, let you distribute
    a modified version without making your changes open source.  The
    GPL-compatible licenses make it possible to combine Python with
    other software that is released under the GPL; the others don't.

Thanks to the many outside volunteers who have worked under Guido's
direction to make these releases possible.

B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
===============================================================

PSF LICENSE AGREEMENT FOR PYTHON 2.2
------------------------------------

1. This LICENSE AGREEMENT is between the Python Software Foundation
("PSF"), and the Individual or Organization ("Licensee") accessing and
otherwise using Python 2.2 software in source or binary form and its
associated documentation.

2. Subject to the terms and conditions of this License Agreement, PSF
hereby grants Licensee a nonexclusive, royalty-free, world-wide
license to reproduce, analyze, test, perform and/or display publicly,
prepare derivative works, distribute, and otherwise use Python 2.2
alone or in any derivative version, provided, however, that PSF's
License Agreement and PSF's notice of copyright, i.e., "Copyright (c)
2001 Python Software Foundation; All Rights Reserved" are retained in
Python 2.2 alone or in any derivative version prepared by Licensee.

3. In the event Licensee prepares a derivative work that is based on
or incorporates Python 2.2 or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python 2.2.

4. PSF is making Python 2.2 available to Licensee on an "AS IS"
basis.  PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 2.2 WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
2.2 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 2.2,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

7. Nothing in this License Agreement shall be deemed to create any
relationship of agency, partnership, or joint venture between PSF and
Licensee.  This License Agreement does not grant permission to use PSF
trademarks or trade name in a trademark sense to endorse or promote
products or services of Licensee, or any third party.

8. By copying, installing or otherwise using Python 2.2, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.

BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0
-------------------------------------------

BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1

1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
Individual or Organization ("Licensee") accessing and otherwise using
this software in source or binary form and its associated
documentation ("the Software").

2. Subject to the terms and conditions of this BeOpen Python License
Agreement, BeOpen hereby grants Licensee a non-exclusive,
royalty-free, world-wide license to reproduce, analyze, test, perform
and/or display publicly, prepare derivative works, distribute, and
otherwise use the Software alone or in any derivative version,
provided, however, that the BeOpen Python License is retained in the
Software, alone or in any derivative version prepared by Licensee.

3. BeOpen is making the Software available to Licensee on an "AS IS"
basis.  BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

5. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

6. This License Agreement shall be governed by and interpreted in all
respects by the law of the State of California, excluding conflict of
law provisions.  Nothing in this License Agreement shall be deemed to
create any relationship of agency, partnership, or joint venture
between BeOpen and Licensee.  This License Agreement does not grant
permission to use BeOpen trademarks or trade names in a trademark
sense to endorse or promote products or services of Licensee, or any
third party.  As an exception, the "BeOpen Python" logos available at
http://www.pythonlabs.com/logos.html may be used according to the
permissions granted on that web page.

7. By copying, installing or otherwise using the software, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.

CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1
---------------------------------------

1. This LICENSE AGREEMENT is between the Corporation for National
Research Initiatives, having an office at 1895 Preston White Drive,
Reston, VA 20191 ("CNRI"), and the Individual or Organization
("Licensee") accessing and otherwise using Python 1.6.1 software in
source or binary form and its associated documentation.

2. Subject to the terms and conditions of this License Agreement, CNRI
hereby grants Licensee a nonexclusive, royalty-free, world-wide
license to reproduce, analyze, test, perform and/or display publicly,
prepare derivative works, distribute, and otherwise use Python 1.6.1
alone or in any derivative version, provided, however, that CNRI's
License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
1995-2001 Corporation for National Research Initiatives; All Rights
Reserved" are retained in Python 1.6.1 alone or in any derivative
version prepared by Licensee.  Alternately, in lieu of CNRI's License
Agreement, Licensee may substitute the following text (omitting the
quotes): "Python 1.6.1 is made available subject to the terms and
conditions in CNRI's License Agreement.  This Agreement together with
Python 1.6.1 may be located on the Internet using the following
unique, persistent identifier (known as a handle): 1895.22/1013.  This
Agreement may also be obtained from a proxy server on the Internet
using the following URL: http://hdl.handle.net/1895.22/1013".

3. In the event Licensee prepares a derivative work that is based on
or incorporates Python 1.6.1 or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python 1.6.1.

4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
basis.  CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

7. This License Agreement shall be governed by the federal
intellectual property law of the United States, including without
limitation the federal copyright law, and, to the extent such
U.S. federal law does not apply, by the law of the Commonwealth of
Virginia, excluding Virginia's conflict of law provisions.
Notwithstanding the foregoing, with regard to derivative works based
on Python 1.6.1 that incorporate non-separable material that was
previously distributed under the GNU General Public License (GPL), the
law of the Commonwealth of Virginia shall govern this License
Agreement only as to issues arising under or with respect to
Paragraphs 4, 5, and 7 of this License Agreement.  Nothing in this
License Agreement shall be deemed to create any relationship of
agency, partnership, or joint venture between CNRI and Licensee.  This
License Agreement does not grant permission to use CNRI trademarks or
trade name in a trademark sense to endorse or promote products or
services of Licensee, or any third party.

8. By clicking on the "ACCEPT" button where indicated, or by copying,
installing or otherwise using Python 1.6.1, Licensee agrees to be
bound by the terms and conditions of this License Agreement.

        ACCEPT

CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2
--------------------------------------------------

Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
The Netherlands.  All rights reserved.

Permission to use, copy, modify, and distribute this software and its
documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation, and that the name of Stichting Mathematisch
Centrum or CWI not be used in advertising or publicity pertaining to
distribution of the software without specific, written prior
permission.

STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.


A. HISTORY OF THE SOFTWARE
==========================

Python was created in the early 1990s by Guido van Rossum at Stichting
Mathematisch Centrum (CWI, see http://www.cwi.nl) in the Netherlands
as a successor of a language called ABC.  Guido remains Python's
principal author, although it includes many contributions from others.

In 1995, Guido continued his work on Python at the Corporation for
National Research Initiatives (CNRI, see http://www.cnri.reston.va.us)
in Reston, Virginia where he released several versions of the
software.

In May 2000, Guido and the Python core development team moved to
BeOpen.com to form the BeOpen PythonLabs team.  In October of the same
year, the PythonLabs team moved to Digital Creations (now Zope
Corporation, see http://www.zope.com).  In 2001, the Python Software
Foundation (PSF, see http://www.python.org/psf/) was formed, a
non-profit organization created specifically to own Python-related
Intellectual Property.  Zope Corporation is a sponsoring member of
the PSF.

All Python releases are Open Source (see http://www.opensource.org for
the Open Source Definition).  Historically, most, but not all, Python
releases have also been GPL-compatible; the table below summarizes
the various releases.

    Release         Derived     Year        Owner       GPL-
            from                                compatible? (1)

    0.9.0 thru 1.2              1991-1995   CWI         yes
    1.3 thru 1.5.2  1.2         1995-1999   CNRI        yes
    1.6             1.5.2       2000        CNRI        no
    2.0             1.6         2000        BeOpen.com  no
    1.6.1           1.6         2001        CNRI        no
    2.1             2.0+1.6.1   2001        PSF         no
    2.0.1           2.0+1.6.1   2001        PSF         yes
    2.1.1           2.1+2.0.1   2001        PSF         yes
    2.2             2.1.1       2001        PSF         yes
    2.1.2           2.1.1       2002        PSF         yes
    2.1.3           2.1.2       2002        PSF         yes
    2.2.1           2.2         2002        PSF         yes
    2.2.2           2.2.1       2002        PSF         yes
    2.3             2.2.2       2002-2003   PSF         yes

Footnotes:

(1) GPL-compatible doesn't mean that we're distributing Python under
    the GPL.  All Python licenses, unlike the GPL, let you distribute
    a modified version without making your changes open source.  The
    GPL-compatible licenses make it possible to combine Python with
    other software that is released under the GPL; the others don't.

Thanks to the many outside volunteers who have worked under Guido's
direction to make these releases possible.

B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
===============================================================

PSF LICENSE AGREEMENT FOR PYTHON 2.3
--------------------------------------

1. This LICENSE AGREEMENT is between the Python Software Foundation
("PSF"), and the Individual or Organization ("Licensee") accessing and
otherwise using Python 2.3 software in source or binary form and its
associated documentation.

2. Subject to the terms and conditions of this License Agreement, PSF
hereby grants Licensee a nonexclusive, royalty-free, world-wide
license to reproduce, analyze, test, perform and/or display publicly,
prepare derivative works, distribute, and otherwise use Python 2.3
alone or in any derivative version, provided, however, that PSF's
License Agreement and PSF's notice of copyright, i.e., "Copyright (c)
2001, 2002, 2003 Python Software Foundation; All Rights Reserved" are
retained in Python 2.3 alone or in any derivative version prepared by
Licensee.

3. In the event Licensee prepares a derivative work that is based on
or incorporates Python 2.3 or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python 2.3.

4. PSF is making Python 2.3 available to Licensee on an "AS IS"
basis.  PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 2.3 WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
2.3 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 2.3,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

7. Nothing in this License Agreement shall be deemed to create any
relationship of agency, partnership, or joint venture between PSF and
Licensee.  This License Agreement does not grant permission to use PSF
trademarks or trade name in a trademark sense to endorse or promote
products or services of Licensee, or any third party.

8. By copying, installing or otherwise using Python 2.3, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.

BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0
-------------------------------------------

BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1

1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
Individual or Organization ("Licensee") accessing and otherwise using
this software in source or binary form and its associated
documentation ("the Software").

2. Subject to the terms and conditions of this BeOpen Python License
Agreement, BeOpen hereby grants Licensee a non-exclusive,
royalty-free, world-wide license to reproduce, analyze, test, perform
and/or display publicly, prepare derivative works, distribute, and
otherwise use the Software alone or in any derivative version,
provided, however, that the BeOpen Python License is retained in the
Software, alone or in any derivative version prepared by Licensee.

3. BeOpen is making the Software available to Licensee on an "AS IS"
basis.  BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

5. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

6. This License Agreement shall be governed by and interpreted in all
respects by the law of the State of California, excluding conflict of
law provisions.  Nothing in this License Agreement shall be deemed to
create any relationship of agency, partnership, or joint venture
between BeOpen and Licensee.  This License Agreement does not grant
permission to use BeOpen trademarks or trade names in a trademark
sense to endorse or promote products or services of Licensee, or any
third party.  As an exception, the "BeOpen Python" logos available at
http://www.pythonlabs.com/logos.html may be used according to the
permissions granted on that web page.

7. By copying, installing or otherwise using the software, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.

CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1
---------------------------------------

1. This LICENSE AGREEMENT is between the Corporation for National
Research Initiatives, having an office at 1895 Preston White Drive,
Reston, VA 20191 ("CNRI"), and the Individual or Organization
("Licensee") accessing and otherwise using Python 1.6.1 software in
source or binary form and its associated documentation.

2. Subject to the terms and conditions of this License Agreement, CNRI
hereby grants Licensee a nonexclusive, royalty-free, world-wide
license to reproduce, analyze, test, perform and/or display publicly,
prepare derivative works, distribute, and otherwise use Python 1.6.1
alone or in any derivative version, provided, however, that CNRI's
License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
1995-2001 Corporation for National Research Initiatives; All Rights
Reserved" are retained in Python 1.6.1 alone or in any derivative
version prepared by Licensee.  Alternately, in lieu of CNRI's License
Agreement, Licensee may substitute the following text (omitting the
quotes): "Python 1.6.1 is made available subject to the terms and
conditions in CNRI's License Agreement.  This Agreement together with
Python 1.6.1 may be located on the Internet using the following
unique, persistent identifier (known as a handle): 1895.22/1013.  This
Agreement may also be obtained from a proxy server on the Internet
using the following URL: http://hdl.handle.net/1895.22/1013".

3. In the event Licensee prepares a derivative work that is based on
or incorporates Python 1.6.1 or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python 1.6.1.

4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
basis.  CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

7. This License Agreement shall be governed by the federal
intellectual property law of the United States, including without
limitation the federal copyright law, and, to the extent such
U.S. federal law does not apply, by the law of the Commonwealth of
Virginia, excluding Virginia's conflict of law provisions.
Notwithstanding the foregoing, with regard to derivative works based
on Python 1.6.1 that incorporate non-separable material that was
previously distributed under the GNU General Public License (GPL), the
law of the Commonwealth of Virginia shall govern this License
Agreement only as to issues arising under or with respect to
Paragraphs 4, 5, and 7 of this License Agreement.  Nothing in this
License Agreement shall be deemed to create any relationship of
agency, partnership, or joint venture between CNRI and Licensee.  This
License Agreement does not grant permission to use CNRI trademarks or
trade name in a trademark sense to endorse or promote products or
services of Licensee, or any third party.

8. By clicking on the "ACCEPT" button where indicated, or by copying,
installing or otherwise using Python 1.6.1, Licensee agrees to be
bound by the terms and conditions of this License Agreement.

        ACCEPT

CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2
--------------------------------------------------

Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
The Netherlands.  All rights reserved.

Permission to use, copy, modify, and distribute this software and its
documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation, and that the name of Stichting Mathematisch
Centrum or CWI not be used in advertising or publicity pertaining to
distribution of the software without specific, written prior
permission.

STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.