EP 10 — Public API and Backwards Compatibility Policy

Motivation

Docutils has a large user base and is used in production at several places (Python documentation, Linux kernel documentation, CMake documentation, readthedocs, ...). OTOH, Docutils has a version number below 1.0 (widely seen as an indicator of "beta" status of a project).

The current Docutils Project Policies section on version identification concentrates on the formal definition of the version specifier but leaves open what consists a "major change in the design or API".

The current backwards compatibility policy is a stub referencing PEP 387.

Rationale

Clearly defining how we will balance evolution with stability is important to both users and project developers.

People affected by changes in Docutils include:

Authors

writing or maintaining reStructuredText documents.

End-Users

of the Docutils front-end tools (optionally with 3rd-party drop-in extensions) or alternative tools using Docutils either as a library (Sphinx, …) or via the command line interface (build systems, Makefiles, scripts in other languages).

Developers

i.e. authors and maintainers of

• projects that use Docutils as a library (Sphinx, rsted, Leo, Pelican, ebookmaker, MyST, readthedocs, rinohtype, …),

• drop-in components (pycmark, rst2pdf, rst2beamer, …),

• alternative front-end tools,

• custom stylesheets (CSS style sheets, LaTeX styles, ODT styles), or

• re-implementations of the reStructuredText Markup Specification, e.g. Pandoc or Text-Restructured (prest).

A person may belong to more than one of these categories.

Public API

Docutils' public API includes:

• the reStructuredText Markup Specification [restructuredtext],

• the Docutils Document Tree [doctree] and the Docutils Generic Document Type Definition [docutils.dtd],

• names, command-line arguments and behaviour of the "console_scripts" entry points [tools] [settings],

• the core docutils Python package API:

  • the "publisher" interface for programmatic use [publisher],

  • component interfaces as defined by the abstract base classes docutils.readers.Reader, docutils.parsers.Parser, docutils.writer.Writer, and docutils.transform.Transform,

• behaviour and names of documented Python objects,

• output templates and style sheets provided with the writers,

• the interface to custom stylesheets -- elements, macros and classes used by writers to represent Doctree nodes in the output format.

Exemptions:

• Python objects, stylesheets, and templates can "opt-out" of the public API with a docstring noting that the object is provisional or internal.

• All undocumented objects should be assumed to be internal. [1]

Backwards Compatibility

Beginning with version 1.0, Docutils will follow the rules of Semantic Versioning. All incompatible changes to the public APIs require increasing the major part of the version specifier. Backwards compatible changes can be done in minor releases.

Security Implications

If required, critical bug fixes may change the public API without advance warning.

How to Teach This

• Copy the backwards compatibility declaration to the Docutils Project Policies. Link back to the public API specification in this document.

• Complete the API documentation and keep it up to date.

  • maintain and extend the API Reference Material for Client-Developers

  • Fix/enhance/add docstrings to improve the output of pydoc.

• Put the following text at a suitable place in the documentation:

  To find out if an object from the "docutils" package is safe to use, look up its docstring and the docstring of its parent(s) [2].

  If there is no documentation or the documentation says provisional or internal, the name, behaviour, and existence of the object is not guaranteed to be stable.

  Code relying on non-public objects should be made robust using public alternatives. If there is a no such alternative or the required change would be a problem, contact the Docutils developers or file a feature request.

Rejected Ideas

Use type annotations as an indication of status in the public API.

• There is no known precedence for this approach.

• Type annotations may be helpful also for non-public code.

Use Calendar Versioning (CalVer).

• Would be a break from current versioning without clear advantages.

Allow breaking API changes in minor versions after prior announcement and a deprecation period.

• Breaks the principle of least surprise.

Mark all internal objects with a prefix underscore.

• May needlessly break applications using internal objects.

Use the __all__ attribute to declare modules, classes, and functions that form the public API.

• This may be done later to improve "pydoc" output, but should not hold up the declaration of a Backwards Compatibility Policy.

Open Issues

• Differentiate between "core API" and "extended API"?

  There are many objects that were not intended for public use but are still used by downstream projects.

  Cf. the Docutils Project Policies

  When Docutils reaches version 1.0, the major APIs will be considered frozen.

  The major number [...] may be incremented later if there is a major change in the design or API.

• Define a minimum deprecation time similar to DocBook?

  A "major" release […] may contain backward-incompatible changes if:

  • the change was announced in the release notes for the previous version (major or minor) and

  • the change was announced in a release that occurred at least six months previously.

  By these rules, the technical committee can announce, in 5.1, for example, plans to make a backward-incompatible change in 6.0. Then, in 6.0, if it’s been at least six months since 5.1 was released, they can make that change.

  —https://tdg.docbook.org/tdg/5.1/ch01.html#bwcompat

• Generate API documentation with Sphinx?

  • nicely format rST docstrings

  • include attribute docstrings (ignored by pydoc).

References

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.