============================================= Public API and Backwards Compatibility Policy ============================================= :Author: Günter Milde, Adam Turner, open to all Docutils developers :Discussions-To: docutils-develop@lists.sf.net, https://sourceforge.net/p/docutils/feature-requests/89/ :Status: Draft :Type: Process :Created: 2022-01-21 :Docutils-Version: 0.19 Abstract ======== .. A short (<~200 word) description of the technical issue being addressed. This document suggests a definition of the public APIs provided by the Docutils project and the backwards compatibility policy. Motivation ========== .. Clearly explain why the existing specification is inadequate to address the problem that the proposal solves. Docutils has a large user base and is used in production at several places (Python documentation, Linux kernel documentation, CMake documentation, readthedocs, ...). OTOH, Docutils has a version number below 1.0 (widely seen as an indicator of "beta" status of a project). The current `Docutils Project Policies`_ section on `version identifcation`_ concentrates on the formal definition of the version specifier but leaves open what consists a "major change in the design or API". The current `backwards compatibility policy`__ is only a stub referencing :PEP:`387`. Clearly defining how we will balance evolution with stability is important to both users and project developers. .. _version specifier: .. _version identifcation: ../../docutils/docs/dev/policies.html#version-identification __ ../../docutils/docs/dev/policies.html#backwards-compatibility-policy Rationale ========= .. Describe why particular design decisions were made. People affected by changes in Docutils include: .. class:: description Authors writing or maintaining reStructuredText documents. End-Users of Docutils native `front-end tools`_ (optionally with 3rd-party drop-in extensions) or alternative tools using Docutils either as a library (Sphinx_, …) or via the command line interface (build systems, Makefiles, scripts in other languages). Developers authors and maintainers of - projects that use Docutils as a library (Sphinx_, rsted_, Leo_, Pelican_, ebookmaker_, MyST_, readthedocs_, rinohtype_, …), - drop-in components (pycmark_, rst2pdf_, rst2beamer_, …), - alternative front-end tools, - custom stylesheets (CSS style sheets, LaTeX styles, ODT styles), or - re-implementations of the `reStructuredText specification`_, e.g. PanDoc_ or Text-Restructured_ (prest). A person may belong to more than one of these catgories. .. _Sphinx: https://www.sphinx-doc.org/ .. _rsted: https://github.com/anru/rsted .. _Leo: https://leoeditor.com .. _Pelican: https://docs.getpelican.com/en/latest/ .. _MyST: https://myst-parser.readthedocs.io/ .. _readthedocs: https://readthedocs.org/ .. _pycmark: https://github.com/pycmark/pycmark .. _rst2pdf: https://rst2pdf.org/ .. _rst2beamer: https://docutils.sourceforge.io/sandbox/rst2beamer/ .. _rinohtype: https://pypi.org/project/rinohtype/ .. _ebookmaker: https://pypi.org/project/ebookmaker/ .. _PanDoc: https://pandoc.org/ .. _Text-Restructured: https://metacpan.org/dist/Text-Restructured Specification ============= .. Describe the syntax and semantics of any new feature. Docutils public APIs are: * the `reStructuredText specification`_, * the `Docutils document structure`_ (`Docutils Document Tree`), * names, command-line arguments and behaviour of the `"console_scripts" entry points`_ (`Front-end Tools`), * the core ``docutils`` Python package API: - the `Docutils Publisher`_ interface for programmatic use, - component interfaces as defined by the base `docutils.reader.Reader`, `docutils.writer.Writer`, and `docutils.transform.Transform` classes, * behaviour and names of all *documented Python objects*, [#]_ * *output templates* and *style sheets* provided with the writers, .. template.txt, default.tex, titlepage.tex, xelatex.tex .. html4css1.css, minimal.css, docutils.sty (LaTeX), styles.odt * the interface to custom stylesheets -- elements, macros and classes used by writers to represent doctree_ nodes in the output format. Exemptions: Python objects, stylesheets and templates can explicitly "opt-out" of the public API with a docstring noting that the object is provisional_ or internal. All undocumented objects should be assumed to be internal. [#]_ See also the `API Reference Material for Client-Developers`_. .. [#] Cf. `PEP 387: Backwards Compatibility Rules`_ .. [#] Cf. `PEP 008: Public and Internal Interfaces`_ Backwards Compatibility ======================= .. Describe potential impact and severity on pre-existing code. Beginning with version 1.0, Docutils will follow the rules of `Semantic Versioning`_. All incompatible changes to the public APIs require increasing the `major` part of the `version specifier`_. Backwards compatible changes can be done in `minor` releases. Security Implications ===================== If required, critical bug fixes may change the public API without advance warning. How to Teach This ================= .. How to teach users, new and experienced, how to apply the proposal to their work. * Move the API specification_ and the backwards compatibility declaration to the `Docutils Project Policies`_. * Complete the `api/`_ documentation and keep it up to date. * Generate "docutils" package API documentation from the docstrings: - Fix/enhance/add docstrings to improve the output of `pydoc`_. - Generate API documentation with Sphinx: - nicely format rST docstrings - include attribute docstrings (ignored by pydoc_). * Put the following text at a suitable place in the documentation: To find out if an object from the "docutils" package is safe to use, look up its docstring and the docstring of its parent [#]_. If there is no documentation or the documentation says "provisional" or "internal", the name, behaviour, and existence of the object is not guaranteed to be stable. Code relying on non-public objects should be made robust using public alternatives. If there is a no such alternative or the required change would be a problem, `contact the Docutils developers`__ or file a `feature request`_. .. [#] `Attribute docstrings`_ are not shown by pydoc_. To find out whether attributes have a docstring, check the source. .. _attribute docstrings: https://peps.python.org/pep-0258/#attribute-docstrings .. _pydoc: https://docs.python.org/3/library/pydoc.html __ https://docutils.sourceforge.io/docs/user/mailing-lists.html .. _feature request: https://sourceforge.net/p/docutils/feature-requests/ Rejected Ideas ============== .. Why certain ideas that were brought while discussing this proposal were not ultimately pursued. * Use type annotations as an indication of status in the public API. - There is no known precedence for this approach. - Type annotations may be helpful also for non-public code. * Use Calendar Versioning (CalVer). - Would be a break from current versioning without clear advantages. * Allow breaking API changes in *minor* versions after prior announcement and a deprecation period. - Breaks the principle of least surprise. * Enumerate all modules, classes, and functions that form the public API. * Mark all private objects with a prefix underscore. - May needlessly break applications that use "internal" objects by the current name. - Too much work. Open Issues =========== .. Any points that are still being decided/discussed. * Differentiate between "core API" and "extended API"? Cf. the `Docutils Project Policies`_ When Docutils reaches version 1.0, the major APIs will be considered frozen. The major number [...] may be incremented later if there is a major change in the design or API. * Formalise the wording for docstrings for public/private/provisional (ideally this would be a single regex pattern)? * The keyword provisional_ is well defined. ✓ * Use "private" or "internal"? * Declare only objects included in the ``__all__`` attribute of their parent objects as public rsp. explicitely list all public objects in ``__all__`` attribute of their parents? This would hide private objects from `pydoc` help on the parent objects. * Define a minimum deprecation time similar to Docbook__? E.g. * A "major" release may contain backward-incompatible changes if: * the change was announced in the release notes for the previous version (major or minor) and * the change was announced in a release that occurred at least six months previously. By these rules, Docutils developers can announce, in release 5.1, for example, its plans to make a backward-incompatible change in release 6.0. Then, in 6.0, if it’s been at least six months since 5.1 was released, they can make that change. __ https://tdg.docbook.org/tdg/5.1/ch01.html#bwcompat References ========== .. A collection of URLs used as references through the proposal. .. _api/: .. _API Reference Material for Client-Developers: ../../docutils/docs/index.html #api-reference-material-for-client-developers .. _doctree: .. _Docutils document structure: ../../docutils/docs/ref/doctree.html .. _docutils.dtd: ../../docutils/docs/ref/docutils.dtd .. _Docutils Design Specification: ../../docutils/docs/peps/pep-0258.html .. _Docutils Project Policies: ../../docutils/docs/dev/policies.html .. _"console_scripts" entry points: .. _front-end tools: ../../docutils/docs/user/tools.html .. _Docutils Publisher: ../../docutils/docs/api/publisher.html .. _Docutils Transforms: ../../docutils/docs/ref/transforms.html .. _HISTORY: ../../docutils/HISTORY.html .. _RELEASE-NOTES: ../../docutils/RELEASE-NOTES.html .. _reStructuredText specification: ../../docutils/docs/ref/rst/restructuredtext.html .. _`PEP 387: backwards compatibility rules`: https://peps.python.org/pep-0387/#backwards-compatibility-rules .. _`PEP 008: Public and Internal Interfaces`: https://peps.python.org/pep-0008/#public-and-internal-interfaces .. _provisional: https://docs.python.org/3/glossary.html#term-provisional-API .. _Semantic Versioning: https://semver.org/ Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: