Language Module Names

Language modules are named using a case-insensitive language identifier as defined in RFC 1766, converting hyphens to underscores [2]. A typical language identifier consists of a 2-letter language code from ISO 639 (3-letter codes can be used if no 2-letter code exists; RFC 1766 is currently being revised to allow 3-letter codes). The language identifier can have an optional subtag, typically for variations based on country (from ISO 3166 2-letter country codes). If no language identifier is specified, the default is "en" for English. Examples of module names include en.py, fr.py, ja.py, and pt_br.py.

Python Code

All Python code in Docutils will be ASCII-only. In language modules, Unicode-escapes will have to be used for non-ASCII characters. Although it may be possible for some developers to store non-ASCII characters directly in strings, it will cause problems for other developers whose locales are set up differently.

PEP 263 introduces source code encodings to Python modules, implemented beginning in Python 2.3. Until PEP 263 is fully implemented as a well-established convention, proven robust in daily use, and the tools (editors, CVS, email, etc.) recognize this convention, Docutils shall remain conservative.

As mentioned in the note above, developers are invited to explore "gettext" and other i18n technologies.

Docutils Language Module

Modules in docutils/languages contain language mappings for markup-independent language-specific features of Docutils. To make a new language module, just copy the en.py file, rename it with the code for your language (see Language Module Names above), and translate the terms as described below.

Each Docutils language module contains three module attributes:

labels

This is a mapping of node class names to language-dependent boilerplate label text. The label text is used by Writer components when they encounter document tree elements whose class names are the mapping keys.

The entry values (not the keys) should be translated to the target language.

bibliographic_fields

This is a mapping of language-dependent field names (converted to lower case) to canonical field names (keys of DocInfo.biblio_notes in docutils.transforms.frontmatter). It is used when transforming bibliographic fields.

The keys should be translated to the target language.

author_separators

This is a list of strings used to parse the 'Authors' bibliographic field. They separate individual authors' names, and are tried in order (i.e., earlier items take priority, and the first item that matches wins). The English-language module defines them as [';', ',']; semi-colons can be used to separate names like "Arthur Pewtie, Esq.".

Most languages won't have to "translate" this list.

reStructuredText Language Module

Modules in docutils/parsers/rst/languages contain language mappings for language-specific features of the reStructuredText parser. To make a new language module, just copy the en.py file, rename it with the code for your language (see Language Module Names above), and translate the terms as described below.

Each reStructuredText language module contains two module attributes:

directives

This is a mapping from language-dependent directive names to canonical directive names. The canonical directive names are registered in docutils/parsers/rst/directives/__init__.py, in _directive_registry.

The keys should be translated to the target language. Synonyms (multiple keys with the same values) are allowed; this is useful for abbreviations.

roles

This is a mapping language-dependent role names to canonical role names for interpreted text. The canonical directive names are registered in docutils/parsers/rst/states.py, in Inliner._interpreted_roles (this may change).

The keys should be translated to the target language. Synonyms (multiple keys with the same values) are allowed; this is useful for abbreviations.

Testing the Language Modules

Whenever a new language module is added or an existing one modified, the unit tests should be run. The test modules can be found in the docutils/test directory from CVS or from the latest CVS snapshot.

The test_language.py module can be run as a script. With no arguments, it will test all language modules. With one or more language codes, it will test just those languages. For example:

$ python test_language.py en
..
----------------------------------------
Ran 2 tests in 0.095s

OK

Use the "alltests.py" script to run all test modules, exhaustively testing the parser and other parts of the Docutils system.

Submitting the Language Modules

If you do not have CVS write access and want to contribute your language modules, feel free to submit them at the SourceForge patch tracker.