#270 `python setup.py check --restructuredtext --strict --metadata` fails with `warning: check: Could not finish the parsing.` if the RST document uses `code` or `code-block` directives.
python setup.py check --restructuredtext --strict --metadata fails with:
warning: check: Could not finish the parsing.
if the RST document uses code or code-block directives.
This is annoying because the document is valid, but it appears to be invalid
and confuses people. For example, see
https://github.com/ionelmc/pytest-benchmark/pull/4#issuecomment-66940307
How to reproduce this bug
Clone a repo that has a README.rst with code-block directives in it. E.g.:
$ git clone git@github.com:ionelmc/pytest-benchmark.git
$ cd pytest-benchmark
$ git checkout ab0b08f6fccb06a7909905a8409f8faa8b01e0d8
Observe that it has "code-blocks" in it:
$ grep 'code-block' README.rst
.. code-block:: python
.. code-block:: python
Observe that RST document is valid, according to rst2html.py:
$ rst2html.py --halt=1 README.rst > README.html && echo "RST was OK."
RST was OK.
$ head -n 3 README.html
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
Observe that python setup.py check --restructuredtext --strict --metadata fails:
$ python setup.py check --restructuredtext --strict --metadata
running check
warning: check: Could not finish the parsing.
error: Please correct your package.
$ echo $?
1
What was expected: python setup.py check --restructuredtext --strict
--metadata should succeed with no warnings, just as rst2html.py did, because
README.rst is a valid RST document.
What actually happened: python setup.py check --restructuredtext --strict
--metadata prints a warning and an error and fails, unlike rst2html.py
The error is coming from here:
https://github.com/python/cpython/blob/master/Lib/distutils/command/check.py#L142
It's happening because of this line:
https://github.com/python/cpython/blob/master/Lib/distutils/command/check.py#L125
settings = frontend.OptionParser().get_default_values()
If this is changed to:
settings = frontend.OptionParser(components=(Parser,)).get_default_values()
then things work much better (this is how tools/quicktest.py does it for example)
so this might actually be a bug in distutils, but changing CPython (and maybe PyPy) is a more laborious and slow process, so I'm thinking it might be nice to work around this in docutils, if isn't too much of a burden. Docutils is on a much shorter release cycle than CPython, so changing docutils is a way to get this fix out to users much faster.
Here's a patch to docutils that solves the problem for me:
diff --git a/docutils/frontend.py.2014-12-15-070721 b/docutils/frontend.py
index f837c62..7427e15 100644
--- a/docutils/frontend.py.2014-12-15-070721
+++ b/docutils/frontend.py
@@ -41,6 +41,7 @@ import docutils
import docutils.utils
import docutils.nodes
from docutils.utils.error_reporting import locale_encoding, ErrorOutput, ErrorString
+from docutils.parsers import Parser
def store_multiple(option, opt, value, parser, *args, **kwargs):
@@ -561,7 +562,20 @@ class OptionParser(optparse.OptionParser, docutils.SettingsSpec):
self.version = self.version_template
# Make an instance copy (it will be modified):
self.relative_path_settings = list(self.relative_path_settings)
+ from docutils.parsers.rst import Parser
self.components = (self,) + tuple(components)
+
+ # distutils.command.check does:
+ # settings = frontend.OptionParser().get_default_values()
+ # and as a result is missing important defaults like
+ # `syntax_highlight`, causing
+ # `python setup.py check --restructuredtext --strict --metadata` to
+ # fail with `warning: check: Could not finish the parsing.` if the RST
+ # document uses `code` or `code-block` directives.
+ if len(self.components) == 1:
+ from docutils.parsers.rst import Parser
+ self.components += (Parser,)
+
self.populate_from_components(self.components)
self.set_defaults_from_dict(defaults or {})
if read_config_files and not self.defaults['_disable_config']:
Attaching
bug_270.patch, a patch to docutils that fixes this issue ...Last edit: marcman 2014-12-15
After some discussion on the mailing list with David Goodger, I am revising this a bit. He wanted it to check that it was being called by docutils before messing with anything.
I also created a Python issue (and patch) for this:
http://bugs.python.org/issue23063
but it would be nice still to add a workaround to docutils, so that folks don't need Python 3.5 for this to work.
I also created a branch and pull request on GitHub for this, in case folks find that easier to read and pull from:
https://github.com/msabramo/docutils/pull/1
It seems like this is fixed upstream now (no need to wait for 3.5):
http://bugs.python.org/msg234051
Thank you for the report(s), analysis, patch, and patience.