0.11.12: pytest fails in src/ablog/tests/test_parallel.py::test_not_safe_for_parallel_read unit

[kloczek]

Describe the bug

Looks like pyta=est fails in src/ablog/tests/test_parallel.py::test_not_safe_for_parallel_read unit.

To Reproduce

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using installer module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Screenshots

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-ablog-0.11.12-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-ablog-0.11.12-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0
libraries: Sphinx-8.1.3, docutils-0.21.2
base tempdir: /tmp/pytest-of-tkloczko/pytest-50
rootdir: /home/tkloczko/rpmbuild/BUILD/ablog-0.11.12
configfile: setup.cfg
collected 10 items

src/ablog/tests/test_build.py ..                                                                                                                                                      [ 20%]
src/ablog/tests/test_canonical.py .                                                                                                                                                   [ 30%]
src/ablog/tests/test_external.py .                                                                                                                                                    [ 40%]
src/ablog/tests/test_parallel.py F                                                                                                                                                    [ 50%]
src/ablog/tests/test_postlist.py ..                                                                                                                                                   [ 70%]
src/ablog/tests/test_templates.py ...                                                                                                                                                 [100%]

========================================================================================= FAILURES ==========================================================================================
______________________________________________________________________________ test_not_safe_for_parallel_read ______________________________________________________________________________

rootdir = PosixPath('/home/tkloczko/rpmbuild/BUILD/ablog-0.11.12/roots'), tmp_path = PosixPath('/tmp/pytest-of-tkloczko/pytest-50/test_not_safe_for_parallel_rea0')

    @pytest.mark.xfail("win" in sys.platform, reason="Passes on Windows")
    def test_not_safe_for_parallel_read(rootdir: Path, tmp_path: Path):
        """
        Ablog is NOT safe for parallel read.

        In such case, it doesn't collect any posts.
        """
        # https://github.com/sunpy/ablog/issues/297
        # Very ugly hack to change the parallel_read_safe value to True
        good_read_safe = '"parallel_read_safe": False'
        bad_read_safe = '"parallel_read_safe": True'
        init_py_path = Path(__file__).parent.parent / "__init__.py"
        assert good_read_safe in init_py_path.read_text(encoding="utf-8")
        bad_init_py = init_py_path.read_text().replace(good_read_safe, bad_read_safe)
        init_py_path.write_text(bad_init_py, encoding="utf-8")

        # liborjelinek: I wasn't able to demonstrate the issue with the `parallel` argument to the `sphinx` fixture
        # @pytest.mark.sphinx("html", testroot="parallel", parallel=2)
        # therefore running sphinx-build externally
        indir = rootdir / "test-parallel"
>       run(["sphinx-build", "-b", "html", indir.as_posix(), tmp_path.as_posix(), "-j", "auto"], check=True)

src/ablog/tests/test_parallel.py:27:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['sphinx-build', '-b', 'html', '/home/tkloczko/rpmbuild/BUILD/ablog-0.11.12/roots/test-parallel', '/tmp/pytest-of-tkloczko/pytest-50/test_not_safe_for_parallel_rea0', '-j', ...],)
kwargs = {}, process = <Popen: returncode: 2 args: ['sphinx-build', '-b', 'html', '/home/tkloczko/r...>, stdout = None, stderr = None, retcode = 2

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them,
        or pass capture_output=True to capture both.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['sphinx-build', '-b', 'html', '/home/tkloczko/rpmbuild/BUILD/ablog-0.11.12/roots/test-parallel', '/tmp/pytest-of-tkloczko/pytest-50/test_not_safe_for_parallel_rea0', '-j', 'auto']' returned non-zero exit status 2.

/usr/lib64/python3.10/subprocess.py:526: CalledProcessError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
Running Sphinx v8.1.3
loading translations [en]... done
building [mo]: targets for 0 po files that are out of date
writing output...
building [html]: targets for 6 source files that are out of date
updating environment: [new config] 6 added, 0 changed, 0 removed
reading sources... [ 17%] index
reading sources... [ 33%] post1
reading sources... [ 50%] post2
reading sources... [ 67%] post3
reading sources... [ 83%] post4
reading sources... [100%] postlist

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
copying assets...
copying static files...
Writing evaluated template result to /tmp/pytest-of-tkloczko/pytest-50/test_not_safe_for_parallel_rea0/_static/basic.css
Writing evaluated template result to /tmp/pytest-of-tkloczko/pytest-50/test_not_safe_for_parallel_rea0/_static/documentation_options.js
Writing evaluated template result to /tmp/pytest-of-tkloczko/pytest-50/test_not_safe_for_parallel_rea0/_static/language_data.js
Writing evaluated template result to /tmp/pytest-of-tkloczko/pytest-50/test_not_safe_for_parallel_rea0/_static/alabaster.css
copying static files: done
copying extra files...
copying extra files: done
copying assets: done
writing output... [ 20%] post1
writing output... [ 40%] post2
writing output... [ 60%] post3
writing output... [ 80%] post4
writing output... [100%] postlist

generating indices... genindex done
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
WARNING: html_static_path entry '/home/tkloczko/rpmbuild/BUILDROOT/python-ablog-0.11.12-2.fc37.x86_64/usr/lib/python3.10/site-packages/ablog/stylesheets' does not exist
WARNING: the ablog extension is not safe for parallel reading
WARNING: doing serial read

Theme error:
An error happened in rendering the page blog/archive.
Reason: TemplateNotFound("'ablog/catalog.html' not found in ['/usr/lib/python3.10/site-packages/alabaster', '/usr/lib/python3.10/site-packages/sphinx/themes/basic']")
===================================================================================== warnings summary ======================================================================================
../../../../../usr/lib/python3.10/site-packages/_pytest/config/__init__.py:1506
  /usr/lib/python3.10/site-packages/_pytest/config/__init__.py:1506: PytestConfigWarning: No files were found in testpaths; consider removing or adjusting your testpaths configuration. Searching recursively from the current directory instead.
    self.args, self.args_source = self._decide_args(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== short test summary info ==================================================================================
FAILED src/ablog/tests/test_parallel.py::test_not_safe_for_parallel_read - subprocess.CalledProcessError: Command '['sphinx-build', '-b', 'html', '/home/tkloczko/rpmbuild/BUILD/ablog-0.11.12/roots/test-parallel', '/tmp/pytest-of-tkloczko/pytest-50/test_not_sa...
========================================================================== 1 failed, 9 passed, 1 warning in 3.03s ===========================================================================

Please let me know if you need more details or want me to perform some diagnostics.

System Details

List of installed modules in build env:

Package                       Version
----------------------------- -----------
alabaster                     0.7.16
babel                         2.16.0
build                         1.2.2.post1
charset-normalizer            3.4.0
defusedxml                    0.7.1
distro                        1.9.0
docutils                      0.21.2
exceptiongroup                1.1.3
feedgen                       1.0.0
imagesize                     1.4.1
importlib_metadata            8.5.0
iniconfig                     2.0.0
installer                     0.7.0
invoke                        2.2.0
Jinja2                        3.1.4
lexicon                       2.0.1
lxml                          5.2.2
markdown-it-py                3.0.0
MarkupSafe                    3.0.2
mdit-py-plugins               0.4.2
mdurl                         0.1.2
myst-parser                   4.0.0
packaging                     24.0
pluggy                        1.5.0
Pygments                      2.18.0
pyproject_hooks               1.2.0
pytest                        8.2.2
python-dateutil               2.9.0.post0
PyYAML                        6.0.2
requests                      2.32.3
setuptools                    75.1.0
setuptools-scm                8.1.0
snowballstemmer               2.2.0
Sphinx                        8.1.3
sphinx-automodapi             0.18.0
sphinxcontrib-applehelp       2.0.0
sphinxcontrib-devhelp         1.0.6
sphinxcontrib-htmlhelp        2.1.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          2.0.0
sphinxcontrib-serializinghtml 2.0.0
tokenize_rt                   6.1.0
tomli                         2.0.1
urllib3                       2.2.2
watchdog                      5.0.3
wheel                         0.45.0
zipp                          3.21.0

Installation method

from source (.tar.gz), git checkout

[nabobalis]

Thanks for the report! For now, it's safe to skip the test and I can try to debug and fix this to avoid you having to deal it in the future.