[macOS] Unable to "make html" due to locale.Error: unsupported locale setting

[DelazJ]

Describe the bug

Hi,
I'm trying to build docs locally on a macOS machine with recent versions and running make html reports

Traceback (most recent call last):
File "/Users/DelazJ/Documents/gh/anotherDocBranch/venv/bin/sphinx-build", line 8, in
sys.exit(main())
File "/Users/DelazJ/Documents/gh/anotherDocBranch/venv/lib/python3.9/site-packages/sphinx/cmd/build.py", line 326, in main
locale.setlocale(locale.LC_ALL, '')
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/locale.py", line 610, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting
make: *** [html] Error 1

Since my 4.x version of Sphinx buils without issue, and trying to understand, I found that this was a macOS issue worked around by #5724 some time ago, and this patch was removed in c7d7f29 (I couldn't find the rationale but assumed because the issue on Python side is resolved?)

How to Reproduce

I think the issue relates to the OS and a Sphinx version >= 6.2.0. Any project could be used I think.

Environment Information

Platform:              darwin; (macOS-13.6-arm64-arm-64bit)
Python version:        3.9.6 (default, May  7 2023, 23:32:44) 
[Clang 14.0.3 (clang-1403.0.22.14.1)])
Python implementation: CPython
Sphinx version:        7.2.6
Docutils version:      0.18.1
Jinja2 version:        3.1.2
Pygments version:      2.16.1

Sphinx extensions

No response

Additional context

No response

[latk]

Seeing the same problem on Linux. As a workaround, running under the LC_ALL=C.UTF-8 environment variable works.

[feefladder]

I had the same problem under Linux as well while building QGIS documentation. Environment variable didn't work, but directly calling sphinx-build worked

[picnixz]

I don't know why it doesn't work and I think it would be too difficult to handle each corner case. At least it appears that there is a workaround for those on Linux. As for those on Mac, this will be hard to replicate since I don't use Mac.

Maybe your environment has some invalid locale settings or things like this? I think this issue will likely stay opened a long time because it's very hard to reproduce and I think it might be an issue with the OS itself.

[AA-Turner]

This is likely due to c7d7f29. Previously we ignored locale errors (#5724) due to an issue in CPython (python/cpython#62578 (comment)).

Please check in a Python console that import locale; locale.setlocale(locale.LC_ALL, '') succeeds.

A

[DelazJ]

Please check in a Python console that import locale; locale.setlocale(locale.LC_ALL, '') succeeds.

I get 'fr_FR.UTF-8'. TBH I don't know what to do next 😢.

[AA-Turner]

Is that running on Python 3.9? What's the output of the Python -VV?

[DelazJ]

Yes

Python 3.9.6 (default, Nov 10 2023, 13:38:27)
[Clang 15.0.0 (clang-1500.1.0.2.5)]

[feefladder]

he lol, put it in a docker with us/utf-8 locale settings :D

[MrTob]

same problem on an self-hosted ubuntu runner for github actions

[akshay-joshi]

Do you have any idea when this issue will be resolved? We have to pin the Sphinx version to 6.1.3.

sphinx-build -W -b epub -d _build/doctrees   . _build/epub
 Traceback (most recent call last):
   File "/home/jenkins/workspace/pgadmin4-epub-snapshot/venv/bin/sphinx-build", line 8, in <module>
     sys.exit(main())
              ^^^^^^
   File "/home/jenkins/workspace/pgadmin4-epub-snapshot/venv/lib/python3.11/site-packages/sphinx/cmd/build.py", line 369, in main
     locale.setlocale(locale.LC_ALL, '')
   File "/usr/lib/python3.11/locale.py", line 626, in setlocale
     return _setlocale(category, locale)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 locale.Error: unsupported locale setting

I have tried all the workarounds above; nothing works on our build machines.

[thiagocrepaldi]

import locale; locale.setlocale(locale.LC_ALL, '')

Python fails with en_US.UTF-8`

LC_ALL="en_US.UTF-8" python -c "import locale; locale.setlocale(locale.LC_ALL, '')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/thiagofc/miniconda/envs/dev311/lib/python3.11/locale.py", line 627, in setlocale
    return _setlocale(category, locale)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
locale.Error: unsupported locale setting

but LC_ALL="C.UTF-8" python -c "import locale; locale.setlocale(locale.LC_ALL, '')" succeeds

[akshay-joshi]

Do you have any idea when this issue will be resolved? We have to pin the Sphinx version to 6.1.3.

sphinx-build -W -b epub -d _build/doctrees   . _build/epub
 Traceback (most recent call last):
   File "/home/jenkins/workspace/pgadmin4-epub-snapshot/venv/bin/sphinx-build", line 8, in <module>
     sys.exit(main())
              ^^^^^^
   File "/home/jenkins/workspace/pgadmin4-epub-snapshot/venv/lib/python3.11/site-packages/sphinx/cmd/build.py", line 369, in main
     locale.setlocale(locale.LC_ALL, '')
   File "/usr/lib/python3.11/locale.py", line 626, in setlocale
     return _setlocale(category, locale)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 locale.Error: unsupported locale setting

I have tried all the workarounds above; nothing works on our build machines.

Is there any update on this?

[AA-Turner]

Is there any update on this?

I have no access to macOS hardware, so it is hard to fix this myself. This is an open-source project so we don't have funds to get a Mac, and all development is done by volunteers. I use Windows and I believe most other regular contributors use some variant of Linux. I am happy to review and accept a PR to fix this, though.

A

[AA-Turner]

I have added macOS to the GitHub Actions CI in #13284, but no locale.Error was raised. It would be helpful if someone could add a test or adjustment to the GHA settings that demonstrates this locale error.

A

[DelazJ]

@AA-Turner I don't know if this is the kind of information you are looking for or if even helpful, but I made some tests here.
We usually run "make html" and "make doctest" as GH actions.

1. Next to our Ubuntu-hosted actions, I added a macos-based one, just to build the HTML (Testing HTML build with macOS DelazJ/QGIS-Documentation#103). As you can see, the macos build failed as it did in local build.

2. Some time ago, we added export LC_ALL=C.UTF-8 to our Makefile in order to have the build pass.
   i. Removing it, the html build fails on local macOS and GH Linux but it passes on GH macOS and the "make doctest" passes on GH Ubuntu (Remove export LC_ALL=C.UTF-8 from Makefile DelazJ/QGIS-Documentation#106)
   ii. Switching that call to export LC_ALL=en_US.UTF-8, I could run successfully "make html" locally on a macOS machine and on the GH macOS and Ubuntu. But "make doctest" however failed (Update LC_ALL to en_US.UTF-8 in Makefile DelazJ/QGIS-Documentation#105)
   iii. Switching that call to export LC_ALL=C, "make html" successfully passed locally and on the GH macOS and Ubuntu and the "make doctest" also passed (Update LC_ALL to C in Makefile DelazJ/QGIS-Documentation#104). 🥇

I'm not familiar with all these and don't know what switching to C only implies but here are what I came to.

Hope that it sheds some light.

[commiterate]

Current and available locales can be listed on Unix-like systems with locale and locale --all-locales/locale -a. Here's the output on the current GitHub-managed Ubuntu 24.04 runners.

locale

locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=C.UTF-8
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=

locale --all-locales/locale -a

locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_COLLATE to default locale: No such file or directory
C
POSIX
en_US.utf8

There seems to be a bug in the Ubuntu image where all locales are set to C.UTF-8 but the locale isn't actually available.

For Sphinx, users currently need to set LC_ALL to one of the locales listed by locale --all-locales/locale -a. For example:

LC_ALL=en_US.utf8 sphinx-build ...

The C.* locales are the POSIX default locales.

https://www.gnu.org/software/libc/manual/html_node/Standard-Locales.html

The locale choice affects how locale-specific text is rendered (e.g. dates).

https://wiki.archlinux.org/title/Locale#Variables

https://man.archlinux.org/man/locale.7