Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trio fails under gevent with NotImplementedError: unsupported platform #3013

Closed
pg1671 opened this issue Jun 9, 2024 · 16 comments
Closed

Trio fails under gevent with NotImplementedError: unsupported platform #3013

pg1671 opened this issue Jun 9, 2024 · 16 comments

Comments

@pg1671
Copy link

pg1671 commented Jun 9, 2024

Trio is installed as a dependancy of httpx.

If I run the code:

from gevent.monkey import patch_all; patch_all()
import gevent
import httpx

def worker(n):
     print(n)

if __name__ == '__main__':
    jobs = [gevent.spawn(worker, job_no) for job_no in range(10)]
    gevent.joinall(jobs)

I get:

Traceback (most recent call last):
  File "/tests/gevent_test.py", line 3, in
    import httpx
  File "/work/env3.11/lib/python3.11/site-packages/httpx/init.py", line 2, in
    from ._api import delete, get, head, options, patch, post, put, request, stream
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_api.py", line 6, in
    from ._client import Client
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_client.py", line 32, in
    from ._transports.default import AsyncHTTPTransport, HTTPTransport
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 32, in
    import httpcore
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/init.py", line 1, in
    from ._api import request, stream
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_api.py", line 5, in
    from ._sync.connection_pool import ConnectionPool
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_sync/init.py", line 1, in
    from .connection import HTTPConnection
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 12, in
    from .._synchronization import Lock
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_synchronization.py", line 11, in
    import trio
  File "/work/env3.11/lib/python3.11/site-packages/trio/init.py", line 23, in
    from ._core import TASK_STATUS_IGNORED as TASK_STATUS_IGNORED # isort: split
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/init.py", line 21, in
    from ._local import RunVar, RunVarToken
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/_local.py", line 9, in
    from . import _run
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/_run.py", line 2840, in
    raise NotImplementedError("unsupported platform")
NotImplementedError: unsupported platform
@A5rocks
Copy link
Contributor

A5rocks commented Jun 10, 2024

Hi, what trio version are you using? (Just to double check, though I haven't tried to reproduce yet)

@pg1671
Copy link
Author

pg1671 commented Jun 10, 2024

Hi .. it was the latest 0.25.1.

@A5rocks
Copy link
Contributor

A5rocks commented Jun 10, 2024

While I'm not sure this will fix it, maybe try pip install git+https://github.com/python-trio/trio? #2928 was merged since v0.25.1 was released

@pg1671
Copy link
Author

pg1671 commented Jun 10, 2024

Hi .. it doesn't appear to fix it in the example code above. Still getting the same error.

@pg1671
Copy link
Author

pg1671 commented Jun 10, 2024

I tried the same commands as the top of #2928 and got the error as well.

I am using Python 3.11.

@CoolCat467
Copy link
Member

What operating system are you using?
If you can post the results of

python3.11 -m platform

that would be helpful.

@A5rocks
Copy link
Contributor

A5rocks commented Jun 10, 2024

We should probably include that in the assert message tbh.

I'm also curious about your platform because after pip install trio httpx gevent in a venv on Windows 11, I get this: (notice: no errors)

>>> from gevent.monkey import patch_all; patch_all()
True
>>> import gevent
>>> import httpx
>>>
>>> def worker(n):
...      print(n)
...
>>> if __name__ == '__main__':
...     jobs = [gevent.spawn(worker, job_no) for job_no in range(10)]
...     gevent.joinall(jobs)
...
0
1
2
3
4
5
6
7
8
9
[<Greenlet at 0x16460a8bce0: _run>, <Greenlet at 0x16460acbe20: _run>, <Greenlet at 0x16460acbec0: _run>, <Greenlet at 0x16460acbba0: _run>, <Greenlet at 0x16460acbf60: _run>, <Greenlet at 0x16460adc040: _run>, <Greenlet at 0x16460adc0e0: _run>, <Greenlet at 0x16460adc180: _run>, <Greenlet at 0x16460adc220: _run>, <Greenlet at 0x16460adc2c0: _run>]

... I assume some sort of Linux? Looking at the code, windows uses a set of primitives always, rather than trying to find something that works.

I think your error totally makes sense and we can probably figure something out (detecting gevent monkey patching?), I'm just surprised this hasn't happened before as far as I know. As a short term fix, when I pip install httpx it doesn't look like it pulls in trio, so I don't think that's a required dependency. Maybe your application will work after uninstalling trio to force httpx to use something else?

@CoolCat467
Copy link
Member

Because of the way the logic works, the system this is failing on is not windows, not linux, not epoll-based, and not kqueue-based. Given the what the select docs say, I predict the target system is Solaris or a Solaris derivative, which uses devpoll for polling.

@CoolCat467
Copy link
Member

Might not actually go anywhere because of issues with getting a solaris runner in CI, but created #3014 for attempting to add solaris support

@njsmith
Copy link
Member

njsmith commented Jun 10, 2024 via email

@pg1671
Copy link
Author

pg1671 commented Jun 10, 2024

Hi, the platform is ubuntu 22.04.

Here is the output from python3.11 -m platform:

Linux-6.5.0-1018-aws-x86_64-with-glibc2.35

@danpe
Copy link

danpe commented Oct 30, 2024

Same issue here on

trio==0.26.2
trio-websocket==0.11.1
Python 3.12.4

@CoolCat467
Copy link
Member

@danpe Could you share the list of all packages installed with

python -m pip freeze

?

@danpe
Copy link

danpe commented Oct 31, 2024

@danpe Could you share the list of all packages installed with

python -m pip freeze

?

aiofiles==24.1.0
aiohappyeyeballs==2.3.5
aiohttp @ file:///private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_4dbk4qcpk1/croot/aiohttp_1725527761629/work
aiosignal @ file:///home/conda/feedstock_root/build_artifacts/aiosignal_1667935791922/work
alembic==1.13.2
annotated-types==0.7.0
anthropic==0.34.2
anyio==4.4.0
appdirs==1.4.4
asgiref==3.8.1
asttokens==2.4.1
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1722977137225/work
auth0-python==4.7.2
babel==2.16.0
backoff==2.2.1
bcrypt==4.2.0
beautifulsoup4==4.12.3
bidict==0.23.1
blinker==1.8.2
boto3==1.34.158
botocore==1.34.158
build==1.2.1
CacheControl==0.14.0
cachetools==5.4.0
certifi==2024.7.4
cffi==1.17.1
charset-normalizer==3.3.2
chroma-hnswlib==0.7.3
chromadb==0.4.24
cleo==2.1.0
click @ file:///home/conda/feedstock_root/build_artifacts/click_1692311806742/work
cohere==5.8.0
colorama @ file:///home/conda/feedstock_root/build_artifacts/colorama_1666700638685/work
coloredlogs==15.0.1
contourpy==1.3.0
crashtest==0.4.1
crewai==0.70.1
crewai-tools==0.0.1
cryptography==43.0.1
cycler==0.12.1
dataclasses-json==0.6.7
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.14
deprecation==2.1.0
distlib==0.3.8
distro==1.9.0
dnspython==2.7.0
docker==7.1.0
docstring_parser==0.16
docx==0.2.4
docx2txt==0.8
docxcompose==1.4.0
docxtpl==0.18.0
dulwich==0.21.7
embedchain==0.1.122
eventlet==0.37.0
executing==2.1.0
fastapi==0.112.0
fastavro==1.9.5
fastjsonschema @ file:///home/conda/feedstock_root/build_artifacts/python-fastjsonschema_1718477020893/work/dist
filelock==3.15.4
Flask==3.0.3
Flask-Cors==5.0.0
Flask-SocketIO==5.4.1
flatbuffers==24.3.25
fonttools==4.54.1
frozenlist @ file:///Users/builder/cbouss/perseverance-python-buildout/croot/frozenlist_1699254257028/work
fsspec==2024.6.1
gevent==24.10.3
gevent-websocket==0.10.1
google-ai-generativelanguage==0.6.10
google-api-core==2.19.1
google-api-python-client==2.140.0
google-auth==2.33.0
google-auth-httplib2==0.2.0
google-auth-oauthlib==1.2.1
google-cloud-aiplatform==1.61.0
google-cloud-bigquery==3.25.0
google-cloud-core==2.4.1
google-cloud-monitoring==2.22.2
google-cloud-resource-manager==1.12.5
google-cloud-storage==2.18.2
google-crc32c==1.5.0
google-generativeai==0.8.3
google-resumable-media==2.7.2
googleapis-common-protos==1.63.2
gpt4all==2.8.2
gptcache==0.1.44
greenlet==3.1.1
grpc-google-iam-v1==0.13.1
grpcio==1.65.4
grpcio-status==1.62.3
grpcio-tools==1.62.3
h11==0.14.0
h2==4.1.0
hpack==4.0.0
httpcore==1.0.5
httplib2==0.22.0
httptools==0.6.1
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.24.5
humanfriendly==10.0
hyperframe==6.0.1
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1726459485162/work
importlib_metadata==8.0.0
importlib_resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1725921340658/work
imutils==0.5.4
iniconfig==2.0.0
installer==0.7.0
instructor==1.3.3
ipython==8.28.0
itsdangerous==2.2.0
jaraco.classes==3.4.0
jedi==0.19.1
Jinja2==3.1.4
jiter==0.4.2
jmespath==1.0.1
joblib==1.4.2
json5==0.9.25
json_repair==0.25.3
jsonpatch==1.33
jsonpickle==3.3.0
jsonpointer==3.0.0
jsonref==1.1.0
jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema_1720529478715/work
jsonschema-specifications @ file:///tmp/tmpkv1z7p57/src
jupyter_core @ file:///home/conda/feedstock_root/build_artifacts/jupyter_core_1727163409502/work
keyring==24.3.1
kiwisolver==1.4.7
kubernetes==30.1.0
lancedb==0.5.7
langchain==0.3.5
langchain-anthropic==0.2.3
langchain-cohere==0.1.9
langchain-community==0.3.3
langchain-core==0.3.13
langchain-experimental==0.0.65
langchain-google-genai==2.0.1
langchain-openai==0.2.2
langchain-text-splitters==0.3.1
langdetect==1.0.9
langfuse==2.51.2
langsmith==0.1.129
litellm==1.48.7
llvmlite==0.43.0
lxml==5.3.0
mailjet-rest==1.3.4
Mako==1.3.5
markdown-it-py==3.0.0
MarkupSafe==2.1.5
marshmallow==3.21.3
matplotlib==3.9.2
matplotlib-inline==0.1.7
mdurl==0.1.2
mem0ai==0.1.17
mmh3==4.1.0
monotonic==1.6
more-itertools==10.5.0
mpmath==1.3.0
msgpack==1.1.0
multidict @ file:///private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_10voz9m15i/croot/multidict_1701096890858/work
mypy-extensions==1.0.0
nbformat @ file:///home/conda/feedstock_root/build_artifacts/nbformat_1712238998817/work
neo4j==5.25.0
networkx==3.3
nodeenv==1.9.1
numba==0.60.0
numpy==1.26.4
oauthlib==3.2.2
onnxruntime==1.18.1
openai==1.50.2
opencv-contrib-python==4.10.0.84
opencv-python==4.10.0.84
opentelemetry-api==1.26.0
opentelemetry-exporter-otlp-proto-common==1.26.0
opentelemetry-exporter-otlp-proto-grpc==1.26.0
opentelemetry-exporter-otlp-proto-http==1.26.0
opentelemetry-instrumentation==0.47b0
opentelemetry-instrumentation-asgi==0.47b0
opentelemetry-instrumentation-fastapi==0.47b0
opentelemetry-proto==1.26.0
opentelemetry-sdk==1.26.0
opentelemetry-semantic-conventions==0.47b0
opentelemetry-util-http==0.47b0
orjson==3.10.7
outcome==1.3.0.post0
overrides==7.7.0
packaging==24.1
pandas==2.2.2
parameterized==0.9.0
parso==0.8.4
pdf2image==1.17.0
pexpect==4.9.0
pigar @ file:///home/conda/feedstock_root/build_artifacts/pigar_1721171135631/work
pillow==10.4.0
pkginfo==1.11.1
pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1694617248815/work
platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1726613481435/work
pluggy==1.5.0
poetry==1.8.3
poetry-core==1.9.0
poetry-plugin-export==1.8.0
portalocker==2.10.1
posthog==3.5.0
prompt_toolkit==3.0.48
proto-plus==1.24.0
protobuf==4.25.4
ptyprocess==0.7.0
pulsar-client==3.5.0
pure_eval==0.2.3
py==1.11.0
pyarrow==17.0.0
pyasn1==0.6.0
pyasn1_modules==0.4.0
pycparser==2.22
pydantic==2.8.2
pydantic-settings==2.5.2
pydantic_core==2.20.1
Pygments==2.18.0
PyJWT==2.9.0
pylance==0.9.18
PyMuPDF==1.24.12
pyparsing==3.1.2
pypdf==4.3.1
PyPDF2==3.0.1
PyPika==0.48.9
pyproject_hooks==1.1.0
pyright==1.1.375
pysbd==0.3.4
PySocks==1.7.1
pytesseract==0.3.13
pytest==8.3.2
python-dateutil==2.9.0.post0
python-docx==1.1.2
python-dotenv==1.0.1
python-engineio==4.10.1
python-socketio==5.11.4
pytube==15.0.0
pytz==2024.1
pyvis==0.3.2
PyYAML==6.0.2
qdrant-client==1.10.1
rank-bm25==0.2.2
RapidFuzz==3.10.0
ratelimiter==1.2.0.post0
referencing @ file:///home/conda/feedstock_root/build_artifacts/referencing_1714619483868/work
regex==2024.9.11
requests==2.32.3
requests-oauthlib==2.0.0
requests-toolbelt==1.0.0
retry==0.9.2
rich==13.7.1
rpds-py @ file:///Users/builder/cbouss/perseverance-python-buildout/croot/rpds-py_1699262599461/work
rsa==4.9
s3transfer==0.10.2
safetensors==0.4.5
schema==0.7.7
scikit-learn==1.5.2
scipy==1.14.1
selenium==4.23.1
semver==3.0.2
sentence-transformers==3.1.1
setuptools==72.1.0
shapely==2.0.5
shellingham==1.5.4
simple-websocket==1.1.0
six==1.16.0
sniffio==1.3.1
sortedcontainers==2.4.0
soupsieve==2.5
SQLAlchemy==2.0.32
stack-data==0.6.3
starlette==0.37.2
sympy==1.13.2
tabulate==0.9.0
temporalio==1.8.0
tenacity==8.5.0
threadpoolctl==3.5.0
tiktoken==0.7.0
tokenizers==0.20.0
tomlkit==0.13.2
torch==2.4.1
tqdm==4.66.5
traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1713535121073/work
transformers==4.45.1
trio==0.27.0
trio-websocket==0.11.1
trove-classifiers==2024.9.12
typer==0.12.3
types-protobuf==5.28.0.20240924
types-requests==2.32.0.20240712
typing-inspect==0.9.0
typing_extensions==4.12.2
tzdata==2024.1
uritemplate==4.1.1
urllib3==2.2.2
uvicorn==0.30.5
uvloop==0.19.0
virtualenv==20.26.6
watchfiles==0.23.0
wcwidth==0.2.13
websocket-client==1.8.0
websockets==12.0
Werkzeug==3.0.3
wheel==0.43.0
wrapt==1.16.0
wsproto==1.2.0
xattr==1.1.0
yarl @ file:///private/var/folders/k1/30mswbxs7r1g6zwn8y4fyt500000gp/T/abs_9et2w8mlgu/croot/yarl_1725976501637/work
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1726248574750/work
zope.event==5.0
zope.interface==7.1.1

@A5rocks
Copy link
Contributor

A5rocks commented Oct 31, 2024

If you update httpcore to be the recently released version 1.0.6, httpx handles this transparently. This is ultimately because trio and gevent monkey patching are incompatible -- we've decided that error should be raised up front rather than let the user run into it later.

Essentially the only thing that we could do is delay that error with monkeypatching, but we have tried PRs in that direction and it's a game of whack-a-mole. Since httpx handles this and that seems to be the main context where this happens, I'm going to be closing this issue. There's #3087 for improving this error message.

Please reopen this issue if httpcore 1.0.6 doesn't actually fix this.

@A5rocks A5rocks closed this as completed Oct 31, 2024
@A5rocks
Copy link
Contributor

A5rocks commented Oct 31, 2024

Same issue here on

trio==0.26.2
trio-websocket==0.11.1
Python 3.12.4

@danpe

Cause this isn't exactly using httpx, I'm confused on why you're using trio-websocket in an environment where gevent has monkeypatched things away. Is there a reason you're expecting this to work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants