Emscripten support

.github/workflows/test-suite.yml

@@ -22,6 +22,23 @@ jobs:
         with:
           python-version: "${{ matrix.python-version }}"
           allow-prereleases: true
+      - uses: pyodide/pyodide-actions/install-browser@v2
+        if: ${{ matrix.python-version ==3.12 }}
+        with:
+          runner: selenium
+          browser: chrome
+          browser-version: 120
+      - uses: pyodide/pyodide-actions/install-browser@v2
+        if: ${{ matrix.python-version ==3.12 }}
+        with:
+          runner: selenium
+          browser: node
+          browser-version: 22
+      - uses: pyodide/pyodide-actions/download-pyodide@v2
+        if: ${{ matrix.python-version ==3.12 }}
+        with:
+          version: 0.26.2
+          to: pyodide_dist
       - name: "Install dependencies"
         run: "scripts/install"
       - name: "Run linting checks"

.gitignore

@@ -10,3 +10,4 @@ venv*/
 .python-version
 build/
 dist/
+pyodide_dist

CHANGELOG.md

@@ -4,13 +4,17 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+## [Unreleased]
+
+* Support for Emscripten (#3330)
+
 ## 0.28.1 (6th December, 2024)
 
 * Fix SSL case where `verify=False` together with client side certificates.
 
 ## 0.28.0 (28th November, 2024)
 
-The 0.28 release includes a limited set of deprecations.
+The 0.28 release includes a limited set of backwards incompatible changes.
 
 **Deprecations**:
 

docs/advanced/emscripten.md

@@ -0,0 +1,57 @@
+---
+template: pyodide.html
+---
+# Emscripten Support
+
+httpx has support for running on Webassembly / Emscripten using [pyodide](https://github.com/pyodide/pyodide/). 
+
+In Emscripten, all network connections are handled by the enclosing Javascript runtime. As such, there is limited control over various features. In particular:
+
+- Proxy servers are handled by the runtime, so you cannot control them.
+- httpx has no control over connection pooling.
+- Certificate handling is done by the browser, so you cannot modify it.
+- Requests are constrained by cross-origin isolation settings in the same way as any request that is originated by Javascript code.
+- On browsers, timeouts will not work in the main browser thread unless your browser supports [Javascript Promise Integration](https://github.com/WebAssembly/js-promise-integration/blob/main/proposals/js-promise-integration/Overview.md). This is currently behind a flag on chrome, and not yet supported by non-chromium browsers.
+- On node.js, synchronous requests will only work if you enable Javascript Promise Integration. You can do this using the `--experimental-wasm-stack-switching` flag when you run the node executable.
+
+## Try it in your browser
+
+Use the following live example to test httpx in your web browser. You can change the code below and hit run again to test different features or web addresses.
+
+<div id="pyodide_buttons"></div>
+
+<div id="pyodide_output">
+</div>
+
+<div id="pyodide_editor">
+import httpx
+print("Sending response using httpx in the browser:")
+print("--------------------------------------------")
+r=httpx.get("http://www.example.com")
+print("Status = ",r.status_code)
+print("Response = ",r.text[:50],"...")
+</div>
+
+
+## Build it
+Because this is a pure python module, building is the same as ever (`python -m build`), or use the built wheel from pypi.
+
+## Testing Custom Builds of httpx in Emscripten
+Once you have a wheel you can test it in your browser. You can do this using the [pyodide console](
+https://pyodide.org/en/latest/console.html), or by hosting your own web page. 
+
+1) To test in pyodide console, serve the wheel file via http (e.g. by calling python -m `http.server` in the dist directory.) Then in the [pyodide console](
+https://pyodide.org/en/latest/console.html), type the following, replacing the URL of the locally served wheel.
+
+```
+import pyodide_js as pjs
+import ssl,certifi,idna
+pjs.loadPackage("<URL_OF_THE_WHEEL>")
+import httpx
+# Now httpx should work
+```
+
+2) To test a custom built wheel in your own web page, create a page which loads the pyodide javascript (see the [instructions](https://pyodide.org/en/stable/usage/index.html) on the pyodide website), then call `pyodide.loadPackage` on your pyodide instance, pointing it at the wheel file. Then make sure you load the dependencies by loading the ssl,certifi and idna packages (which are part of pyodide - call `pyodide.loadPackage` for each one and pass it just the package name.)
+
+3) To test in node.js, make sure you have a  pyodide distribution downloaded to a known folder, then load pyodide following the instructions on the pyodide website (https://pyodide.org/en/stable/usage/index.html). You can then call await `pyodide.loadPackage('<wheel location>');` and httpx should be available as a package in pyodide. You need at version 0.26.2 or later of pyodide.
+

docs/index.md

@@ -148,3 +148,6 @@ $ pip install httpx[brotli,zstd]
 HTTPX requires Python 3.8+
 
 [sync-support]: https://github.com/encode/httpx/issues/572
+
+## Pyodide / Emscripten Support
+There is experimental support for running in Webassembly under the pyodide runtime. See the [Emscripten](advanced/emscripten.md) page for more details.

docs/overrides/pyodide.html

@@ -0,0 +1,128 @@
+{% extends "main.html" %}
+{% block styles %}
+    {{ super() }}
+    <link href="
+https://cdn.jsdelivr.net/npm/[email protected]/css/ace.min.css
+" rel="stylesheet"></link>
+    <style>
+        #pyodide_editor { 
+            width:100%;
+            height:10em;
+            margin-bottom:2em;
+
+        }
+        #pyodide_output {
+            width:100%;
+            height:20em;
+            whitespace:pre;
+            background-color:#eee; 
+            font-family: 'Courier New', monospace;
+            font-size:smaller;
+            overflow-y: auto;
+        }
+
+        #pyodide_output p {
+            margin:0;
+        }
+        .pyodide_error
+        {
+            color:#e00;
+        }
+         </style>
+{% endblock %}
+
+{% block scripts %}
+{{ super() }}
+<script src="https://cdn.jsdelivr.net/npm/[email protected]/src-noconflict/ace.min.js"> </script>
+<script src="https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"> </script>
+<script language="javascript">
+    var pyodide;
+    var loadPromise;
+    var editor = ace.edit("pyodide_editor");
+    editor.setTheme("ace/theme/monokai");
+    editor.session.setMode("ace/mode/python");
+    let outputDiv = document.getElementById("pyodide_output")
+    outputDiv.innerText="Loading python in your browser";
+
+    async function initPyodide()
+    {
+        pyodide = await loadPyodide();
+        pyodide.setStdout({batched:stdoutLine})
+        pyodide.setStderr({batched:stderrLine});
+        if(document.location.origin.startsWith("http://127.0.0.1") 
+        || document.location.origin.startsWith("http://localhost"))
+        {
+            // if we are on a development machine
+            // use the wheel from the dist folder
+            await pyodide.loadPackage("/test.whl");
+            await pyodide.loadPackage("ssl");
+            await pyodide.loadPackage("idna");            
+            await pyodide.loadPackage("certifi");            
+        }else{
+            // otherwise load it from pypi
+            // (once a pyodide supporting version is on there)
+            await pyodide.loadPackage("micropip");
+            await pyodide.runPythonAsync("await micropip.install('httpx')")
+        }
+        outputDiv.innerText="Python runtime ready...\n"
+    }
+
+    loadPromise = initPyodide();
+
+    function stdoutLine(line)
+    {
+        var content = document.createElement("p");
+        content.textContent=line;
+        outputDiv.appendChild(content);
+    }
+
+    function stderrLine(line)
+    {
+        var content = document.createElement("p");
+        content.className="pyodide_error"
+        content.textContent=line;
+
+        outputDiv.appendChild(content);
+    }
+
+
+    async function runCode()
+    {
+        await loadPromise;
+        code = editor.getValue();
+        await pyodide.loadPackagesFromImports(code);
+        try
+        {
+            await pyodide.runPythonAsync(code);
+        }catch(e)
+        {
+            stderrLine(e.message);
+        }
+    }
+
+    function clearOutput()
+    {
+        outputDiv.innerHTML="";
+    }
+
+
+    run_button=document.createElement("button");
+    run_button.className="md-button";
+    run_button.onclick=runCode;
+    run_button.innerText="Run";
+    clear_button=document.createElement("button");
+    clear_button.className="md-button";
+    clear_button.onclick=clearOutput;
+    clear_button.innerText="Clear output";
+    clear_button.style="float:right";
+    var buttonDiv=document.getElementById("pyodide_buttons");
+    buttonDiv.appendChild(run_button);
+    buttonDiv.appendChild(clear_button);
+
+    window.runCode=runCode;    
+</script>
+{% endblock %}
+
+
+
+

httpx/_api.py

@@ -3,6 +3,9 @@
 import typing
 from contextlib import contextmanager
 
+if typing.TYPE_CHECKING:
+    import ssl  # pragma: no cover
+
 from ._client import Client
 from ._config import DEFAULT_TIMEOUT_CONFIG
 from ._models import Response

httpx/_client.py

@@ -28,8 +28,12 @@
 )
 from ._models import Cookies, Headers, Request, Response
 from ._status_codes import codes
-from ._transports.base import AsyncBaseTransport, BaseTransport
-from ._transports.default import AsyncHTTPTransport, HTTPTransport
+from ._transports import (
+    AsyncBaseTransport,
+    AsyncHTTPTransport,
+    BaseTransport,
+    HTTPTransport,
+)
 from ._types import (
     AsyncByteStream,
     AuthTypes,
@@ -51,6 +55,7 @@
 if typing.TYPE_CHECKING:
     import ssl  # pragma: no cover
 
+
 __all__ = ["USE_CLIENT_DEFAULT", "AsyncClient", "Client"]
 
 # The type annotation for @classmethod and context managers here follows PEP 484

httpx/_models.py

@@ -1072,6 +1072,7 @@ async def aclose(self) -> None:
 
         if not self.is_closed:
             self.is_closed = True
+            print("ACLOSE _models")
             with request_context(request=self._request):
                 await self.stream.aclose()
 

httpx/_transports/__init__.py

@@ -1,9 +1,24 @@
+import sys
+
 from .asgi import *
 from .base import *
-from .default import *
 from .mock import *
 from .wsgi import *
 
+if sys.platform == "emscripten":  # pragma: nocover
+    # in emscripten we use javascript fetch
+    from .jsfetch import *
+
+    # override default transport names
+    HTTPTransport = JavascriptFetchTransport
+    AsyncHTTPTransport = AsyncJavascriptFetchTransport
+else:
+    # everywhere else we use httpcore
+    from .httpcore import *
+
+    HTTPTransport = HTTPCoreTransport
+    AsyncHTTPTransport = AsyncHTTPCoreTransport
+
 __all__ = [
     "ASGITransport",
     "AsyncBaseTransport",

httpx/_transports/default.py → httpx/_transports/httpcore.py

@@ -11,16 +11,16 @@
 
 # Disable HTTP/2 on a single specific domain.
 mounts = {
-    "all://": httpx.HTTPTransport(http2=True),
-    "all://*example.org": httpx.HTTPTransport()
+    "all://": httpx.HTTPCoreTransport(http2=True),
+    "all://*example.org": httpx.HTTPCoreTransport()
 }
 
 # Using advanced httpcore configuration, with connection retries.
-transport = httpx.HTTPTransport(retries=1)
+transport = httpx.HTTPCoreTransport(retries=1)
 client = httpx.Client(transport=transport)
 
 # Using advanced httpcore configuration, with unix domain sockets.
-transport = httpx.HTTPTransport(uds="socket.uds")
+transport = httpx.HTTPCoreTransport(uds="socket.uds")
 client = httpx.Client(transport=transport)
 """
 
@@ -57,16 +57,16 @@
 from .._urls import URL
 from .base import AsyncBaseTransport, BaseTransport
 
-T = typing.TypeVar("T", bound="HTTPTransport")
-A = typing.TypeVar("A", bound="AsyncHTTPTransport")
+T = typing.TypeVar("T", bound="HTTPCoreTransport")
+A = typing.TypeVar("A", bound="AsyncHTTPCoreTransport")
 
 SOCKET_OPTION = typing.Union[
     typing.Tuple[int, int, int],
     typing.Tuple[int, int, typing.Union[bytes, bytearray]],
     typing.Tuple[int, int, None, int],
 ]
 
-__all__ = ["AsyncHTTPTransport", "HTTPTransport"]
+__all__ = ["AsyncHTTPCoreTransport", "HTTPCoreTransport"]
 
 HTTPCORE_EXC_MAP: dict[type[Exception], type[httpx.HTTPError]] = {}
 
@@ -132,7 +132,7 @@ def close(self) -> None:
             self._httpcore_stream.close()
 
 
-class HTTPTransport(BaseTransport):
+class HTTPCoreTransport(BaseTransport):
     def __init__(
         self,
         verify: ssl.SSLContext | str | bool = True,
@@ -276,7 +276,7 @@ async def aclose(self) -> None:
             await self._httpcore_stream.aclose()
 
 
-class AsyncHTTPTransport(AsyncBaseTransport):
+class AsyncHTTPCoreTransport(AsyncBaseTransport):
     def __init__(
         self,
         verify: ssl.SSLContext | str | bool = True,