PEP 738: first draft

.github/CODEOWNERS

@@ -614,6 +614,7 @@ peps/pep-0733.rst  @encukou @vstinner @zooba @iritkatriel
 peps/pep-0734.rst  @ericsnowcurrently
 peps/pep-0735.rst  @brettcannon
 peps/pep-0737.rst  @vstinner
+peps/pep-0738.rst  @encukou
 # ...
 # peps/pep-0754.rst
 # ...

peps/pep-0738.rst

@@ -0,0 +1,350 @@
+PEP: 738
+Title: Adding Android as a supported platform
+Author: Malcolm Smith <[email protected]>
+Sponsor: Petr Viktorin <[email protected]>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 12-Dec-2023
+Python-Version: 3.13
+
+
+Abstract
+========
+
+This PEP proposes adding Android as a supported platform in CPython. The initial
+goal is for Android to achieve Tier 3 support in Python 3.13.
+
+This PEP is based on :pep:`730` – "Adding iOS as a supported platform" by
+Russell Keith-Magee, and covers many of the same issues. Notable differences
+between the two platforms can be found by searching for the word "iOS".
+
+
+Motivation
+==========
+
+Over the last 15 years, mobile platforms have become increasingly important
+parts of the computing landscape. Android is one of two operating systems that
+control the vast majority of these devices. However, there is no official
+support for Android in CPython.
+
+The `BeeWare <https://beeware.org>`__ and `Kivy <https://kivy.org>`__ projects
+have both supported Android for many years, and they have both been used to
+generate applications that have been accepted for publication in the Google Play
+Store. This demonstrates the technical feasibility of Android support.
+
+It is important for the future of Python as a language that it is able to be
+used on any hardware or OS that has widespread adoption. If Python cannot be
+used a on a platform that has widespread use, adoption of the language will be
+impacted as potential users will adopt other languages that *do* provide support
+for these platforms.
+
+
+Rationale
+=========
+
+Development landscape
+---------------------
+
+Android is broadly a POSIX platform, including full support for threading and
+network communication. The Android development tools are equally supported on
+Linux, macOS and Windows.
+
+Android uses a Linux kernel and the ELF binary format, but provides its own C
+library implementation called Bionic. As a result, it is generally not
+binary-compatible with any other Linux distribution, even if the architecture
+matches.
+
+Source-compatibility with Linux is much better. In the early years of Android,
+the C library was very incomplete, but most of the gaps were filled by around
+2014. Since then, any C code that compiles for Linux can usually be compiled for
+Android, unless it involves direct access to hardware devices or operating
+system services.
+
+This is also true of CPython. Although it has never officially supported
+Android, recent versions (since 3.6) can already be compiled for Android with
+minimal patching.
+
+
+Version numbering
+-----------------
+
+Each Android version can be identified in three ways:
+
+* A conventional version number
+* A sequential integer "API level" (the most common form in developer
+  documentation)
+* An alphabetic confectionery-themed code name
+
+There is no consistent pattern to link one of these to another; they must be
+looked up in `a table <https://en.wikipedia.org/wiki/Android_version_history>`__.
+
+For Python 3.13 we propose the minimum Android version to be 6.0 (API level 23).
+This would support `98% of active devices
+<https://dl.google.com/android/studio/metadata/distributions.json>`__, and would
+allow us to rely on a number of `dynamic linker improvements
+<https://android.googlesource.com/platform/bionic/+/refs/heads/master/android-changes-for-ndk-developers.md>`__
+which simplify the use of dynamic libraries.
+
+
+Architectures
+-------------
+
+Android currently supports 4 architectures. Their names as used by the Android
+tools are:
+
+* ``armeabi-v7a``
+* ``arm64-v8a``
+* ``x86``
+* ``x86_64``
+
+Virtually all current physical devices use one of the ARM architectures. ``x86``
+and ``x86_64`` are supported for use in emulators on development machines.
+
+``arm64-v8a`` is also supported in emulators running on macOS with Apple
+silicon. Unlike on iOS, an ARM64 emulator uses the same ABI as a real ARM64
+device, and can run the same binaries.
+
+We propose that Tier 3 support will only cover the 64-bit platforms
+(``arm64-v8a`` and ``x86_64``):
+
+* ``x86`` has not been supported as a development platform since 2020, and no
+  new emulator images have been released since then.
+* ``armeabi-v7a`` is still used, but its proportion of active devices is now
+  `less than 10% and steadily falling
+  <https://github.com/chaquo/chaquopy/issues/709#issuecomment-1744541892>`__.
+
+However, no changes should be made which would impede any downstream projects
+which still wish to build for the 32-bit architectures.
+
+
+App lifecycle
+-------------
+
+The primary programming language on Android is Java, or its modern descendant
+Kotlin. As such, an app does not provide its own executable file. Instead, all
+apps start off as a Java virtual machine running an executable provided by the
+operating system. The app's Java code can then add native code to the process by
+loading dynamic libraries and calling them through JNI.
+
+Unlike iOS, creating subprocesses *is* supported on Android. However apps may
+only run executables in `certain locations
+<https://issuetracker.google.com/issues/128554619#comment4>`__, none of which
+are writable at runtime. Long-running subprocesses are `officially discouraged
+<https://issuetracker.google.com/issues/128554619#comment4>`__, and are not
+guaranteed to be supported in future Android versions.
+
+Android does provide a command-line shell, but this is intended only for use by
+developers, and is not available to the typical end user.
+
+For these reasons, the primary way of running Python on Android will be by
+loading ``libpython3.x.so`` into the main app process. Although there will also
+be a ``python3.x`` executable linked against ``libpython3.x.so``, this is only
+for debugging, not production use.
+
+
+Specification
+=============
+
+Scope of work
+-------------
+
+The focus of this work will be to produce an Android equivalent to the existing
+Windows embeddable package. This will be used by app developers, not end users
+directly, so no installer will be required.
+
+Adding Android as a Tier 3 platform only requires adding support for compiling
+an Android-compatible build from the unpatched CPython source code. It does not
+require there to be any officially distributed Android artefacts, although these
+could be added to python.org in the future.
+
+A Gradle project will be provided for the purpose of running the CPython test
+suite. Tooling will be provided to automate the process of building the test
+suite app, starting the emulator, installing the test suite, and executing
+it.
+
+
+Linkage
+-------
+
+For the reasons discussed in `App lifecycle`_, Python will be included in the
+app as a dynamic ``libpython3.x.so`` library. Android extension modules should
+all be linked against this library. This allows packages to use the
+``-Wl,--no-undefined`` option to detect missing symbols at build time, which can
+be a significant time-saver.
+
+An extension module linked against ``libpython3.x.so`` cannot be loaded by an
+executable that has been statically linked against ``libpython3.x.a``.
+Therefore, a static ``libpython3.x.a`` library will not be supported on Android.
+This is the same pattern used by CPython on Windows.
+
+Unlike iOS, Android allows dynamic libraries to be loaded from any location, so
+a directory tree containing co-located .py, .pyc and .so files can be handled by
+Python's standard importer.
+
+
+Standard library
+----------------
+
+A number of standard library modules will not be supported on Android because
+the underlying C APIs are not provided by the operating system:
+
+* ``crypt`` – Scheduled for removal in Python 3.13
+* ``curses`` and ``readline``
+* ``dbm.gnu`` and ``dbm.ndbm``
+* ``grp``, ``nis`` and ``spwd``
+
+Android does not support any part of the `System V IPC API
+<https://man7.org/linux/man-pages/man7/sysvipc.7.html>`__, so the
+``multiprocessing`` module will not work in its current form. However,
+subprocesses in general are allowed, with some limitations – see `App
+lifecycle`_.
+
+``tkinter`` and ``turtle`` will not be included, because that would require an
+Android build of Tk itself, which is not officially supported.
+
+
+Platform identification
+-----------------------
+
+``sys``
+'''''''
+
+``sys.platform`` will return ``"android"``. Although Android is based on Linux,
+it differs in enough significant ways that a separate name is justified.
+
+``platform``
+''''''''''''
+
+Most of the values returned by the ``platform`` module will match those returned
+by ``os.uname()``, with the exception of:
+
+* ``platform.system()`` - ``"Android"``, instead of the default ``"Linux"``
+
+* ``platform.release()`` - Android version number, as a string (e.g. ``"14"``),
+  instead of the Linux kernel version
+
+In addition, a ``platform.android_ver()`` method will be added, which returns a
+namedtuple containing the following:
+
+* ``release`` - Android version, as a string (e.g. ``"14"``)
+* ``api_level`` - Android API level, as an integer (e.g. ``34``)
+* ``min_api_level`` - Minimum API level this build of Python can run on, as
+  an integer (e.g. ``23``). This is the same as ``sys.getandroidapilevel``.
+* ``model`` - the model name of the device, as a string (e.g. ``"Pixel 7"``).
+
+``os``
+''''''
+
+``os.uname()`` will return the raw result of a POSIX ``uname()`` call. This will
+result in the following values:
+
+* ``sysname`` - ``"Linux"``
+
+* ``release`` - The Linux kernel version (e.g.
+  ``"5.10.157-android13-4-00003-gdfb1120f912b-ab10994928"``)
+
+This approach treats the ``os`` module as a "raw" interface to system APIs, and
+``platform`` as a higher-level API providing more generally useful values.
+
+
+CI resources
+------------
+
+`Anaconda <https://anaconda.com>`__ has offered to provide physical hardware to
+run Android buildbots.
+
+GitHub Actions is able to host Android emulators on their Linux and macOS
+runners. The free tier currently only provides x86_64 machines; however ARM64
+macOS runners `recently became available on paid plans <https://github.blog/
+2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/>`__.
+
+
+Packaging
+---------
+
+Android wheels will use tags in the format ``android_<api-level>_<abi>``. For
+example:
+
+* ``android_23_arm64_v8a``
+* ``android_23_x86_64``
+
+For the meaning of ``<api-level>``, see `Version numbering`_. In the context of
+the wheel tag, it indicates the minimum Android version that was selected when
+the wheel was compiled. Installation tools such as pip should interpret this in
+a similar way to the existing macOS tags, i.e. an app with a minimum API level
+of N can incorporate wheels tagged with API level N or older.
+
+This format originates from the Chaquopy project, which currently maintains a
+`wheel repository <https://chaquo.com/pypi-13.1/>`__ with tags varying between
+API levels 16 and 21.
+
+However, relying on a small group of Android enthusiasts to build the whole
+Python ecosystem is not a scalable solution. Until prominent libraries routinely
+release their own Android wheels on PyPI, the ability of the community to adopt
+Python on Android will be limited.
+
+Therefore, it will be necessary to clearly document how projects can add Android
+builds to their CI and release tooling. Adding Android support to tools like
+`crossenv <https://crossenv.readthedocs.io/>`__ and `cibuildwheel
+<https://cibuildwheel.readthedocs.io/>`__ may be one way to achieve this.
+
+
+PEP 11 Update
+-------------
+
+:pep:`11` will be updated to include the two supported Android ABIs. Autoconf
+already identifies them with the following triplets:
+
+* ``aarch64-linux-android``
+* ``x86_64-linux-android``
+
+Petr Viktorin will serve as the initial core team contact for these ABIs.
+
+
+Backwards Compatibility
+=======================
+
+Adding a new platform does not introduce any backwards compatibility concerns to
+CPython itself. However, there may be some backwards compatibility implications
+on the projects that have historically provided CPython support (i.e., BeeWare
+and Kivy) if the final form of any CPython patches don't align with the patches
+they have historically used.
+
+
+Security Implications
+=====================
+
+Adding a new platform does not add any security implications.
+
+
+How to Teach This
+=================
+
+The education needs related to this PEP mostly relate to how developers can
+build Python into an Android app and use it at runtime. Automating and
+documenting this will be the responsibility of higher-level tools such as
+BeeWare's `Chaquopy <https://chaquo.com/chaquopy/>`__ and Kivy's `Buildozer
+<https://buildozer.readthedocs.io/en/latest/>`__, rather than CPython itself.
+
+
+Reference Implementation
+========================
+
+The `Chaquopy repository
+<https://github.com/chaquo/chaquopy/tree/master/target>`__ contains a reference
+patch and build scripts. These will have to be decoupled from the other
+components of Chaquopy before they can be upstreamed.
+
+`Briefcase <https://briefcase.readthedocs.org>`__ provides a reference
+implementation of code to execute test suites on Android devices and emulators.
+The `Toga Testbed <https://github.com/beeware/toga/tree/main/testbed>`__ is an
+example of a test suite that is executed on the Android emulator using GitHub
+Actions.
+
+
+Copyright
+=========
+
+This document is placed in the public domain or under the CC0-1.0-Universal
+license, whichever is more permissive.