Fail to install on Ubuntu 24.04

[hutzelmann]

Atheris apparently cannot be installed on a Ubuntu 24.04.

Steps to Reproduce

1. Get a fresh copy of Ubuntu 24.04
2. Install the main dependencies

sudo apt install -y python3-pip python3-venv llvm clang

3. (optional) get a virtual environment

python3 -m venv .venv && source .venv/bin/activate

4. Try to install atheris

pip3 install atheris

results in the following error

Collecting atheris
  Downloading atheris-2.3.0.tar.gz (304 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 304.1/304.1 kB 6.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: atheris
  Building wheel for atheris (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for atheris (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [41 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-312
      copying atheris_no_libfuzzer.py -> build/lib.linux-x86_64-cpython-312
      creating build/lib.linux-x86_64-cpython-312/atheris
      copying src/coverage_test.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/version_dependent.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/custom_crossover_fuzz_test.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/utils.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/function_hooks.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/pyinstaller_coverage_test.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/instrument_bytecode.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/__init__.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/custom_mutator_fuzz_test.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/fuzz_test.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/coverage_test_helper.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/fuzzed_data_provider_test.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/regex_match_generation_test.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/import_hook.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/fuzz_test_lib.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/custom_mutator_and_crossover_fuzz_test.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/hook-atheris.py -> build/lib.linux-x86_64-cpython-312/atheris
      copying src/coverage_g3test.py -> build/lib.linux-x86_64-cpython-312/atheris
      running build_ext
      Your libFuzzer is up-to-date.
      creating tmp
      x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/home/user/.venv/include -I/usr/include/python3.12 -c /tmp/tmpj0ufddpk.cpp -o tmp/tmpj0ufddpk.o -std=c++14
      building 'atheris.native' extension
      creating build/temp.linux-x86_64-cpython-312
      creating build/temp.linux-x86_64-cpython-312/src
      creating build/temp.linux-x86_64-cpython-312/src/native
      x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -DVERSION_INFO='2.3.0' -DATHERIS_MODULE_NAME=native -I/tmp/pip-build-env-_fvgqyh7/normal/lib/python3.12/site-packages/pybind11/include -I/home/user/.venv/include -I/usr/include/python3.12 -c src/native/atheris.cc -o build/temp.linux-x86_64-cpython-312/src/native/atheris.o -DPYBIND11_DETAILED_ERROR_MESSAGES=1 -Wno-attributes -Wno-address -Wno-deprecated-declarations -std=c++14
      x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -DVERSION_INFO='2.3.0' -DATHERIS_MODULE_NAME=native -I/tmp/pip-build-env-_fvgqyh7/normal/lib/python3.12/site-packages/pybind11/include -I/home/user/.venv/include -I/usr/include/python3.12 -c src/native/codetable_gen.cc -o build/temp.linux-x86_64-cpython-312/src/native/codetable_gen.o -DPYBIND11_DETAILED_ERROR_MESSAGES=1 -Wno-attributes -Wno-address -Wno-deprecated-declarations -std=c++14
      src/native/codetable_gen.cc: In lambda function:
      src/native/codetable_gen.cc:46:17: error: ‘PRECALL’ was not declared in this scope; did you mean ‘CALL’?
         46 |   opcode_caches[PRECALL] = 1;
            |                 ^~~~~~~
            |                 CALL
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for atheris
Failed to build atheris
ERROR: Could not build wheels for atheris, which is required to install pyproject.toml-based projects

pip3 install --no-binary atheris atheris and trying to build with pip from the sources results in the same error. On Ubuntu 22.04 the above works without any problems.

Additional Information

python --version
Python 3.12.3

clang --version
Ubuntu clang version 18.1.3 (1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

x86_64-linux-gnu-gcc --version
x86_64-linux-gnu-gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0

It would be great if you can have a closer look into this problem. Please let me know if you need further help in reproducing this bug.

[tecknicaltom]

It looks like this is a bigger issue affecting building Atheris against python 3.12, not Ubuntu specific. The PRECALL instruction was removed before 3.12 which the Atheris code references.

See python/cpython#92925 and https://docs.python.org/3.12/whatsnew/3.12.html#cpython-bytecode-changes

[mrquokka]

Some news? Cannot install anymore

[Hnasar]

Just wanted to note that there's a draft PR to fix this, but it seems to have stalled? #60

[SweetVishnya]

@jvoisin, any news on supporting Atheris compilation on Ubuntu 24.04?

[AidenRHall]

Julien is no longer actively maintaining Atheris, its just me now. Unfortunately it's been hard to find time to work on it given the current environment, working on Atheris is one small part of my overall job and we've been focusing only on core priorities in the immediate past. So I would like to acknowledge that Atheris isn't in the best state that it could be in, and that's on me.

I think this issue is really about adding support for python 3.12, which I am actively working on internally - I was hoping to finish before the holidays but that looks unlikely at this point. Long term I would like to focus more on the OSS use case and move all development to Github instead of using Google's internal tools, hopefully that will happen in 2025. The VM keeps changing and we need to explore some options to reduce the complexity of adding support for new Python versions.

The other part of why there hasn't been more activity is that there hasn't been a clear vision for Atheris to move towards, I am also hoping to change that in 2025. Stay tuned, and thanks for your patience and understanding in the meantime.