Failing on Apple M1

[indrasvat]

Trying to launch a multi-process Sanic micro service with scalene to cpu profile it. It's complaining about incompatible architecture.

# In project root
$ source venv/bin/activate

$ ls -Altr `which python` && python -V && pip -V
lrwxr-xr-x  1 user  staff    52B Jul  6 10:02 ~/my-service/venv/bin/python@ -> /opt/homebrew/Cellar/[email protected]/3.8.13_1/bin/python3
Python 3.8.13
pip 21.3.1 from ~/my-service/venv/lib/python3.8/site-packages/pip (python 3.8)

$ python -m pip install scalene
Collecting scalene
  Downloading scalene-1.5.8-cp38-cp38-macosx_10_14_universal2.whl (355 kB)
     |████████████████████████████████| 355 kB 770 kB/s
Requirement already satisfied: numpy in ./venv/lib/python3.8/site-packages (from scalene) (1.23.1)
Collecting pynvml>=11.0.0
  Using cached pynvml-11.4.1-py3-none-any.whl (46 kB)
Collecting rich>=9.2.0
  Using cached rich-12.5.1-py3-none-any.whl (235 kB)
Collecting cloudpickle>=1.5.0
  Using cached cloudpickle-2.1.0-py3-none-any.whl (25 kB)
Collecting commonmark<0.10.0,>=0.9.0
  Using cached commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
Collecting pygments<3.0.0,>=2.6.0
  Downloading Pygments-2.12.0-py3-none-any.whl (1.1 MB)
     |████████████████████████████████| 1.1 MB 7.4 MB/s
Collecting typing-extensions<5.0,>=4.0.0
  Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Installing collected packages: typing-extensions, pygments, commonmark, rich, pynvml, cloudpickle, scalene
Successfully installed cloudpickle-2.1.0 commonmark-0.9.1 pygments-2.12.0 pynvml-11.4.1 rich-12.5.1 scalene-1.5.8 typing-extensions-4.3.0

$ pip list | rg scalene
scalene              1.5.8

$ python -m scalene --help
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/[email protected]/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/homebrew/Cellar/[email protected]/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "~/my-service/venv/lib/python3.8/site-packages/scalene/__main__.py", line 4, in <module>
    from scalene import scalene_profiler
  File "~/my-service/venv/lib/python3.8/site-packages/scalene/scalene_profiler.py", line 52, in <module>
    from scalene.scalene_mapfile import ScaleneMapFile
  File "~/my-service/venv/lib/python3.8/site-packages/scalene/scalene_mapfile.py", line 8, in <module>
    from scalene import get_line_atomic  # type: ignore
ImportError: dlopen(~/my-service/venv/lib/python3.8/site-packages/scalene/get_line_atomic.abi3.so, 0x0002): tried: '~/my-service/venv/lib/python3.8/site-packages/scalene/get_line_atomic.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/get_line_atomic.abi3.so' (no such file), '/usr/lib/get_line_atomic.abi3.so' (no such file)

Error:

ImportError: dlopen(~/my-service/venv/lib/python3.8/site-packages/scalene/get_line_atomic.abi3.so, 0x0002): tried: '~/my-service/venv/lib/python3.8/site-packages/scalene/get_line_atomic.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/get_line_atomic.abi3.so' (no such file), '/usr/lib/get_line_atomic.abi3.so' (no such file)

OS:

$ sw_vers
ProductName:	macOS
ProductVersion:	12.0.1
BuildVersion:	21A559

$ uname -a
Darwin MY_HOST 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:01 PDT 2021; root:xnu-8019.41.5~1/RELEASE_ARM64_T6000 arm64

Any ideas?

[emeryberger]

Just in case we happened to fix this with the latest release, please try updating (version 1.5.9). Thanks! (My development machine is an M1 and it works fine there.)

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: scalene in ./git/scalene (1.5.9)
Requirement already satisfied: rich>=9.2.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from scalene) (12.4.4)
Requirement already satisfied: cloudpickle>=1.5.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from scalene) (2.1.0)
Requirement already satisfied: pynvml>=11.0.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from scalene) (11.4.1)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from rich>=9.2.0->scalene) (0.9.1)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from rich>=9.2.0->scalene) (2.12.0)
% python3 -m pip list | grep scalene
scalene           1.5.9                      /Users/emery/git/scalene
% python3 -m scalene --help
usage: scalene [-h] [--version] [--column-width COLUMN_WIDTH] [--outfile OUTFILE] [--html] [--json] [--cli] [--web]
               [--port PORT] [--reduced-profile] [--profile-interval PROFILE_INTERVAL] [--cpu-only] [--profile-all]
               [--profile-only PROFILE_ONLY] [--profile-exclude PROFILE_EXCLUDE] [--use-virtual-time]
               [--cpu-percent-threshold CPU_PERCENT_THRESHOLD] [--cpu-sampling-rate CPU_SAMPLING_RATE]
               [--allocation-sampling-window ALLOCATION_SAMPLING_WINDOW] [--malloc-threshold MALLOC_THRESHOLD]
               [--program-path PROGRAM_PATH] [--memory-leak-detector] [--on | --off]

Scalene: a high-precision CPU and memory profiler, version 1.5.9
https://github.com/plasma-umass/scalene

I doubt the software version makes a difference, but in any event:

% sw_vers
ProductName:	macOS
ProductVersion:	12.5
BuildVersion:	21G72
% uname -a
Darwin HOSTNAME 21.6.0 Darwin Kernel Version 21.6.0: Sat Jun 18 17:07:22 PDT 2022; root:xnu-8020.140.41~1/RELEASE_ARM64_T6000 arm64

[indrasvat]

1.5.9 totally worked! Thanks a lot, @emeryberger 🤗!