-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add architecture detection logic to setup.py #13
Comments
To make suggestions, that give me some information on my AArch64 machine (so this will only work on Linux and maybe similar):
|
Thanks, that's very helpful! I think it's reasonable to focus first on Linux support. I don't think non-x86 OSX is even possible currently. |
I have actually looked some more into it. Also after a bit more research I found, that Intel and AMD CPU's dump a hex identifying their micro-architecture in I started writing a small script that would detect the micro-architecture on ARM systems with Linux, by comparing the found import platform
if platform.machine() == "aarch64": # we are on aarch64, hence we can check for "CPU part" to determine the micro-arch
with open("/proc/cpuinfo", "r") as f:
for line in f:
if not line.strip(): # ignore multiple (logical and physical) cores
break
if "cpu part" in line.lower():
parsed = line.rpartition("0x")
if parsed[1]:
print("Micro-Arch:", parsed[2]) # found hex-value
# prints `Micro-Arch: d03` on my aarch64 system
# now what? find the corresponding micro-arch! somehow...
#microarch_from_id(parsed[2]) This would be a crude but simple, self-contained way to detect aarch64 micro-architectures on Linux. But this would have to be maintained, and I could not find a good source for matching micro-architectures to identifiers, except maybe the file I already linked in pytorch's library. I also quickly searched on flame/blis, since the config_registry mentioned they wanted to add support for this, and I found this, which also uses All in all, I think it would be best to use a tool such as pytorch/cpuinfo (works on multiple OS) or |
Mine output:
|
This is a fix for bug explosion#13 : add logic to setup.py to detect aarch64 LITTLE cores for which BLIS has configure flag BLIS_ARCH="cortexa53". All other cores will configure with BLIS_ARCH="cortexa57".
This is a fix for bug explosion#13 : add logic to setup.py to detect aarch64 LITTLE cores for which BLIS has configure flag BLIS_ARCH="cortexa53". All other cores will configure with BLIS_ARCH="cortexa57".
This is a fix for bug explosion#13 : add logic to setup.py to detect aarch64 LITTLE cores for which BLIS has configure flag BLIS_ARCH="cortexa53". All other cores will configure with BLIS_ARCH="cortexa57".
This is a fix for bug #13 : add logic to setup.py to detect aarch64 LITTLE cores for which BLIS has configure flag BLIS_ARCH="cortexa53". All other cores will configure with BLIS_ARCH="cortexa57".
I see #26 has been reverted again. Is there a specific reason why? Maybe because not every aarch64 system has the required binaries. How about my suggested solution for this using /proc/cpuinfo? That would allow for distinction between all aarch64 CPUs, especially most commonly cortexA53 and cortexA57. |
I wondered where this had gone, too, and it looks like If you'd like to test it you should be able to run something like this to install from the PR branch:
Run the (minimal) tests with:
I'd appreciate any testing, especially on platforms I don't have access to! I would like to release v0.4.2 with python 3.9 wheels, but we need figure out a few details about what happens for users who are upgrading spacy on platforms that no longer have binary wheels. A lot of existing packages have |
Well I had testing blis on the Raspberry Pi 3 (with Edit: Also just noticed I mistakenly put referenced PR 41 instead of 44 above, wooups. |
The
setup.py
needs a smarter way to pick non-x86_64 architectures automatically, so that folks don't need to use theBLIS_ARCH
environment variable as often. This is currently hard for me to test as I don't have access to machines with alternate architectures, so --- help wanted!The text was updated successfully, but these errors were encountered: