AArch64 architecture support #9
Comments
|
I have same problem compiling with Jetson Nano. |
|
I'm still getting such error And I see in Are these files expected to be generated? |
|
Please fix this issue fast :-) |
|
I am also having this issue. Especially:
when building from source. It is impossible to install via pip, because it uses |
|
Sorry for the delay on this. You should now be able to: a) Installing with generic arch support:
b) Building ARM64 support: See here for list of architectures: https://github.com/flame/blis/blob/0.5.1/config_registry Fingers crossed, this will build you a wheel that supports your platform. You could then make a PR with the |
|
Hey, great that you're back with a fix for this! Much appreciated.
Are you sure about these exact steps/commands for building? In your original reply there was some I have not tried building it on my AArch64 system with the new configuration; I will try to do so in the next couple of days/weeks and keep you updated. |
|
@Impelon Thanks, updated to say What machine are you using, by the way? Do you know any cloud computing resources that provide these ARM architectures so that I can test things? |
|
@honnibal Both builds failed on Jetson Nano - |
|
Could you describe the failure? If you clone and build |
|
@honnibal I am on a Raspberry Pi 3 with Arch Linux ARM AArch64. Also as far as I understand the Raspberry Pi 3 actually uses the coretex-A53 micro-architecture, so I'll be using that. Update 1This is the pip-log after the failed installation with the first method. Will try the 2nd option now. Update 2I have now tried the second method and succeded. I continued afterwards with the building process by running [...]
/home/server/cython-blis/blis/_src/include/linux-cortexa53/blis.h:4330:15: warning: 'bli_is_odd' defined but not used -Wunused-function]
static bool_t bli_is_odd( gint_t a )
^~~~~~~~~~
/home/server/cython-blis/blis/_src/include/linux-cortexa53/blis.h:4310:16: warning: 'bli_round_to_mult' defined but not used [-Wunused-function]
static guint_t bli_round_to_mult( guint_t val, guint_t mult )
^~~~~~~~~~~~~~~~~
/home/server/cython-blis/blis/_src/include/linux-cortexa53/blis.h:4303:15: warning: 'bli_round' defined but not used [-Wunused-function]
static double bli_round( double a )
^~~~~~~~~
/home/server/cython-blis/blis/_src/include/linux-cortexa53/blis.h:1802:13: warning: 'bli_obj_init_subpart_from' defined but not used [-Wunused-function]
static void bli_obj_init_subpart_from( obj_t* a, obj_t* b )
^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/server/cython-blis/include/numpy/ndarraytypes.h:1728,
from /home/server/cython-blis/include/numpy/ndarrayobject.h:17,
from /home/server/cython-blis/include/numpy/arrayobject.h:15,
from blis/py.c:598:
/home/server/cython-blis/include/numpy/npy_deprecated_api.h:11:2: warning: #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION"
^~~~~~~Finally I executed In conclusion: I did manage to compile a blis-wheel on my AArch64-system. |
|
Looking through the first log I posted above, I noticed that blis 0.2.4 was requested by thinc. |
|
@Impelon Thanks! Yes I need to push new versions of Thinc and spaCy that allow v0.3.1. I think it's just a matter of updating the requirements though. |
|
I just tried to install my compiled wheel, though. (env) [_redacted_ dist]$ pip install blis-0.3.1-cp37-cp37m-linux_aarch64.whl
Processing ./blis-0.3.1-cp37-cp37m-linux_aarch64.whl
Requirement already satisfied: numpy>=1.15.0 in /usr/lib/python3.7/site-packages (from blis==0.3.1) (1.17.0)
Installing collected packages: blis
Successfully installed blis-0.3.1
(env) [_redacted_ dist]$ python3
Python 3.7.4 (default, Jul 30 2019, 12:59:34)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import blis
Segmentation fault (core dumped)Any ideas? |
|
Hmm. I'm really not sure how to debug that :(. Can you try the generic kernel? |
|
Still not able to build on ARM64 :( Will try |
|
Nevermind, there is a typo in readme. PR created. |
|
I've succeded build and importing blis on my ARM64 server there are some problems with benchmarking, but hope it is minor |
|
Sorry I have not responded these past few days, I have been quite sick. Eitherway, I am going to try to give @honnibal some more info to debug this. I've run python with gdb: (env) [server@alarm cython-blis]$ gdb python
Reading symbols from python...
(No debugging symbols found in python)
(gdb) run
Starting program: /home/server/cython-blis/env/bin/python
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Python 3.7.4 (default, Jul 30 2019, 12:59:34)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import blis
Program received signal SIGSEGV, Segmentation fault.
0x0000ffffbed87620 in bli_gks_lookup_ind_cntx ()
from /home/server/cython-blis/blis/cy.cpython-37m-aarch64-linux-gnu.so
(gdb) backtrace
#0 0x0000ffffbed87620 in bli_gks_lookup_ind_cntx () from /home/server/cython-blis/blis/cy.cpython-37m-aarch64-linux-gnu.so
#1 0x0000ffffbed986b0 in bli_ind_init () from /home/server/cython-blis/blis/cy.cpython-37m-aarch64-linux-gnu.so
#2 0x0000ffffbed87ee0 in bli_init_apis () from /home/server/cython-blis/blis/cy.cpython-37m-aarch64-linux-gnu.so
#3 0x0000ffffbf32c378 in __pthread_once_slow () from /usr/lib/libpthread.so.0
#4 0x0000ffffbebef350 in __pyx_pymod_exec_cy (__pyx_pyinit_module=<optimized out>) at blis/cy.c:22101
#5 0x0000ffffbf424700 in PyModule_ExecDef () from /usr/lib/libpython3.7m.so.1.0
#6 0x0000ffffbf4e149c in ?? () from /usr/lib/libpython3.7m.so.1.0
#7 0x0000ffffbf3ddfb8 in _PyMethodDef_RawFastCallDict () from /usr/lib/libpython3.7m.so.1.0
#8 0x0000ffffbf3de024 in _PyCFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#9 0x0000ffffbf3b5ce0 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#10 0x0000ffffbf4c6b6c in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#11 0x0000ffffbf3dd154 in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#12 0x0000ffffbf3b6ffc in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#13 0x0000ffffbf3ad44c in ?? () from /usr/lib/libpython3.7m.so.1.0
#14 0x0000ffffbf3b70a8 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#15 0x0000ffffbf3ad44c in ?? () from /usr/lib/libpython3.7m.so.1.0
#16 0x0000ffffbf3b4c44 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#17 0x0000ffffbf3ad44c in ?? () from /usr/lib/libpython3.7m.so.1.0
#18 0x0000ffffbf3b4c44 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#19 0x0000ffffbf3ad44c in ?? () from /usr/lib/libpython3.7m.so.1.0
#20 0x0000ffffbf3dd0b0 in _PyFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#21 0x0000ffffbf3df050 in ?? () from /usr/lib/libpython3.7m.so.1.0
#22 0x0000ffffbf3df36c in _PyObject_CallMethodIdObjArgs () from /usr/lib/libpython3.7m.so.1.0
#23 0x0000ffffbf4e4ff8 in PyImport_ImportModuleLevelObject () from /usr/lib/libpython3.7m.so.1.0
#24 0x0000ffffbf3b7b14 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#25 0x0000ffffbf4c6b6c in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#26 0x0000ffffbf4c6f00 in PyEval_EvalCodeEx () from /usr/lib/libpython3.7m.so.1.0
#27 0x0000ffffbf4c6f44 in PyEval_EvalCode () from /usr/lib/libpython3.7m.so.1.0
#28 0x0000ffffbf4c3cfc in ?? () from /usr/lib/libpython3.7m.so.1.0
#29 0x0000ffffbf3ddf90 in _PyMethodDef_RawFastCallDict () from /usr/lib/libpython3.7m.so.1.0
#30 0x0000ffffbf3de024 in _PyCFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#31 0x0000ffffbf3b5ce0 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#32 0x0000ffffbf4c6b6c in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#33 0x0000ffffbf3dd154 in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#34 0x0000ffffbf3b6ffc in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#35 0x0000ffffbf3ad44c in ?? () from /usr/lib/libpython3.7m.so.1.0
#36 0x0000ffffbf3b70a8 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#37 0x0000ffffbf3ad44c in ?? () from /usr/lib/libpython3.7m.so.1.0
#38 0x0000ffffbf3b4c44 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#39 0x0000ffffbf3ad44c in ?? () from /usr/lib/libpython3.7m.so.1.0
#40 0x0000ffffbf3b4c44 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#41 0x0000ffffbf3ad44c in ?? () from /usr/lib/libpython3.7m.so.1.0
#42 0x0000ffffbf3dd0b0 in _PyFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#43 0x0000ffffbf3df050 in ?? () from /usr/lib/libpython3.7m.so.1.0
#44 0x0000ffffbf3df36c in _PyObject_CallMethodIdObjArgs () from /usr/lib/libpython3.7m.so.1.0
#45 0x0000ffffbf4e4ff8 in PyImport_ImportModuleLevelObject () from /usr/lib/libpython3.7m.so.1.0
#46 0x0000ffffbf3b7b14 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#47 0x0000ffffbf4c6b6c in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#48 0x0000ffffbf4c6f00 in PyEval_EvalCodeEx () from /usr/lib/libpython3.7m.so.1.0
#49 0x0000ffffbf4c6f44 in PyEval_EvalCode () from /usr/lib/libpython3.7m.so.1.0
#50 0x0000ffffbf4fdf2c in ?? () from /usr/lib/libpython3.7m.so.1.0
#51 0x0000ffffbf4fe280 in PyRun_InteractiveLoopFlags () from /usr/lib/libpython3.7m.so.1.0
#52 0x0000ffffbf4fed50 in PyRun_AnyFileExFlags () from /usr/lib/libpython3.7m.so.1.0
#53 0x0000ffffbf5234d4 in ?? () from /usr/lib/libpython3.7m.so.1.0
#54 0x0000ffffbf5239c0 in _Py_UnixMain () from /usr/lib/libpython3.7m.so.1.0
#55 0x0000ffffbf1cb6e4 in __libc_start_main () from /usr/lib/libc.so.6
#56 0x0000aaaaaaaaa910 in _start ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)Also I did try the generic kernel, you can see my failed attempt and the log-file in my previous post, where I documented the installation process. @gaiar I think you should make a PR with the cortexa57 configuration as mentioned by @honnibal. Then others may not have to worry about this in the future. |
|
Yes, sure, I'll create a PR, but I've into a problem, that numpy even after compilation doesn't pickup blis library :( I've tried release version and compiled from master. |
|
I am not sure that this matters. I mean correct me if I am entirely wrong about this, but I think as long as we were able to successfully build the wheel the configuration was correct. I think they are used to collect information about the architectures capabilities. I think the same configuration-files will be generated on the same architectures, regardless if the produced wheel is working or not. Also I'm a bit unsure on what problem you are having with numpy. Looking through your error above when benchmarking this strikes my eye: File "/root/developer/cython-blis/env3.6/lib/python3.6/site-packages/blis/benchmark.py", line 19, in get_numpy_blas
blas_libs = numpy.__config__.blas_opt_info["libraries"]
KeyError: 'libraries'That error has nothing to do with blis. >>> import numpy
>>> numpy.__config__.blas_opt_info
{'define_macros': [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)], 'libraries': ['cblas', 'blas', 'blas'], 'library_dirs': ['/usr/lib'], 'include_dirs': ['/usr/local/include', '/usr/include'], 'language': 'c'}
>>> numpy.__config__.blas_opt_info["libraries"]
['cblas', 'blas', 'blas']Again, correct me if I am wrong, but I don't think numpy is supposed to work together with blis; blis is module you can use to perform some operations faster than you could with numpy, or not? |
|
Yes, you are correct. But after I'm installing I can try uploading the wheel, so and then maybe or someone can try installing it and running - to have another pair of eyes. |
|
Here is the wheel |
|
Hello :) EDIT: |
I have the same error! Please anyone recommend me! |
|
I managed to get spaCy compiling and running on my NVIDIA Jetson TX1 (
The above trick works because spaCy 2.1.8 depends on blis 0.2 which doesn't have generic arch support JSON file, which is included in release 0.4 as per requirements.txt on master branch; but if you compile the development version you cannot download any model because they are not released yet. So I'm using the stable branch with latest development dependencies. BUT: I didn't manage (yet) to compile it with GPU support. |
|
I tried the various solutions as above on Jetson TX2 but none of them worked. Can someone please provide a whl? It seems to choose x86 architecture always when compiling blis from source. gcc -c /home/nvidia/cython-blis/blis/_src/kernels/zen/1/bli_amaxv_zen_int.c -o /tmp/tmp7oxv4z9t/bli_amaxv_zen_int.o -O3 -mavx2 -mfma -mfpmath=sse -march=core-avx2 -fPIC -std=c99 -D_POSIX_C_SOURCE=200112L -DBLIS_VERSION_STRING="0.5.1" -DBLIS_IS_BUILDING_LIBRARY -Iinclude/linux-x86_64 -I./frame/3/ -I./frame/ind/ukernels/ -I./frame/1m/ -I./frame/1f/ -I./frame/1/ -I./frame/include -I/home/nvidia/cython-blis/blis/_src/include/linux-x86_64 |
|
I created PR #25 to add the generated files for cortex-a57. |
|
PR #25 got merged. Thanks!
Could somebody please propagate the change to https://pypi.org/project/blis/#files |
|
I got past the These files were generated on a Raspberry Pi 3 Model B V1.2 with an armv71 chip with a model name of cortex-a53. The CPU information of the single-board system was gathered via
The Raspberry Pi is running Arch Linux ARM with kernel version 4.19.115-1. I seem to be facing the same error that @sreevan is facing. |
|
@sebpop Sorry for the delay on this (and sorry for the delay to others as well!) I had a lot of trouble updating I've now made a new release that also includes build files for a cortexa57 architecture, which is now supported by Travis CI. I'm hoping we can set up a build system for more architectures as well. |
|
Can someone write a new tutorial on how to install this on a Pi 4... going from here to stackoverflow to figure out how to install this is getting exhausting. I’ve been trying to build the blis wheel for 3 days now... |
1 similar comment
|
Can someone write a new tutorial on how to install this on a Pi 4... going from here to stackoverflow to figure out how to install this is getting exhausting. I’ve been trying to build the blis wheel for 3 days now... |
|
After lots of hiccups, finally could compile for Jetson TX2 (Cortex-A57): |
When I try building spaCy after installing this in a virtualenv, pip still tries to install |
|
Hi, this issue is fairly old and we don't have any idea about the quality/suitability of the wheel from that link. Can you open a new issue describing the particular problems you're running into installing |
|
@adrianeboyd Thanks for the quick response! I'm trying to install a fairly old spaCy version as dependency for AllenNLP on a Jetson Nano. I'll open a separate issue for this. |
|
For other people searching for issues related to linux-cortexa57.jsonl not being found, Google brought me to this issue. You want to look at #112 |
Problem when trying to build on ARM64 machine, similar to #6
Command executed:
That problem is a blocker for running SpaCy on ARM64 and platforms like Nvidia Jetson.
The text was updated successfully, but these errors were encountered: