"Failed to create device: 1008" for Llama3.2-3B on 8295P automotive (dsp arch v66 & soc model 31)

[mikel-brostrom]

I am trying to run a Llama-v3.2-3b-chat-quantized on a 8295P chip so I have set

dsp_arch = "v66"
soc_model = 31

in htp_backend_ext_config.json accordingly. I installed hexagonsdk 4.1.0.4 by:

qpm-cli --install hexagonsdk4.x -v 4.1.0.4

to match the version recommendation for my hexagon architecture (v66) specified under https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/linux_setup.html#htp-and-dsp

After:

adb push genie_bundle /data/local/tmp
adb shell
cd /data/local/tmp/genie_bundle
export LD_LIBRARY_PATH=$PWD
./genie-t2t-run -c genie_config.json -p "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nWhat is France's capital?<|eot_id|><|start_header_id|>assistant<|end_header_id|>"

I get the following error:

Using libGenie.so version 1.2.0

[ERROR] "Failed to create device: 1008"
[ERROR] "Device Creation failure"
Failure to initialize model
Failed to create the dialog.

[xiaomiovea]

@mikel-brostrom Try to set ADSP_LIBRARY_PATH, I solved this problem after setting it up! (Failed to create device: 1008)

adb push genie_bundle /data/local/tmp
adb shell
cd /data/local/tmp/genie_bundle
export LD_LIBRARY_PATH=$PWD
export ADSP_LIBRARY_PATH="$PWD;/odm/lib/rfsa/adsp;/vendor/lib/rfsa/adsp/;/system/lib/rfsa/adsp;/system/vendor/lib/rfsa/adsp;/dsp"
./genie-t2t-run -c genie_config.json -p "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nWhat is France's capital?<|eot_id|><|start_header_id|>assistant<|end_header_id|>"

[mikel-brostrom]

I wont have access to the 8295P device untill after christmas. Reporting back then 😊

[mikel-brostrom]

@mikel-brostrom Try to set ADSP_LIBRARY_PATH, I solved this problem after setting it up! (Failed to create device: 1008)

adb push genie_bundle /data/local/tmp
adb shell
cd /data/local/tmp/genie_bundle
export LD_LIBRARY_PATH=$PWD
export ADSP_LIBRARY_PATH="$PWD;/odm/lib/rfsa/adsp;/vendor/lib/rfsa/adsp/;/system/lib/rfsa/adsp;/system/vendor/lib/rfsa/adsp;/dsp"
./genie-t2t-run -c genie_config.json -p "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nWhat is France's capital?<|eot_id|><|start_header_id|>assistant<|end_header_id|>"

Trying this out today 🚀

[mikel-brostrom]

Did not help to add that ADSP export @xiaomiovea ... Still the same error

Using libGenie.so version 1.2.0

[ERROR] "Failed to create device: 1008"
[ERROR] "Device Creation failure"
Failure to initialize model
Failed to create the dialog.

[mikel-brostrom]

Ok, I guess I am here now:

https://app.aihub.qualcomm.com/docs/hub/faq.html#why-am-i-seeing-error-1008-when-trying-to-use-htp

According to this I have two steps left, basically:

1. "The environment variable ADSP_LIBRARY_PATH must be set to a directory containing the skel files corresponding to your device’s DSP architecture"

2. "Using HTP in automotive guest VMs requires an additional step beyond setting ADSP_LIBRARY_PATH as outlined above: two files must be copied from the QNX host into the guest VM. Specifically, these files must be copied into a directory in ADSP_LIBRARY_PATH, typically the one containing the appropriate skeleton libraries."

   For targets with /dspfw and /dsplib partitions:

    /dsplib/image/dsp/cdsp0/libc++.so.1
    /dsplib/image/dsp/cdsp0/libc++abi.so.1
   

   For targets without /dspfw and /dsplib partitions:

    /mnt/etc/images/cdsp0/libc++.so.1
    /mnt/etc/images/cdsp0/libc++abi.so.1
   

[mikel-brostrom]

For 1)

I found these folders containing skel files on the target device. Which is the right one?

root@4bf5a16c41bc:/# find / -type f -name "*skel*" 2>/dev/null
/system/app/AmpeService/lib/arm64/libskeletoncore.so
/system/app/AmpeService/lib/arm64/libtts_skeleton.so
/vendor/lib/rfsa/adsp/tests/libhap_power_test_skel.so
/vendor/lib/rfsa/adsp/tests/libtest_dl1_skel.so
/vendor/lib/rfsa/adsp/tests/libtest_dl_perf1_skel.so
/vendor/lib/rfsa/adsp/tests/libtest_dl_perf_skel.so
/vendor/lib/rfsa/adsp/tests/librpcperf_skel.so
/vendor/lib/rfsa/adsp/tests/libtest_dl_skel.so
/vendor/lib/rfsa/adsp/tests/librpcperf1_skel.so
/vendor/lib/rfsa/adsp/tests/libfastrpc_tests_skel.so
/vendor/lib/rfsa/adsp/tests/libbenchmark_skel.so
/vendor/lib/rfsa/adsp/tests/libfastrpc_tests1_skel.so
/vendor/lib/rfsa/adsp/tests/libunsigned_pd_test_skel.so
/vendor/lib/rfsa/adsp/libfastcvdsp_v68_skel.so
/vendor/lib/rfsa/adsp/libdspCV_v66_skel.so
/vendor/lib/rfsa/adsp/libdspCV_v68_skel.so
/vendor/lib/rfsa/adsp/libfastcvdsp_v66_skel.so
/vendor/lib/libreverse_rpc_tests_skel.so
/vendor/lib/libsysmon_cdsp_skel.so
/vendor/lib64/libreverse_rpc_tests_skel.so
/vendor/lib64/libsysmon_cdsp_skel.so

I guess /vendor/lib/rfsa/adsp? So export ADSP_LIBRARY_PATH="/vendor/lib/rfsa/adsp" should work?

[mikel-brostrom]

For 2)

I found both libc++.so.1 and libc++abi.so.1 files under /root/Qualcomm/HEXAGON_Tools/8.4.05/Tools/lib/ after installing qpm-cli --extract hexagon8.4. Are these the right files?

root@4bf5a16c41bc:/# find / -type f -name "libc++.so.1" 2>/dev/null
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/musl/lib/x86_64-unknown-linux-musl/libc++.so.1
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/musl/lib/i686-unknown-linux-musl/libc++.so.1
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/lib/i386-unknown-linux-gnu/libc++.so.1
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/lib/x86_64-unknown-linux-gnu/libc++.so.1
/local/mnt/workspace/Qualcomm/Hexagon_SDK/4.1.0.4/tools/HEXAGON_Tools/8.4.05/Tools/lib/libc++.so.1
/local/mnt/workspace/Qualcomm/Hexagon_SDK/4.1.0.4/tools/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/lib64/libc++.so.1
/local/mnt/workspace/Qualcomm/Hexagon_SDK/4.1.0.4/tools/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/lib/lib64/libc++.so.1

root@4bf5a16c41bc:/# find / -type f -name "libc++.so.1" 2>/dev/null
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/musl/lib/x86_64-unknown-linux-musl/libc++.so.1
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/musl/lib/i686-unknown-linux-musl/libc++.so.1
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/lib/i386-unknown-linux-gnu/libc++.so.1
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/lib/x86_64-unknown-linux-gnu/libc++.so.1
/local/mnt/workspace/Qualcomm/Hexagon_SDK/4.1.0.4/tools/HEXAGON_Tools/8.4.05/Tools/lib/libc++.so.1
/local/mnt/workspace/Qualcomm/Hexagon_SDK/4.1.0.4/tools/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/lib64/libc++.so.1
/local/mnt/workspace/Qualcomm/Hexagon_SDK/4.1.0.4/tools/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/lib/lib64/libc++.so.1
root@4bf5a16c41bc:/# find / -type f -name "libc++abi.so.1" 2>/dev/null
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/lib/i386-unknown-linux-gnu/libc++abi.so.1
/opt/android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/lib/x86_64-unknown-linux-gnu/libc++abi.so.1
/local/mnt/workspace/Qualcomm/Hexagon_SDK/4.1.0.4/tools/HEXAGON_Tools/8.4.05/Tools/lib/libc++abi.so.1
/local/mnt/workspace/Qualcomm/Hexagon_SDK/4.1.0.4/tools/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/lib64/libc++abi.so.1

[mikel-brostrom]

I ran export ADSP_LIBRARY_PATH="/vendor/lib/rfsa/adsp" and moved the two files to:

• /mnt/etc/images/cdsp0/libc++.so.1
• /mnt/etc/images/cdsp0/libc++abi.so.1

Still the same error

[mikel-brostrom]

Is it because I am using the 8.4.05 hexagon tools and not 8.4.06

? Are they even possible to download with qpm-cli? If not where do I find them?