Skip to content

Commit

Permalink
redo conflict changes
Browse files Browse the repository at this point in the history
  • Loading branch information
@chraac
chraac committed Nov 29, 2024
1 parent 67b183c commit 6d4feae
Show file tree
Hide file tree
Showing 5 changed files with 48 additions and 1 deletion.
3 changes: 2 additions & 1 deletion ggml/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,8 @@ set(GGML_PUBLIC_HEADERS
include/ggml-metal.h
include/ggml-rpc.h
include/ggml-sycl.h
include/ggml-vulkan.h)
include/ggml-vulkan.h
include/ggml-qnn.h)

set_target_properties(ggml PROPERTIES PUBLIC_HEADER "${GGML_PUBLIC_HEADERS}")
#if (GGML_METAL)
Expand Down
1 change: 1 addition & 0 deletions ggml/src/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,7 @@ ggml_add_backend(MUSA)
ggml_add_backend(RPC)
ggml_add_backend(SYCL)
ggml_add_backend(Vulkan)
ggml_add_backend(QNN)

foreach (target ggml-base ggml)
target_include_directories(${target} PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/../include> $<INSTALL_INTERFACE:include>)
Expand Down
11 changes: 11 additions & 0 deletions ggml/src/ggml-backend-reg.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,14 @@
#include "ggml-kompute.h"
#endif

#ifdef GGML_USE_KOMPUTE
#include "ggml-kompute.h"
#endif

#ifdef GGML_USE_QNN
#include "ggml-qnn.h"
#endif

struct ggml_backend_reg_entry {
ggml_backend_reg_t reg;
void * handle;
Expand Down Expand Up @@ -98,6 +106,9 @@ struct ggml_backend_registry {
#ifdef GGML_USE_KOMPUTE
register_backend(ggml_backend_kompute_reg());
#endif
#ifdef GGML_USE_QNN
register_backend(ggml_backend_qnn_reg());
#endif
#ifdef GGML_USE_CPU
register_backend(ggml_backend_cpu_reg());
#endif
Expand Down
34 changes: 34 additions & 0 deletions ggml/src/ggml-qnn/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
message(STATUS "Using QNN backend")

if(CMAKE_SYSTEM_NAME STREQUAL "Android")
find_library(LOG_LIB log)
find_library(ANDROID_LIB android)

This comment has been minimized.

Sign in to view

Copy link
@Dnonmi

Dnonmi Dec 5, 2024 •
edited
Loading

What are those variable names '_LIB' supposed to find?

This comment has been minimized.

Sign in to view

Copy link
@chraac

chraac Dec 5, 2024

Author Owner

here we relied on anroid ndk tool chain file to find those platform spec libs, for more detail:
https://developer.android.com/ndk/guides/cmake#file

This comment has been minimized.

Sign in to view

Copy link
@Dnonmi

Dnonmi Dec 5, 2024

I understand, but won't this generate warnings for users not using the NDK too?

This comment has been minimized.

Sign in to view

Copy link
@chraac

chraac Dec 5, 2024 •
edited
Loading

Author Owner

since we checked if the system is android at line 3, thought it will not entering this branch when at other system

This comment has been minimized.

Sign in to view

Copy link
@Dnonmi

Dnonmi Dec 6, 2024

Setting the platform version (minSdkVersion) to at least 28, and a few other Android env vars are a requirement then. The cache configuration will fail otherwise.
Built on Termux, with some many arguments.

This comment has been minimized.

Sign in to view

Copy link
@chraac

chraac Dec 6, 2024 •
edited
Loading

Author Owner

interesting,
my builder image using android-31 currently:
Image

and there's no cmake configuration warning popup wile compiling:
Image

do you mind pasting your error log here? can have a further look

This comment has been minimized.

Sign in to view

Copy link
@Dnonmi

Dnonmi Dec 6, 2024

This looks like the expected behavior. I used version 28 because it's the minimum for 'alligned_alloc' used in 'utils.cpp'.

But I don't know if my log will be of any help since I can't point out which env var was missing at that point. They're also missing information, I'll try to reproduce it later.
log_out.txt

This comment has been minimized.

Sign in to view

Copy link
@chraac

chraac Dec 6, 2024

Author Owner

looks like you're not build with the ndk cmake toolchain file, thought you could include it into you build command, like

cmake -H.. -B. -DGGML_QNN=on -DBUILD_SHARED_LIBS=off -DGGML_OPENMP=off -DANDROID_ABI="$$TARGET_ARCH" -DANDROID_PLATFORM="$$ANDROID_PLATFORM" -DANDROID_NDK="$$ANDROID_NDK_HOME" -DCMAKE_TOOLCHAIN_FILE="$$ANDROID_NDK_HOME/build/cmake/android.toolchain.cmake" -DGGML_QNN_SDK_PATH="$$QNN_SDK_PATH" -DCMAKE_BUILD_TYPE="$$BUILD_TYPE"

can have a further look here: https://github.com/chraac/llama-cpp-qnn-builder/blob/768090ece27f28028b8b695167dc587782f10fd8/docker/docker-compose-compile.yml#L27C9-L27C335

This comment has been minimized.

Sign in to view

Copy link
@Dnonmi

Dnonmi Dec 6, 2024 •
edited
Loading

Thank you, I know. I was later able to build it with the following:
cmake -B build -DGGML_QNN=ON -DGGML_OPENMP=OFF -DGGML_AMX=OFF -DCMAKE_ANDROID_NDK=~/android-sdk/ndk/26.1.10909125/ -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a -DCMAKE_C_COMPILER=/data/data/com.termux/files/home/android-sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/linux-aarch64/bin/clang -DCMAKE_CXX_COMPILER=/data/data/com.termux/files/home/android-sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/linux-aarch64/bin/clang++ -DCMAKE_TOOLCHAIN_FILE=~/android-sdk/ndk/26.1.10909125/build/cmake/android.toolchain.cmake -DANDROID_PLATFORM=android-28

It's segfaulting when loading the models, is it expected behavior?
out.log
set(QNN_LINK_LIBRARIES ${LOG_LIB} ${ANDROID_LIB})
set(QNN_DEFAULT_LIB_SEARCH_PATH "/data/local/tmp/" CACHE STRING "customized library search path for QNN backend")
else()
message(FATAL_ERROR "QNN now only available on Android")
endif()

if(NOT DEFINED GGML_QNN_SDK_PATH)
# try read from environment variable
if(DEFINED ENV{QNN_SDK_PATH})
set(GGML_QNN_SDK_PATH $ENV{QNN_SDK_PATH})
else()
message(FATAL_ERROR "GGML_QNN_SDK_PATH not defined")
endif()
endif()

message("QNN_SDK_PATH: ${GGML_QNN_SDK_PATH}")

string(REGEX REPLACE "/$" "" GGML_QNN_DEFAULT_LIB_SEARCH_PATH "${QNN_DEFAULT_LIB_SEARCH_PATH}")
add_compile_definitions(GGML_QNN_DEFAULT_LIB_SEARCH_PATH="${QNN_DEFAULT_LIB_SEARCH_PATH}/")

set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3")

file(GLOB QNN_SOURCES "${CMAKE_CURRENT_LIST_DIR}/*.cpp")
ggml_add_backend_library(ggml-qnn
${QNN_SOURCES}
)

target_include_directories(ggml-qnn PRIVATE ${GGML_QNN_SDK_PATH}/include/QNN ${CMAKE_CURRENT_LIST_DIR})
target_link_libraries(ggml-qnn PRIVATE ${QNN_LINK_LIBRARIES})
0 ggml/src/ggml-qnn.cpp → ggml/src/ggml-qnn/ggml-qnn.cpp
View file
File renamed without changes.

0 comments on commit 6d4feae

Please sign in to comment.