Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate llama.cpp in Aprapipes and a module ImageToTextXForm which can describe an image #345

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
+1,564 −136
Open

Integrate llama.cpp in Aprapipes and a module ImageToTextXForm which can describe an image #345

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
f64bdad
add llama vcpkg port
kushaljain-apra Apr 11, 2024
0f9f19d
update whisper portfile to support cuda
kushaljain-apra Apr 11, 2024
c864c13
add llm model abstract class
kushaljain-apra Apr 17, 2024
5524ad9
add encoder model abstract class
kushaljain-apra Apr 17, 2024
4365203
add Llava Model class
kushaljain-apra Apr 17, 2024
7b00c68
add Clip Encoder Class
kushaljain-apra Apr 17, 2024
bee2754
add Model Strategy Class
kushaljain-apra Apr 17, 2024
efd4082
add SceneDescriptorXform Module
kushaljain-apra Apr 17, 2024
c60e9ec
add unit tests
kushaljain-apra Apr 17, 2024
d099726
update cmakelists.txt
kushaljain-apra Apr 17, 2024
76d37ac
update framemetadata.h
kushaljain-apra Apr 17, 2024
926a07f
Updated code to resolve PR conversations
kushaljain-apra May 2, 2024
0a42a70
change SceneDescriptorXForm Module name to ImageToTextXForm Module
kushaljain-apra May 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update framemetadata.h
@kushaljain-apra
kushaljain-apra committed Apr 17, 2024
commit 76d37ac703cabebfa5e499d1ffada1fb67101e57
3 changes: 2 additions & 1 deletion base/include/FrameMetadata.h
View file
Original file line number Diff line number Diff line change
@@ -51,7 +51,8 @@ class FrameMetadata {
MOTION_VECTOR_DATA,
OVERLAY_INFO_IMAGE,
FACE_LANDMARKS_INFO,
TEXT
TEXT,
IMAGE_EMBEDDING
};

enum MemType