Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

supports jinaai/jina-embeddings-v2-base-code #8

Open
wsxiaoys opened this issue Feb 5, 2024 · 1 comment
Open

supports jinaai/jina-embeddings-v2-base-code #8

wsxiaoys opened this issue Feb 5, 2024 · 1 comment

Comments

@wsxiaoys
Copy link

wsxiaoys commented Feb 5, 2024

I seems to get jina embeddings convert successfully with

python bert_cpp/convert.py jinaai/jina-embeddings-v2-base-code models/jina-f16.gguf

Seems the only change to original bert is ALiBi, as described in https://huggingface.co/jinaai/jina-embeddings-v2-base-code

It'll be nice if we could adapt ggml_alibi into this repo for jina embedding support

@iamlemec
Copy link
Owner

iamlemec commented Feb 6, 2024

Yeah, this would be rad. It does seem like it comes down to adding one or two lines with ggml_alibi. The only concern right now is that we're not using a KV cache, so going up to L=8192 is probably going to be infeasible due to memory. But I do plan on adding in KV cache soon. Let me see if I can get it working in the meantime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants