You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.USER: Write a limerick about python exceptionsUSER: ASSISTANT:
Llama.generate: 48 prefix-match hit, remaining 1 prompt tokens to eval
llama_perf_context_print: load time = 672844.44 ms
llama_perf_context_print: prompt eval time = 0.00 ms / 1 tokens ( 0.00 ms per token, inf tokens per second)
llama_perf_context_print: eval time = 0.00 ms / 32 runs ( 0.00 ms per token, inf tokens per second)
llama_perf_context_print: total time = 906.07 ms / 33 tokens
INFO: 127.0.0.1:40776 - "POST /v1/chat/completions HTTP/1.1" 200 OK
As you can read, there is an additional unwanted USER: at the end of the prompt.
I guess the chat format provided by --chat_format llava-1-5 is not correct.
Environment and Context
llama_cpp installed with pip install llama-cpp-python[server] print(llama_cpp.__version__): 0.3.6 print(openai.__version__): 1.59.7
The text was updated successfully, but these errors were encountered:
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Current Behavior
I'm running llama-server with following command:
(models downloaded from https://huggingface.co/mys/ggml_llava-v1.5-13b/tree/main)
When I call the server using openai python package:
The server output this in console:
As you can read, there is an additional unwanted
USER:
at the end of the prompt.I guess the chat format provided by
--chat_format llava-1-5
is not correct.Environment and Context
llama_cpp installed with
pip install llama-cpp-python[server]
print(llama_cpp.__version__)
: 0.3.6print(openai.__version__)
: 1.59.7The text was updated successfully, but these errors were encountered: