Issues of getting response from my desktop

[LukeW2002]

Hey got an LLM running on my desktop listening on all ports. For some reason I cannot get gen.nvim to give me a response.

[pbrunier]

My first guess is that you didn't select the model correctly in the config. That's what I encountered anyways.
Give this one a try. As soon as I did it this way I got a response:
model = "llama3.1:latest",

[LukeW2002]

I have tried that. When I run curl -X POST http://**computersIP**:11434/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model": "yi-coder:latest", "messages": [{"role": "user", "content": "Write a hello world program in Python"}], "stream": true}'
I get a full response

now I have this in my configs:
use {'David-Kunz/gen.nvim', opts= { model = 'yi-coder:latest', host = "192.168.0.24", debug = true, port = "11434", command = function(options) local body = vim.json.encode({ model = options.model, messages = {{ role = "user", content = options.prompt }}, stream = true }) return string.format( "curl --silent --no-buffer -X POST http://%s:%d/v1/chat/completions -H 'Content-Type: application/json' -d '%s'", options.host, options.port, body ) end, }}

I get no response.

[pbrunier]

@LukeW2002

I first encountered the same after switching to the model you tried. yi-coder:latest
No errors and no output.
But after running the model directly in Ollama, the model got downloaded and somehow registered.
ollama run yi-coder:latest

Now it runs fine in nvim.

My config looks like this:

{
"David-Kunz/gen.nvim",
opts = {
model = "yi-coder:latest", -- The default model to use.
quit_map = "q", -- set keymap for close the response window
retry_map = "", -- set keymap to re-send the current prompt
accept_map = "", -- set keymap to replace the previous selection with the last result
host = "127.0.0.1", -- The host running the Ollama service.
port = "11434", -- The port on which the Ollama service is listening.
display_mode = "float", -- The display mode. Can be "float" or "split" or "horizontal-split".
show_prompt = false, -- Shows the prompt submitted to Ollama.
show_model = false, -- Displays which model you are using at the beginning of your chat session.
no_auto_close = false, -- Never closes the window automatically.
hidden = false, -- Hide the generation window (if true, will implicitly set prompt.replace = true), requires Neovim >= 0.10
init = function(options)
pcall(io.popen, "ollama serve > /dev/null 2>&1 &")
end,
command = function(options)
local body = { model = options.model, stream = true }
return "curl --silent --no-buffer -X POST http://"
.. options.host
.. ":"
.. options.port
.. "/api/chat -d $body"
end,
debug = false, -- Prints errors and the command which is run.
},
},

[Glavnokoman]

Same here. The ollama logs get
[GIN] 2024/09/16 - 16:42:44 | 404 | 617.133µs | 127.0.0.1 | POST "/api/chat"
for every request (and nothing else).
Running the models directly using ollama run works just fine

[David-Kunz]

Hi @Glavnokoman ,

Interesting, can you add debug = true to your options and check the output?

Thanks and best regards,
David

[linuxcult]

Having the same issue on one of my laptops. Did also turn on debug = true but no response. Changed LLM from "llama3.1" to "llama3.1:latest" with no effect still no output. The translation window opens it stays empty. debug or not.

EDIT
I pulled the "llama3.1:latest" LLM for ollama and put it in the config file ("llama3.1:latest") and now it works!? I'm sure I tried this before.