Determine which models exist with API calls rather than hard coding them.

[pmetzger]

OpenAI at least seems to have an API call to determine what models are available. Rather than hard-coding the available models, the code should learn what they are at start time.

[karthink]

learn what they are at start time.

• I don't know what "start time" means, please see the discussion in Enumerate Ollama models on the server pointed to by :host directive #447.

  • I don't want to make network requests when the user's gptel configuration in their init file is loaded. I would be very annoyed if an Emacs package exhibited web-browser-like behavior like this.
  • Similarly, I don't want to make a network request when the user runs M-x gptel -- this should just open up a buffer.
  • Moreover, dedicated chat buffers are just one way to use gptel, and for many users it's not even the preferred way. So there is no guarantee M-x gptel will be called. So when should this API call be made?

• Model info fetched from most APIs (including OpenAI) does not have all the information we include, such as the costs, cutoff-dates, capabilities and context window sizes. Only some of this information is provided. We show this information as annotations when selecting a model:

• In contrast, it is a simple matter to add a model + metadata when OpenAI releases one, usually an interested user makes a pull request to gptel.

• If it's not yet in gptel, you can add models to any gptel-backend yourself:

;; Add model to backend:
(push 'gemini-2.0-flash-thinking-exp (gptel-backend-models gptel-backend))
;; Add model metadata (OPTIONAL)
(put 'gemini-2.0-flash-thinking-exp :description "Gemini model that produces...")
(put 'gemini-2.0-flash-thinking-exp :context-window 32)

where gptel-backend is the active gptel backend (Gemini in this example). In #529, we're trying to provide a more user-friendly way to do this -- although I think the above is pretty standard if you've used elisp.

(Finding available models automatically is more of a requirement for Ollama, where there is no standard list of models that gptel can track.)

[pmetzger]

I don't want to make network requests when the user's gptel configuration in their init file is loaded.

Fine, then when the user requests. It seems unreasonable to be writing these things by hand; it means that as new models are deployed you can't automatically keep up.

[pabl0]

How about a compromise: enumerate the list of available models from the API, and then add the hard-coded extra information for known ones, and list the rest with just the name? The API is quite disappointing in not providing almost any information other than the model name. Let's hope they improve it.

It is useful to see the pricing information listed, but it could be very much out-of-date if the user has not updated in a while, and OpenAI jacks up the price? It is not clear from end user perspective that this information is hard-coded.

[karthink]

How about a compromise: enumerate the list of available models from the API, and then add the hard-coded extra information for known ones, and list the rest with just the name? The API is quite disappointing in not providing almost any information other than the model name. Let's hope they improve it.

"from the API" is misleading. gptel supports at least fifteen different APIs. Many of these are variants of the OpenAI API, but the OpenAI API is itself not a standard. So they each have twists of their own that make it harder to cover fully. The endpoint for fetching the list of models and the corresponding response format is one such difference. That said, this is indeed the plan, at least for the major four APIs. That's why this issue remains open. There is currently a working implementation of this feature for the Ollama API in the feature-ollama-auto-update branch.

It is useful to see the pricing information listed, but it could be very much out-of-date if the user has not updated in a while, and OpenAI jacks up the price? It is not clear from end user perspective that this information is hard-coded.

Well, none of the APIs provide pricing information via an API call. So gptel can either hard-code it or not show it at all.