lmstudio_en
LM Studio is a multi-platform (macOS, Windows, Linux) chat application for large language models, which supports GGUF models (llama.cpp compatible). The following briefly introduces basic usage. Please check their official website for more information.
Create a new folder models, and organize the model file as follows (you can also use ln command here). For example, hfl/chinese-mixtral-instruct model can be organized as follows, where the GGUF model file should be placed.
- hfl
- chinese-mixtral-instruct
- ggml-model-q4_0.gguf
Click the last button. In local models folder, click change, navigate the models folder which was created in the previous step. After this step, all supported models will be listed. Please select Mistral Instruct as the chat template.
Note: Currently, Mixtral model is recognized as LLaMA, which does not affect the model inference.
Click the third button for chatting. You can choose the model on top of the interface. Right side bar shows system prompt, enabling GPU, context size, etc. Left side bar shows history conversations.
