[Bug]: o1 model in LibreChat UI does not work properly with vision inputs.

[rohit901]

What happened?

I was using the o1 model to understand a technical concept, and my initial messages were just in plain text. In the middle I passed some screenshots along with my prompt to the o1 model in LibreChat UI, but it didn't work as intended and the model seems to be different too based on different color of the icon.
Refer to the below screenshot and the garbage response produced by the model:

o1 API should support vision inputs so the above should not be happening. I wonder what is happening behind the scenes in LibreChat when we pass in attachments in our chat session?

long back I had noticed that whenever I used any anthropic model and used attachment/files, I would see usage of two different models. I mean if I had selected sonnet in the dropdown and use some attachment with the prompt like image,file, then there would be usage of both sonnet and haiku in my API dashboard. So that makes me question what exactly might be happening behind the scene when you use attachment with Librechat UI?

I started a brand new chat session with same prompt and it still gave me a garbage response. Refer the below screenshot.

I discussed about this issue on discord server with @berry-13 as well, and he confirmed that gpt-4o model is being used in the backend instead of o1. Please refer the below screenshot.

I hope this issue can be fixed and looking forward to more clarity on why this issue happened in the first place.

Steps to Reproduce

1. Use o1 model in LibreChat in v0.7.6 and attach two images.
2. Ask the model to reason or answer the query based on contents of the attached images.
3. It will produce garbage output.

What browsers are you seeing the problem on?

No response

Relevant log output

No response

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct