-
Notifications
You must be signed in to change notification settings - Fork 324
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(message): update message api types #199
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Amazing work, that was a big one!
Couple comments while I wait for the final commits
…rking, gemini, xai, cohere remains
…/message-api-type
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find the PR super big, we can try to create smaller PRs to cover these changes progressively.
I left some comments
…/message-api-type
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Excellent work @0xMochan !
Found a couple bugs but overall this looks 🔥
I really need this PR in order to finish the tool calls issue (send the tool response back to the model to get human readable answers). |
This is planned to merge in 24 hours! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couple of docs changes but after that let's send it! Amazing work! 🔥
LGTM!!!! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Send it! 🚀
This PR breaks DeepSeek provider, the problems is how DeepSeek provider builds the client, I'll fix this on my next PR |
completion::Message
refactorThis PR performs a very large refactor for how messages are handled. There are 2 parts that are combined to allow for full API access to all providers.
Message
structThis PR defines a general
Message
definition that serves to be the singular data point for message histories alongside the interface for prompting moving forward. This interface encapsulates all of the different input and responses you might get when conversing with any provider, including multi-modal inputs (images, etc), tool calls and results, and assistant powered replies. It covers a vast majority of provider specific and general features.From
andTryFrom
implementationsEach provider now has it's own set of
Message
structs representing a closer model of the provider specific API. The provider is responsible for implementing aFrom
and/orTryFrom
to convert between the generic message <-> provider specific message converting the data between the formats.Prompts are supplied as
Into<Message>
and message history areMessage
. Providers then convert this into a single stream of their ownMessage
to then pass to the provider's API calls. When a response is given, the reverse is made.Future
In the future, there will be more implementations of
Into<Message>
and other abstractions that'll build upon this pattern. This includes further abstractions to make working with images, documents, and audio with specific provider apis more seamless, making tool calling loops easier (prompt -> assistant -> tool call -> tool result -> assistant -> response), and easier management of chat histories.Caveats
This PR has gotten too large, but to avoid it growing even further, some concessions were made. These should result in their own issues for further discussion:
EternalAI
currently is able to accidentally have 2 system messages which would cause an issue with the provider.completion::completion2
is poorly named (can be resolved in the current PR).message::UserContent(Text)
,message::AssistantContent(Text)
, andmessage::ToolResultContent(Text)
take inmessage::Text
but could directly take in aString
for simplicityToolResult
is placed underUserContent
instead of another top-level enum variant.