Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use of anyio vs queue object #112

Open
lppier opened this issue Jun 23, 2024 · 0 comments
Open

use of anyio vs queue object #112

lppier opened this issue Jun 23, 2024 · 0 comments

Comments

@lppier
Copy link

lppier commented Jun 23, 2024

Hi, firstly, thank you so much for doing this.
this is not an issue, not sure where else i could post this. If there is a better place, pls let me know.

i saw that anyio was used in this manner to store the streaming message objects
https://github.com/run-llama/sec-insights/blob/main/backend/app/api/endpoints/conversation.py#L96

Wondering if there are any advantages using this vs using a python asynchronous queue object?
I'm asking because in my company policy bot's RAG implementation i'm currently storing it in a python queue object.

Does using anyio have any latency benefits?

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant