Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(EAI-626): Send slack message on user comment to feedback channel #601

Open
wants to merge 40 commits into
base: main
Choose a base branch
from

Conversation

mongodben
Copy link
Collaborator

@mongodben mongodben commented Jan 24, 2025

Jira: https://jira.mongodb.org/browse/EAI-626

Changes

  • Send Slack message for user feedback
  • Add slack token to staging environment (not in PR)

Notes

  • We rerun the LLM-as-a-judge evals for this. This means that LLM-as-a-judge will rerun for all commented messages (and the subset of all messages that are also commented). This is not compute efficient but given the small number of feedback messages we get I think it's acceptable, in order to separate concerns.
  • Example of message

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant