Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Evaluation prompts #6

Open
paulaonta opened this issue Feb 19, 2024 · 1 comment
Open

Evaluation prompts #6

paulaonta opened this issue Feb 19, 2024 · 1 comment

Comments

@paulaonta
Copy link

Hello!
I'm trying to replicate your model, and while evaluating the model, I noticed that the prompts don't match those in the paper. Are these the final scripts?
Thank you
@Victorwz
Copy link
Owner

Hi, one of the 9 prompts for color reasoning in code is different from the paper appendix. I will update the paper accordingly. Please use the prompts in evaluation_scripts/verify_color_prediction.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Victorwz @paulaonta