Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MMMU和MathVista在gpt-4o的结果 #712

Open
kydxh opened this issue Jan 6, 2025 · 1 comment
Open

MMMU和MathVista在gpt-4o的结果 #712

kydxh opened this issue Jan 6, 2025 · 1 comment

Comments

@kydxh
Copy link

kydxh commented Jan 6, 2025

想请问下MMMU和MathVista上是怎么设置的prompt得到的69.9 | 62.7的结果（GPT-4o-20240806）？因为我在让模型直接回答问题时获得的结果远低于这两个数值，在用上COT之后才能接近，但是看起来在[VLMEvalKit]中并没有使用COT？或者除了prompt设置有其他单独的设置？
@Violettttee
Copy link

想请问下MMMU和MathVista上是怎么设置的prompt得到的69.9 | 62.7的结果（GPT-4o-20240806）？因为我在让模型直接回答问题时获得的结果远低于这两个数值，在用上COT之后才能接近，但是看起来在[VLMEvalKit]中并没有使用COT？或者除了prompt设置有其他单独的设置？

#702
我也问过这个问题..但是我看回复文件里面4o的明显是有cot的（如果指令中没有直出的要求的话，4o走cot可能是因为system prompt）

Clipboard_Screenshot_1736223208

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kydxh @Violettttee