Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MNLI to Full Benchmark #33

Open
epavlick opened this issue Aug 10, 2021 · 3 comments
Open

Add MNLI to Full Benchmark #33

epavlick opened this issue Aug 10, 2021 · 3 comments
Labels
few_shot Benchmark Tasks for Few-Shot Generalization

Comments

@epavlick
Copy link
Contributor

coordinate with whoever is working on SuperGLUE, we only need to include MNLI once. But NLI will be held-out from model training (whereas the other SuperGLUE tasks will not) so interpreting MNLI results is different from other superglue tasks.

use to test generalization to unseen task; maybe use FLEX?

@epavlick epavlick added the few_shot Benchmark Tasks for Few-Shot Generalization label Aug 10, 2021
@PierreColombo
Copy link

I can do it :)

@wilsonyhlee
Copy link
Member

@PierreColombo I'd love to help contribute to this one if you need any help!

@manandey
Copy link
Member

MNLI in PS here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
few_shot Benchmark Tasks for Few-Shot Generalization
Projects
None yet
Development

No branches or pull requests

4 participants