Natural Language Inference on ANLI R1 (test)
44.3AccuracyGLaM
Evaluation Results
| Method | Links | |
|---|---|---|
| GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=22021.12 | 44.3 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=One-shot2021.12 | 42.4 | |
| VariErr distributionEvaluation Model=RoBERTa2024.12 | 40.2 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=Zero-shot2021.12 | 39.2 | |
| GPT-3Model Size=175B, Evaluation Protocol=Few-shot, Shots=502021.12 | 36.8 | |
| VariErr Label-GuidedEvaluation Model=RoBERTa, Explanations=Model-generated2024.12 | 36.2 | |
| Mixtral (MJD) + human explanationsEvaluation Model=RoBERTa, Explanations=Human2024.12 | 36.1 | |
| ChaosNLI HJDEvaluation Model=RoBERTa2024.12 | 35.7 | |
| GPT-3Model Size=175B, Evaluation Protocol=Zero-shot2021.12 | 34.6 | |
| Mixtral (MJD) + human explanationsEvaluation Model=BERT, Explanations=Human2024.12 | 34.4 | |
| VariErr Label-GuidedEvaluation Model=BERT, Explanations=Model-generated2024.12 | 34 | |
| MNLI Label-GuidedEvaluation Model=RoBERTa, Explanations=Model-generated2024.12 | 32.9 | |
| GPT-3Model Size=175B, Evaluation Protocol=One-shot2021.12 | 32 | |
| MNLI distributionEvaluation Model=RoBERTa2024.12 | 31.7 | |
| VariErr distributionEvaluation Model=BERT2024.12 | 30.2 | |
| MNLI-FT-LMEvaluation Model=RoBERTa2024.12 | 29.2 | |
| MNLI Label-GuidedEvaluation Model=BERT, Explanations=Model-generated2024.12 | 27.5 | |
| ChaosNLI HJDEvaluation Model=BERT2024.12 | 26.8 | |
| Label-FreeEvaluation Model=BERT, Explanations=Model-generated2024.12 | 25.2 | |
| Mixtral (MJD)Evaluation Model=BERT, Explanations=None2024.12 | 24.2 | |
| Label-FreeEvaluation Model=RoBERTa, Explanations=Model-generated2024.12 | 24.2 | |
| Mixtral (MJD)Evaluation Model=RoBERTa, Explanations=None2024.12 | 23 | |
| MNLI distributionEvaluation Model=BERT2024.12 | 22.9 | |
| MNLI-FT-LMEvaluation Model=BERT2024.12 | 22 | |
| Out-of-the-box LMEvaluation Model=BERT2024.12 | 17 | |
| Out-of-the-box LMEvaluation Model=RoBERTa2024.12 | 16.7 |