Natural Language Inference on ANLI R3 (test)
44.7AccuracyGLaM
Evaluation Results
| Method | Links | |
|---|---|---|
| GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=42021.12 | 44.7 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=Zero-shot2021.12 | 41.3 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=One-shot2021.12 | 40.8 | |
| GPT-3Model Size=175B, Evaluation Protocol=Few-shot, Shots=502021.12 | 40.2 | |
| GPT-3Model Size=175B, Evaluation Protocol=One-shot2021.12 | 35.1 | |
| GPT-3Model Size=175B, Evaluation Protocol=Zero-shot2021.12 | 34.5 | |
| ChaosNLI HJDEvaluation Model=RoBERTa2024.12 | 33.8 | |
| ChaosNLI HJDEvaluation Model=BERT2024.12 | 33.2 | |
| VariErr distributionEvaluation Model=RoBERTa2024.12 | 32.1 | |
| Mixtral (MJD) + human explanationsEvaluation Model=BERT, Explanations=Human2024.12 | 32 | |
| VariErr distributionEvaluation Model=BERT2024.12 | 31.9 | |
| VariErr Label-GuidedEvaluation Model=BERT, Explanations=Model-generated2024.12 | 31.7 | |
| MNLI Label-GuidedEvaluation Model=BERT, Explanations=Model-generated2024.12 | 30.3 | |
| Mixtral (MJD) + human explanationsEvaluation Model=RoBERTa, Explanations=Human2024.12 | 30 | |
| VariErr Label-GuidedEvaluation Model=RoBERTa, Explanations=Model-generated2024.12 | 29.6 | |
| MNLI-FT-LMEvaluation Model=BERT2024.12 | 29.3 | |
| MNLI Label-GuidedEvaluation Model=RoBERTa, Explanations=Model-generated2024.12 | 29.2 | |
| MNLI distributionEvaluation Model=RoBERTa2024.12 | 28.1 | |
| MNLI distributionEvaluation Model=BERT2024.12 | 27.9 | |
| MNLI-FT-LMEvaluation Model=RoBERTa2024.12 | 25.7 | |
| Label-FreeEvaluation Model=BERT, Explanations=Model-generated2024.12 | 25.5 | |
| Mixtral (MJD)Evaluation Model=BERT, Explanations=None2024.12 | 24.6 | |
| Mixtral (MJD)Evaluation Model=RoBERTa, Explanations=None2024.12 | 24.3 | |
| Label-FreeEvaluation Model=RoBERTa, Explanations=Model-generated2024.12 | 24.3 | |
| Out-of-the-box LMEvaluation Model=BERT2024.12 | 19.7 | |
| Out-of-the-box LMEvaluation Model=RoBERTa2024.12 | 16.8 |