Natural Language Inference on MNLI (val)
92.13AccuracyMNLI + GNLI
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MNLI + GNLIModel Size=T5 XXL2024.02 | 92.13 | — | — | — | |
| MNLIModel Size=T5 XXL2024.02 | 92.11 | — | — | — | |
| ANLI + GNLIModel Size=T5 XXL2024.02 | 91.96 | — | — | — | |
| GNLIModel Size=T5 XXL2024.02 | 91.77 | — | — | — | |
| MNLI + GNLIModel Size=T5 LARGE2024.02 | 91.04 | — | — | — | |
| MNLIModel Size=T5 LARGE2024.02 | 90.83 | — | — | — | |
| GNLIModel Size=T5 LARGE2024.02 | 90.61 | — | — | — | |
| ANLI + GNLIModel Size=T5 LARGE2024.02 | 90.61 | — | — | — | |
| WANLI + GNLIModel Size=T5 XXL2024.02 | 90.34 | — | — | — | |
| ANLIModel Size=T5 XXL2024.02 | 90.01 | — | — | — | |
| WANLI + GNLIModel Size=T5 LARGE2024.02 | 87.8 | — | — | — | |
| ANLIModel Size=T5 LARGE2024.02 | 86.87 | — | — | — | |
| WANLIModel Size=T5 XXL2024.02 | 84.61 | — | — | — | |
| MNLIModel Size=T5 SMALL2024.02 | 83.37 | — | — | — | |
| MNLI + GNLIModel Size=T5 SMALL2024.02 | 82.66 | — | — | — | |
| GNLIModel Size=T5 SMALL2024.02 | 82.18 | — | — | — | |
| WANLIModel Size=T5 LARGE2024.02 | 81.1 | — | — | — | |
| WANLI + GNLIModel Size=T5 SMALL2024.02 | 78.02 | — | — | — | |
| ANLI + GNLIModel Size=T5 SMALL2024.02 | 72.89 | — | — | — | |
| ANLIModel Size=T5 SMALL2024.02 | 70.35 | — | — | — | |
| CoMERAbatch size=128, training stage=early stage2024.05 | 63.3 | 5.9 | 3.4 | — | |
| uncompressed trainingbatch size=1282024.05 | 62.2 | 256 | 253 | — | |
| CoMERAbatch size=128, training stage=late stage, target ratio=0.82024.05 | 62.2 | 4.9 | 2.4 | — | |
| CoMERAbatch size=128, training stage=late stage, target ratio=0.52024.05 | 62.1 | 3.9 | 1.4 | — | |
| CoMERAbatch size=128, training stage=late stage, target ratio=0.22024.05 | 61.5 | 3.2 | 0.7 | — | |
| WANLIModel Size=T5 SMALL2024.02 | 60.4 | — | — | — | |
| BERT-basemodel=BERT-base2026.02 | — | — | — | 0.11 | |
| Deep Ensembleensemble_size=52026.02 | — | — | — | 0.06 | |
| MC Dropoutp=0.1, type=uniform2026.02 | — | — | — | 0.09 | |
| MC Dropouttype=component-specific2026.02 | — | — | — | 0.109 | |
| SNGP2026.02 | — | — | — | 0.021 | |
| Temp. Scalingpost-hoc=true2026.02 | — | — | — | 0.028 | |
| UAT-LITE2026.02 | — | — | — | 0.067 | |
| VI-LastLayer2026.02 | — | — | — | 0.096 |