Natural Language Inference on MNLI mm
90.7AccuracyDeBERTaV3_base
Evaluation results
| Method | Links | |
|---|---|---|
| DeBERTaV3_baseLayers=12, Hidden size=768, Heads=12, Vocabulary Size(K)=128, Backbone #Params(M)=862021.11 | 90.7 | |
| DeBERTa_baseLayers=12, Hidden size=768, Heads=12, Vocabulary Size(K)=50, Backbone #Params(M)=1002021.11 | 88.5 | |
| DeBERTaV3_xsmallLayers=12, Hidden size=384, Heads=6, Vocabulary Size(K)=128, Backbone #Params(M)=222021.11 | 88.3 | |
| DeBERTaV3_smallLayers=6, Hidden size=768, Heads=12, Vocabulary Size(K)=128, Backbone #Params(M)=442021.11 | 87.9 | |
| BERT_baseLayers=12, Hidden size=768, Heads=12, Vocabulary Size(K)=30, Backbone #Params(M)=862021.11 | 84.7 | |
| Fine-tuningn=256, Update model parameters=true2022.12 | 78.7 | |
| Se²2024.02 | 69.8 | |
| DecTn=256, Update model parameters=false2022.12 | 69.4 | |
| Fine-tuningn=64, Update model parameters=true2022.12 | 67.7 | |
| OPT-IML 175BEvaluation protocol=few-shot, k=52022.12 | 64.9 | |
| UPRISE2024.02 | 64.8 | |
| DecTn=64, Update model parameters=false2022.12 | 63.3 | |
| OPT-IML 175BEvaluation protocol=0-shot2022.12 | 62.8 | |
| FLAN 137BEvaluation protocol=few-shot, k=variable, Training strategy=leave-one-category-out2022.12 | 61 | |
| OPT-IML 30BEvaluation protocol=0-shot2022.12 | 61 | |
| OPT-IML 30BEvaluation protocol=few-shot, k=52022.12 | 56.3 | |
| FLAN 137BEvaluation protocol=0-shot, Training strategy=leave-one-category-out2022.12 | 51 | |
| LaMDA-PT 137BEvaluation protocol=few-shot, k=variable2022.12 | 43.8 | |
| Best-of-102024.02 | 40.2 | |
| Instructor2024.02 | 40.1 | |
| SBERT2024.02 | 38.1 | |
| LaMDA-PT 137BEvaluation protocol=0-shot2022.12 | 37 | |
| BM252024.02 | 36.8 | |
| Zero-shot2024.02 | 36.6 | |
| OPT 30BEvaluation protocol=few-shot, k=52022.12 | 35.5 | |
| Random2024.02 | 35.5 | |
| OPT 175BEvaluation protocol=few-shot, k=52022.12 | 32.9 | |
| OPT 30BEvaluation protocol=0-shot2022.12 | 31.8 | |
| OPT 175BEvaluation protocol=0-shot2022.12 | 31.8 | |
| AES2024.02 | 29.5 |