Natural Language Inference on MNLI (test)
0.898AccuracyFine-tuning (full)
Evaluation Results
| Method | Links | |
|---|---|---|
| Fine-tuning (full)Backbone=RoBERTa-large, Training Samples (K)=full, Evaluation Protocol=Full fine-tuning2021.08 | 0.898 | |
| LM-Cocktail10Fine-tune on=MNLI2023.11 | 0.8923 | |
| LM-Cocktail2Fine-tune on=MNLI2023.11 | 0.8888 | |
| Fine-tunedFine-tune on=MNLI2023.11 | 0.879 | |
| CoFi PruningUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=2.7x, Parameters=26M2022.04 | 0.849 | |
| BERTbase (teacher)Unlabeled data (U)=false, Task-specific (T)=false, Inference speedup=1.0x, Parameters=85M2022.04 | 0.848 | |
| TinyBERT6Unlabeled data (U)=true, Task-specific (T)=true, Inference speedup=2.0x, Parameters=43M2022.04 | 0.84 | |
| MobileBERT†Unlabeled data (U)=true, Task-specific (T)=false, Inference speedup=2.3x, Parameters=20M, Teacher Model=Different teacher model used2022.04 | 0.839 | |
| Non-privateBackbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.8373 | |
| Block PruningUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=2.7x, Parameters=25M2022.04 | 0.837 | |
| DistillBERT6Unlabeled data (U)=true, Task-specific (T)=false, Inference speedup=2.0x, Parameters=43M2022.04 | 0.822 | |
| AdamWBackbone=RoBERTa-large, Optimization Method=AdamW, Finetuning Protocol=Few-shot, Iterations=10K2025.11 | 0.814 | |
| Movement PruningUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=1.0x, Parameters=9M2022.04 | 0.812 | |
| CoFi PruningUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=12.1x, Parameters=4.4M2022.04 | 0.806 | |
| TinyBERT4Unlabeled data (U)=true, Task-specific (T)=true, Inference speedup=11.4x, Parameters=4.7M2022.04 | 0.788 | |
| LoRA-SAMBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.10 | 0.787 | |
| LKDEvaluation Protocol=Fine-tuning (FT)2023.05 | 0.783 | |
| LKD+Init.Evaluation Protocol=Fine-tuning (FT)2023.05 | 0.783 | |
| LoRA-oBARBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.10 | 0.783 | |
| ABEX2024.06 | 0.7825 | |
| AutoTinyBERTUnlabeled data (U)=true, Task-specific (T)=true, Inference speedup=9.1x, Parameters=3.3M2022.04 | 0.782 | |
| LoRABackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.10 | 0.781 | |
| LoRA-nBARBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.10 | 0.781 | |
| DynaBERTUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=6.3x, Parameters=11M2022.04 | 0.763 | |
| LKD+Init.Evaluation Protocol=Adapted Prompting (AP)2023.05 | 0.76 | |
| Gold-only2024.06 | 0.7575 | |
| NoiseCurveepsilon=8, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.7341 | |
| ConMeZOBackbone=RoBERTa-large, Optimization Method=ConMeZO, Finetuning Protocol=Few-shot, Iterations=10K, Smoothing parameter=10^-32025.11 | 0.732 | |
| Genius2024.06 | 0.7126 | |
| EDA2024.06 | 0.709 | |
| MeZO+MomentumBackbone=RoBERTa-large, Optimization Method=MeZO+Momentum (Mom.), Finetuning Protocol=Few-shot, Iterations=10K, Smoothing parameter=10^-32025.11 | 0.707 | |
| MeZOBackbone=RoBERTa-large, Optimization Method=MeZO, Finetuning Protocol=Few-shot, Iterations=10K, Smoothing parameter=10^-32025.11 | 0.697 | |
| DP-BANDMFepsilon=8, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.6957 | |
| LM-BFFBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.08 | 0.683 | |
| DARTBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.08 | 0.675 | |
| NoiseCurveepsilon=5, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.6718 | |
| DP-BANDMFepsilon=5, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.638 | |
| DP-SGDepsilon=8, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.6377 | |
| P-TuningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.08 | 0.615 | |
| LKDEvaluation Protocol=Adapted Prompting (AP)2023.05 | 0.565 | |
| DP-SGDepsilon=5, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.5473 | |
| NoiseCurveepsilon=2, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.5387 | |
| GPT-3 in-context learningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=In-context learning2021.08 | 0.52 | |
| Prompt-based zero-shotBackbone=RoBERTa-large, Training Samples (K)=0, Evaluation Protocol=Zero-shot2021.08 | 0.508 | |
| DP-BANDMFepsilon=2, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.5072 | |
| Zero-ShotBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=zero-shot2024.10 | 0.488 | |
| Demo.Evaluation Protocol=Demonstration Learning, Mode=Zero-shot2023.05 | 0.471 | |
| DP-SGDepsilon=2, Backbone=Roberta, Protocol=LoRA finetuning2025.10 | 0.4617 | |
| Fine-tuningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.08 | 0.458 | |
| PromptEvaluation Protocol=Manual Prompting, Mode=Zero-shot2023.05 | 0.444 | |
| MajorityBackbone=RoBERTa-large, Training Samples (K)=0, Evaluation Protocol=Majority class2021.08 | 0.327 | |
| LlamaModel Variant=Base model2023.11 | 0.3214 |