List your company
ResearchFormerly WizWand

Natural Language Inference on MNLI (test)

0.898AccuracyFine-tuning (full)
0.2983360.4540180.60970.765382Aug 30, 2021May 11, 2022Jan 21, 2023Oct 2, 2023Jun 13, 2024Feb 22, 2025Nov 4, 2025
Updated 6mo ago

Evaluation results

MethodLinks
Fine-tuning (full)Backbone=RoBERTa-large, Training Samples (K)=full, Evaluation Protocol=Full fine-tuning2021.080.898
LM-Cocktail10Fine-tune on=MNLI2023.110.8923
LM-Cocktail2Fine-tune on=MNLI2023.110.8888
Fine-tunedFine-tune on=MNLI2023.110.879
CoFi PruningUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=2.7x, Parameters=26M2022.040.849
BERTbase (teacher)Unlabeled data (U)=false, Task-specific (T)=false, Inference speedup=1.0x, Parameters=85M2022.040.848
TinyBERT6Unlabeled data (U)=true, Task-specific (T)=true, Inference speedup=2.0x, Parameters=43M2022.040.84
MobileBERT†Unlabeled data (U)=true, Task-specific (T)=false, Inference speedup=2.3x, Parameters=20M, Teacher Model=Different teacher model used2022.040.839
Non-privateBackbone=Roberta, Protocol=LoRA finetuning2025.100.8373
Block PruningUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=2.7x, Parameters=25M2022.040.837
DistillBERT6Unlabeled data (U)=true, Task-specific (T)=false, Inference speedup=2.0x, Parameters=43M2022.040.822
AdamWBackbone=RoBERTa-large, Optimization Method=AdamW, Finetuning Protocol=Few-shot, Iterations=10K2025.110.814
Movement PruningUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=1.0x, Parameters=9M2022.040.812
CoFi PruningUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=12.1x, Parameters=4.4M2022.040.806
TinyBERT4Unlabeled data (U)=true, Task-specific (T)=true, Inference speedup=11.4x, Parameters=4.7M2022.040.788
LoRA-SAMBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.100.787
LKDEvaluation Protocol=Fine-tuning (FT)2023.050.783
LKD+Init.Evaluation Protocol=Fine-tuning (FT)2023.050.783
LoRA-oBARBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.100.783
ABEX2024.060.7825
AutoTinyBERTUnlabeled data (U)=true, Task-specific (T)=true, Inference speedup=9.1x, Parameters=3.3M2022.040.782
LoRABackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.100.781
LoRA-nBARBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.100.781
DynaBERTUnlabeled data (U)=false, Task-specific (T)=true, Inference speedup=6.3x, Parameters=11M2022.040.763
LKD+Init.Evaluation Protocol=Adapted Prompting (AP)2023.050.76
Gold-only2024.060.7575
NoiseCurveepsilon=8, Backbone=Roberta, Protocol=LoRA finetuning2025.100.7341
ConMeZOBackbone=RoBERTa-large, Optimization Method=ConMeZO, Finetuning Protocol=Few-shot, Iterations=10K, Smoothing parameter=10^-32025.110.732
Genius2024.060.7126
EDA2024.060.709
MeZO+MomentumBackbone=RoBERTa-large, Optimization Method=MeZO+Momentum (Mom.), Finetuning Protocol=Few-shot, Iterations=10K, Smoothing parameter=10^-32025.110.707
MeZOBackbone=RoBERTa-large, Optimization Method=MeZO, Finetuning Protocol=Few-shot, Iterations=10K, Smoothing parameter=10^-32025.110.697
DP-BANDMFepsilon=8, Backbone=Roberta, Protocol=LoRA finetuning2025.100.6957
LM-BFFBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.080.683
DARTBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.080.675
NoiseCurveepsilon=5, Backbone=Roberta, Protocol=LoRA finetuning2025.100.6718
DP-BANDMFepsilon=5, Backbone=Roberta, Protocol=LoRA finetuning2025.100.638
DP-SGDepsilon=8, Backbone=Roberta, Protocol=LoRA finetuning2025.100.6377
P-TuningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.080.615
LKDEvaluation Protocol=Adapted Prompting (AP)2023.050.565
DP-SGDepsilon=5, Backbone=Roberta, Protocol=LoRA finetuning2025.100.5473
NoiseCurveepsilon=2, Backbone=Roberta, Protocol=LoRA finetuning2025.100.5387
GPT-3 in-context learningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=In-context learning2021.080.52
Prompt-based zero-shotBackbone=RoBERTa-large, Training Samples (K)=0, Evaluation Protocol=Zero-shot2021.080.508
DP-BANDMFepsilon=2, Backbone=Roberta, Protocol=LoRA finetuning2025.100.5072
Zero-ShotBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=zero-shot2024.100.488
Demo.Evaluation Protocol=Demonstration Learning, Mode=Zero-shot2023.050.471
DP-SGDepsilon=2, Backbone=Roberta, Protocol=LoRA finetuning2025.100.4617
Fine-tuningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.080.458
PromptEvaluation Protocol=Manual Prompting, Mode=Zero-shot2023.050.444
MajorityBackbone=RoBERTa-large, Training Samples (K)=0, Evaluation Protocol=Majority class2021.080.327
LlamaModel Variant=Base model2023.110.3214