Natural Language Inference on RTE (test)
90.25AccuracyHiddenKey
Evaluation results
| Method | Links | |
|---|---|---|
| HiddenKeyBackbone=LLaMA2-7B, Evaluation Protocol=Finetuned2024.02 | 90.25 | |
| ALIGNModel Variant=large, Parameters=355M, Evaluation Protocol=Zero-shot2023.07 | 89.9 | |
| baselineBackbone=LLaMA2-7B, Evaluation Protocol=Finetuned2024.02 | 88.45 | |
| FLAN-T5Model Variant=large, Parameters=780M, Evaluation Protocol=Zero-shot2023.07 | 87 | |
| ALIGNModel Variant=base, Parameters=125M, Evaluation Protocol=Zero-shot2023.07 | 83.4 | |
| FLAN-T5Model Variant=base, Parameters=250M, Evaluation Protocol=Zero-shot2023.07 | 81.6 | |
| LoRA-nBARBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.10 | 81 | |
| Batch-ICLModel Size=13B, Search Optimal k (Best)=true2024.01 | 80.9 | |
| LoRA-SAMBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.10 | 80.6 | |
| F-OrderedModel Size=13B2024.01 | 80.5 | |
| LoRA-oBARBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.10 | 79.7 | |
| Batch-ICLModel Size=13B2024.01 | 79.2 | |
| N-shot ICLModel Size=13B2024.01 | 78.1 | |
| LoRABackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=few-shot2024.10 | 77.5 | |
| Batch-ICLModel Size=7B, Search Optimal k (Best)=true2024.01 | 67.6 | |
| RETROPROMPTShot=16-shot, Source Domain=QQP2022.05 | 67.3 | |
| RETROPROMPTshot=16-shot, source_dataset=QQP, domain_role=Target Domain2025.12 | 67.3 | |
| LM-BFF (D-demo)Shot=16-shot, Source Domain=QQP2022.05 | 66.2 | |
| LM-BFF (D-demo)shot=16-shot, source_dataset=QQP, domain_role=Target Domain2025.12 | 66.2 | |
| KPTShot=16-shot, Source Domain=QQP2022.05 | 65.8 | |
| KPTshot=16-shot, source_dataset=QQP, domain_role=Target Domain2025.12 | 65.8 | |
| LM-BFF (man)Shot=16-shot, Source Domain=QQP2022.05 | 65.5 | |
| LM-BFF (man)shot=16-shot, source_dataset=QQP, domain_role=Target Domain2025.12 | 65.5 | |
| Batch-ICLModel Size=7B2024.01 | 65.2 | |
| Coherence Boosting (GPT-3 175B)alpha=-0.62021.10 | 60.29 | |
| FedAvgProportion of Attackers=0%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 59.9 | |
| RFLPAProportion of Attackers=0%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 59.6 | |
| BREAProportion of Attackers=10%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 59.2 | |
| BREAProportion of Attackers=0%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 58.4 | |
| RFLPAProportion of Attackers=10%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 58.2 | |
| RFLPAProportion of Attackers=20%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 58.2 | |
| RFLPAProportion of Attackers=30%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 57.7 | |
| GPT-3 175Balpha=-12021.10 | 57.4 | |
| BREAProportion of Attackers=20%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 57 | |
| BREAProportion of Attackers=30%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 56.7 | |
| PCWModel Size=7B2024.01 | 56.6 | |
| PCWModel Size=13B2024.01 | 56.5 | |
| GPT-3 175Balpha=02021.10 | 55.96 | |
| F-OrderedModel Size=7B2024.01 | 53.9 | |
| Coherence Boosting (GPT-2 Small)alpha=-0.32021.10 | 53.79 | |
| N-shot ICLModel Size=7B2024.01 | 53 | |
| Zero-ShotBackbone=RoBERTa-large, Number of Parameters=355M, Evaluation Protocol=zero-shot2024.10 | 51.4 | |
| GPT-2 Small (125M)alpha=02021.10 | 51.26 | |
| FedAvgProportion of Attackers=10%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 50.9 | |
| GPT-2 Small (125M)alpha=-12021.10 | 49.82 | |
| GPT-2 XL (1.6B)alpha=02021.10 | 49.1 | |
| Coherence Boosting (GPT-2 XL)alpha=0.92021.10 | 49.1 | |
| GPT-2 XL (1.6B)alpha=-12021.10 | 48.74 | |
| FedAvgProportion of Attackers=20%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 48.7 | |
| FTShot=16-shot, Source Domain=QQP2022.05 | 48 | |
| FTshot=16-shot, source_dataset=QQP, domain_role=Target Domain2025.12 | 48 | |
| FedAvgProportion of Attackers=30%, Backbone=DistilBERT, Attack Type=Gradient Manipulation2024.05 | 46.2 |