Natural Language Inference on QNLI (test)
93.3AccuracyFine-tuning (full)
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Fine-tuning (full)Backbone=RoBERTa-large, Training Samples (K)=full, Evaluation Protocol=Full fine-tuning2021.08 | 93.3 | — | — | — | — | — | — | — | — | — | — | |
| Meta-training with Demonstration Retrievalmeta-trained=true2023.06 | 84.4 | — | — | — | — | — | — | — | — | — | — | |
| RAGmeta-trained=true2023.06 | 83.2 | — | — | — | — | — | — | — | — | — | — | |
| RAGmeta-trained=false2023.06 | 74.9 | — | — | — | — | — | — | — | — | — | — | |
| FewshotQAmeta-trained=true2023.06 | 71.8 | — | — | — | — | — | — | — | — | — | — | |
| iPET2023.06 | 70.3 | — | — | — | — | — | — | — | — | — | — | |
| LM-BFF2023.06 | 69.2 | — | — | — | — | — | — | — | — | — | — | |
| LM-BFFBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.08 | 68.3 | — | — | — | — | — | — | — | — | — | — | |
| DARTBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.08 | 66.7 | — | — | — | — | — | — | — | — | — | — | |
| P-TuningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.08 | 64.3 | — | — | — | — | — | — | — | — | — | — | |
| FewshotQAmeta-trained=false2023.06 | 61 | — | — | — | — | — | — | — | — | — | — | |
| Fine-tuningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=Few-shot fine-tuning2021.08 | 60.2 | — | — | — | — | — | — | — | — | — | — | |
| RoBERTa2023.06 | 60.2 | — | — | — | — | — | — | — | — | — | — | |
| Batch-ICLModel Size=13B, Search Optimal k (Best)=true2024.01 | 56.9 | — | — | — | — | — | — | — | — | — | — | |
| Batch-ICLModel Size=13B2024.01 | 56.7 | — | — | — | — | — | — | — | — | — | — | |
| F-OrderedModel Size=13B2024.01 | 56.2 | — | — | — | — | — | — | — | — | — | — | |
| Batch-ICLModel Size=7B, Search Optimal k (Best)=true2024.01 | 55.2 | — | — | — | — | — | — | — | — | — | — | |
| GPT-3 in-context learningBackbone=RoBERTa-large, Training Samples (K)=16, Evaluation Protocol=In-context learning2021.08 | 53.8 | — | — | — | — | — | — | — | — | — | — | |
| Batch-ICLModel Size=7B2024.01 | 52.4 | — | — | — | — | — | — | — | — | — | — | |
| F-OrderedModel Size=7B2024.01 | 51.3 | — | — | — | — | — | — | — | — | — | — | |
| PCWModel Size=13B2024.01 | 51.2 | — | — | — | — | — | — | — | — | — | — | |
| N-shot ICLModel Size=13B2024.01 | 51.1 | — | — | — | — | — | — | — | — | — | — | |
| Prompt-based zero-shotBackbone=RoBERTa-large, Training Samples (K)=0, Evaluation Protocol=Zero-shot2021.08 | 50.8 | — | — | — | — | — | — | — | — | — | — | |
| N-shot ICLModel Size=7B2024.01 | 50.7 | — | — | — | — | — | — | — | — | — | — | |
| PCWModel Size=7B2024.01 | 50.1 | — | — | — | — | — | — | — | — | — | — | |
| MajorityBackbone=RoBERTa-large, Training Samples (K)=0, Evaluation Protocol=Majority class2021.08 | 49.5 | — | — | — | — | — | — | — | — | — | — | |
| Majority2023.06 | 49.5 | — | — | — | — | — | — | — | — | — | — | |
| DSRM2023.06 | — | 90.1 | 27.6 | 37.1 | 65.4 | 247.2 | 20.4 | 76.7 | 312.4 | 59.2 | 176.3 | |
| Fine-tune2022.11 | — | 91.6 | 4.7 | 10.5 | — | — | — | — | — | — | — | |
| Fine-tune2023.06 | — | 90.6 | 5.8 | 10.9 | 94.2 | 161.9 | 3.5 | 96.1 | 216.5 | 88 | 98.4 | |
| Flooding-Xtraining_type=adversarial training2023.06 | — | 90.8 | 25.6 | 29.4 | 71.3 | 232.7 | 18.7 | 79.2 | 294.6 | 67.5 | 137.1 | |
| FreeLB2022.11 | — | 90.5 | 12.8 | 12 | — | — | — | — | — | — | — | |
| FreeLBtraining_type=adversarial training2023.06 | — | 90.7 | 12.8 | 29.8 | 85.3 | 189.4 | 21.4 | 76.8 | 324.2 | 69.3 | 143.9 | |
| FreeLB++method_category=SOTA defense2023.06 | — | 91.1 | 16.4 | 30.2 | 81.4 | 193.7 | 20.7 | 77 | 301.7 | 66.7 | 150.1 | |
| InfoBERT2022.11 | — | 91.5 | 16.4 | 20.9 | — | — | — | — | — | — | — | |
| InfoBERTmethod_category=SOTA defense2023.06 | — | 90.4 | 18 | 15.4 | 82.5 | 212.9 | 13.1 | 85.8 | 270.2 | 83.9 | 127.9 | |
| PGDtraining_type=adversarial training2023.06 | — | 90.6 | 14.3 | 27.9 | 81.2 | 201.6 | 17.3 | 80.6 | 268.9 | 67.8 | 134.6 | |
| RobustTSparsity=30%2022.11 | — | 91.5 | 17 | 25.9 | — | — | — | — | — | — | — |