Sentiment Analysis on SST-2 (test) (Adversarial Robustness)
96.43Clean AccuracyVanilla
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| VanillaBackbone=Llama2-7B2024.02 | 96.43 | — | — | — | — | — | — | — | — | 98.58 | — | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Llama2-7B2024.02 | 96.21 | — | — | — | — | — | — | — | — | 16.56 | — | — | — | — | — | — | — | — | — | — | — | |
| STRIPBackbone=Llama2-7B2024.02 | 95.99 | — | — | — | — | — | — | — | — | 57.24 | — | — | — | — | — | — | — | — | — | — | — | |
| STRIPBackbone=Llama2-7B2024.02 | 95.66 | — | — | — | — | — | — | — | — | 97.48 | — | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Llama2-7B2024.02 | 95.61 | — | — | — | — | — | — | — | — | 93.86 | — | — | — | — | — | — | — | — | — | — | — | |
| STRIPBackbone=Llama2-7B2024.02 | 95.44 | — | — | — | — | — | — | — | — | 95.18 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Llama2-7B2024.02 | 95.39 | — | — | — | — | — | — | — | — | 98.13 | — | — | — | — | — | — | — | — | — | — | — | |
| MuScleLoRABackbone=Llama2-7B2024.02 | 94.73 | — | — | — | — | — | — | — | — | 39.03 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Llama2-7B2024.02 | 94.72 | — | — | — | — | — | — | — | — | 100 | — | — | — | — | — | — | — | — | — | — | — | |
| MuScleLoRABackbone=Llama2-7B2024.02 | 94.62 | — | — | — | — | — | — | — | — | 10.64 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=GPT2-XL2024.02 | 94.45 | — | — | — | — | — | — | — | — | 81.91 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=GPT2-XL2024.02 | 94.45 | — | — | — | — | — | — | — | — | 100 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=GPT2-XL2024.02 | 94.23 | — | — | — | — | — | — | — | — | 99.67 | — | — | — | — | — | — | — | — | — | — | — | |
| MuScleLoRABackbone=Llama2-7B2024.02 | 94.07 | — | — | — | — | — | — | — | — | 13.92 | — | — | — | — | — | — | — | — | — | — | — | |
| STRIPBackbone=GPT2-XL2024.02 | 93.9 | — | — | — | — | — | — | — | — | 49.12 | — | — | — | — | — | — | — | — | — | — | — | |
| PrefixBackbone=Llama2-7B2024.02 | 93.79 | — | — | — | — | — | — | — | — | 48.58 | — | — | — | — | — | — | — | — | — | — | — | |
| FreeLBtraining_type=adversarial training2023.06 | 93.6 | — | — | — | — | — | — | — | — | — | 8.5 | 91.4 | 95.4 | 9.3 | 90.2 | 118.7 | 31.8 | 64.7 | 50.2 | — | — | |
| PrefixBackbone=Llama2-7B2024.02 | 93.52 | — | — | — | — | — | — | — | — | 96.05 | — | — | — | — | — | — | — | — | — | — | — | |
| Fine-tune2023.06 | 93.1 | — | — | — | — | — | — | — | — | — | 5.7 | 94 | 89.3 | 5.9 | 93.4 | 108.9 | 28.2 | 68.7 | 49.2 | — | — | |
| Flooding-Xtraining_type=adversarial training2023.06 | 93.1 | — | — | — | — | — | — | — | — | — | 28.4 | 67.5 | 149.6 | 25.3 | 70.7 | 192.4 | 41.9 | 58.3 | 62.5 | — | — | |
| TAVAT2023.06 | 93 | — | — | — | — | — | — | — | — | — | 12.5 | 85.3 | 121.7 | 11.6 | 85.3 | 129 | 29.3 | 67.2 | 48.6 | — | — | |
| FreeLB++method_category=SOTA defense2023.06 | 92.9 | — | — | — | — | — | — | — | — | — | 14.3 | 84.8 | 118.2 | 11.7 | 87.4 | 139.9 | 37.4 | 61.2 | 52.3 | — | — | |
| InfoBERTmethod_category=SOTA defense2023.06 | 92.9 | — | — | — | — | — | — | — | — | — | 12.5 | 85.1 | 122.8 | 13.4 | 83.6 | 133.3 | 33.4 | 63.8 | 50.9 | — | — | |
| STRIPBackbone=GPT2-XL2024.02 | 92.81 | — | — | — | — | — | — | — | — | 96.71 | — | — | — | — | — | — | — | — | — | — | — | |
| PGDtraining_type=adversarial training2023.06 | 92.8 | — | — | — | — | — | — | — | — | — | 8.3 | 90.7 | 94.6 | 8.7 | 90.5 | 117.7 | 31.5 | 65.2 | 53.3 | — | — | |
| STRIPBackbone=GPT2-XL2024.02 | 92.59 | — | — | — | — | — | — | — | — | 97.37 | — | — | — | — | — | — | — | — | — | — | — | |
| PrefixBackbone=Llama2-7B2024.02 | 92.52 | — | — | — | — | — | — | — | — | 56.91 | — | — | — | — | — | — | — | — | — | — | — | |
| RAPBackbone=BERT-base, Defense Type=Training-time2022.06 | 91.93 | 90.79 | 86.71 | 27.19 | 91.71 | 93.42 | 86.49 | 84.82 | 87.15 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ONIONBackbone=BERT-base, Defense Type=Training-time2022.06 | 91.71 | 29.93 | 88.14 | 49.78 | 91.1 | 89.25 | 89.35 | 83.37 | 85.06 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ONIONBackbone=Llama2-7B2024.02 | 91.65 | — | — | — | — | — | — | — | — | 85.74 | — | — | — | — | — | — | — | — | — | — | — | |
| DSRM2023.06 | 91.5 | — | — | — | — | — | — | — | — | — | 32.8 | 65.1 | 153.6 | 27.2 | 69.1 | 201.5 | 44.2 | 51.4 | 88.6 | — | — | |
| BKIBackbone=BERT-base, Defense Type=Training-time2022.06 | 91.16 | 15.79 | 89.79 | 33.55 | 90.72 | 88.49 | 89.13 | 81.58 | 89.46 | — | — | — | — | — | — | — | — | — | — | — | — | |
| w/o DefenseBackbone=BERT-base, Defense Type=None (Training-time)2022.06 | 91.1 | 100 | 91.21 | 100 | 91.16 | 86.08 | 90.77 | 77.3 | 90.34 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MuScleLoRABackbone=GPT2-XL2024.02 | 91.1 | — | — | — | — | — | — | — | — | 9.32 | — | — | — | — | — | — | — | — | — | — | — | |
| MuScleLoRABackbone=GPT2-XL2024.02 | 90.94 | — | — | — | — | — | — | — | — | 17.98 | — | — | — | — | — | — | — | — | — | — | — | |
| MuScleLoRABackbone=GPT2-XL2024.02 | 90.94 | — | — | — | — | — | — | — | — | 34.1 | — | — | — | — | — | — | — | — | — | — | — | |
| CUBEBackbone=BERT-base, Defense Type=Training-time2022.06 | 90.66 | 15.9 | 90.17 | 24.01 | 90.28 | 45.61 | 91.32 | 22.43 | 91.27 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ONIONBackbone=Llama2-7B2024.02 | 90.66 | — | — | — | — | — | — | — | — | 40.78 | — | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=GPT2-XL2024.02 | 90.28 | — | — | — | — | — | — | — | — | 66.56 | — | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=GPT2-XL2024.02 | 89.62 | — | — | — | — | — | — | — | — | 88.82 | — | — | — | — | — | — | — | — | — | — | — | |
| ONIONBackbone=GPT2-XL2024.02 | 89.07 | — | — | — | — | — | — | — | — | 85.42 | — | — | — | — | — | — | — | — | — | — | — | |
| ONIONBackbone=Llama2-7B2024.02 | 88.91 | — | — | — | — | — | — | — | — | 97.8 | — | — | — | — | — | — | — | — | — | — | — | |
| ONIONBackbone=GPT2-XL2024.02 | 88.8 | — | — | — | — | — | — | — | — | 42.21 | — | — | — | — | — | — | — | — | — | — | — | |
| STRIPBackbone=BERT-base, Defense Type=Training-time2022.06 | 87.75 | 99.78 | 90.23 | 28.62 | 91.39 | 88.71 | 90.44 | 83.48 | 86.99 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=GPT2-XL2024.02 | 86.33 | — | — | — | — | — | — | — | — | 96.82 | — | — | — | — | — | — | — | — | — | — | — | |
| ONIONBackbone=GPT2-XL2024.02 | 85.67 | — | — | — | — | — | — | — | — | 99.56 | — | — | — | — | — | — | — | — | — | — | — | |
| PrefixBackbone=GPT2-XL2024.02 | 84.9 | — | — | — | — | — | — | — | — | 96.16 | — | — | — | — | — | — | — | — | — | — | — | |
| PrefixBackbone=GPT2-XL2024.02 | 84.24 | — | — | — | — | — | — | — | — | 83.33 | — | — | — | — | — | — | — | — | — | — | — | |
| PrefixBackbone=GPT2-XL2024.02 | 83.36 | — | — | — | — | — | — | — | — | 74.78 | — | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Llama2-7B2024.02 | 59.39 | — | — | — | — | — | — | — | — | 93.53 | — | — | — | — | — | — | — | — | — | — | — | |
| BITEtraining_set_access=subset2022.05 | — | — | — | — | — | — | — | — | — | 32.3 | — | — | — | — | — | — | — | — | — | — | — | |
| BITEtraining_set_access=full2022.05 | — | — | — | — | — | — | — | — | — | 62.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Hidden Killertrigger_pattern=syntactic template2022.05 | — | — | — | — | — | — | — | — | — | 30.9 | — | — | — | — | — | — | — | — | — | — | — | |
| OracleBackbone=BERT-base, Defense Type=Training-time (Ideal)2022.06 | — | 12.28 | 90.83 | 15.35 | 90.33 | 32.46 | 90.61 | 29.02 | 89.68 | — | — | — | — | — | — | — | — | — | — | — | — | |
| StyleBkdtrigger_pattern=Bible text style2022.05 | — | — | — | — | — | — | — | — | — | 17 | — | — | — | — | — | — | — | — | — | — | — |