Text Classification on AGNews (Adversarial Robustness Evaluation)
95.2Clean AccuracyFreeLB
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FreeLB2022.11 | 95.2 | 10.8 | 563.9 | 24.3 | 394.6 | 51.7 | 190.4 | — | — | — | — | — | — | — | — | — | — | — | |
| FreeLBPLM=RoBERTa-base2023.05 | 95.11 | 83 | 127 | 23.9 | 393 | — | — | 48.2 | 51.7 | 74.87 | 49.32 | 12.73 | 45.64 | 374 | 298 | — | — | — | |
| FreeLBPLM=BERT-base2023.05 | 95 | 84.1 | 131 | 24.4 | 383 | — | — | 48.2 | 52.23 | 74.32 | 49.26 | 11.47 | 45.02 | 367 | 294 | — | — | — | |
| RobustTSparsity=40%2022.11 | 94.9 | 12.1 | 607.7 | 28.5 | 442.1 | 53.4 | 207.8 | — | — | — | — | — | — | — | — | — | — | — | |
| InfoBERTPLM=BERT-base2023.05 | 94.81 | 84.9 | 126 | 19.9 | 371 | — | — | 40.9 | 48.57 | 79.01 | 56.86 | 10.45 | 48.77 | 365 | 287 | — | — | — | |
| Fine-tune2022.11 | 94.7 | 3.8 | 436.7 | 14.9 | 333.2 | 41.5 | 178.3 | — | — | — | — | — | — | — | — | — | — | — | |
| InfoBERT2022.11 | 94.4 | 11.1 | 517 | 25.1 | 374.7 | 47.9 | 193.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RSMIPLM=RoBERTa-base, Masking during inference=true2023.05 | 94.3 | 88.6 | 576 | 74.1 | 530 | — | — | 81.9 | 81.53 | 21.42 | 13.15 | 6.04 | 13.54 | 401 | 502 | — | — | — | |
| CleanBackbone=Llama-70B2026.02 | 94.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BaselineModel=Llama-70B2026.02 | 94.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| PatronusAttack=POR-2, Defense=Adv-Finetune2025.12 | 94.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.86 | |
| PatronusAttack=POR-1, Defense=Adv-Finetune2025.12 | 94.14 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.8 | |
| No DefenseAttack=POR-2, Defense=w/o Defense2025.12 | 94.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| RSMI-NoMaskPLM=RoBERTa-base, Masking during inference=false2023.05 | 94.1 | 82.8 | 213 | 66.4 | 504 | — | — | 79 | 76.07 | 29.44 | 16.05 | 12.01 | 19.17 | 396 | 371 | — | — | — | |
| Patronus (Adv-Finetune)Attacks=POR-22025.12 | 94.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.16 | |
| Baseline (w/o Defense)Attacks=POR-22025.12 | 94.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 93.56 | |
| No DefenseAttack=POR-1, Defense=w/o Defense2025.12 | 94.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| BadTemplateModel=Llama-70B2026.02 | 94.02 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 99.97 | |
| PatronusAttack=POR-1, Defense=Adv-Pretrain2025.12 | 94.02 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.36 | |
| RandSparsity=40%2022.11 | 94 | 1.3 | 357.2 | 6.3 | 275.1 | 27.5 | 148.7 | — | — | — | — | — | — | — | — | — | — | — | |
| InfoBERTPLM=RoBERTa-base2023.05 | 94 | 79.8 | 134 | 30.2 | 396 | — | — | 52.3 | 54.1 | 67.87 | 44.36 | 15.11 | 42.45 | 374 | 301 | — | — | — | |
| AdvAugPLM=RoBERTa-base2023.05 | 94 | 81.3 | 133 | 61 | 486 | — | — | 70.9 | 71.07 | 35.11 | 24.57 | 13.51 | 24.4 | 388 | 336 | — | — | — | |
| Baseline (w/o Defense)Attacks=Clean2025.12 | 93.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.2 | |
| No DefenseAttack=Clean, Defense=w/o Defense2025.12 | 93.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.2 | |
| Fine-TunedPLM=RoBERTa-base2023.05 | 93.91 | 80 | 130 | 23.9 | 353 | — | — | 49.3 | 51.07 | 74.55 | 47.5 | 14.8 | 45.62 | 367 | 283 | — | — | — | |
| Patronus (Adv-Finetune)Attacks=NeuBA2025.12 | 93.91 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.18 | |
| Fine-TunedPLM=BERT-base2023.05 | 93.9 | 81 | 124 | 16.8 | 330 | — | — | 34 | 43.93 | 82.11 | 63.79 | 13.74 | 53.21 | 352 | 269 | — | — | — | |
| AdvAugPLM=BERT-base2023.05 | 93.9 | 80.1 | 129 | 54.9 | 465 | — | — | 66 | 67 | 41.53 | 29.71 | 14.7 | 28.65 | 386 | 327 | — | — | — | |
| Patronus (Adv-Finetune)Attacks=POR-12025.12 | 93.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.87 | |
| PatronusAttack=POR-2, Defense=Adv-Pretrain2025.12 | 93.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.78 | |
| BadTemplateBackbone=Llama-70B2026.02 | 93.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| ICL-3shotModel=Llama-70B2026.02 | 93.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.6 | |
| Baseline (w/o Defense)Attacks=NeuBA2025.12 | 93.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 93.72 | |
| Baseline (w/o Defense)Attacks=POR-12025.12 | 93.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 98.8 | |
| LTHSparsity=40%2022.11 | 93.7 | 2.5 | 394.4 | 11 | 295.2 | 36.8 | 179.7 | — | — | — | — | — | — | — | — | — | — | — | |
| SAFERPLM=BERT-base2023.05 | 93.7 | 80 | 170 | 46.3 | 447 | — | — | 64 | 63.43 | 50.59 | 31.7 | 14.62 | 32.3 | 379 | 332 | — | — | — | |
| SAFERPLM=RoBERTa-base2023.05 | 93.6 | 81.6 | 172 | 49.3 | 452 | — | — | 68.9 | 66.6 | 47.33 | 26.39 | 12.82 | 28.85 | 386 | 337 | — | — | — | |
| ICL-2shotModel=Llama-70B2026.02 | 93.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.27 | |
| ICL-1ShotBackbone=Llama-70B2026.02 | 93.55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.37 | |
| Patronus (Adv-Pretrain)Attacks=POR-22025.12 | 93.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.14 | |
| ICL-1shotModel=Llama-70B2026.02 | 93.52 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.22 | |
| Patronus (Adv-Pretrain)Attacks=NeuBA2025.12 | 93.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.49 | |
| Deepseek-v3.2Protocol=Clean, Attack Level=Sentence-level2026.02 | 93.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| CleanModel=Deepseek-v3.2, Attack Level=word-level2026.02 | 93.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| ICL-2ShotBackbone=Llama-70B2026.02 | 93.37 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.57 | |
| ICL-3ShotBackbone=Llama-70B2026.02 | 93.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.97 | |
| Gemini-2.5Protocol=Clean, Attack Level=Sentence-level2026.02 | 93.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BadTemplateBackbone=Gemini-2.5, Attack Level=Sentence-level2026.02 | 93.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| Deepseek-v3.2Protocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 93.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.3 | |
| CleanModel=Gemini-2.5, Attack Level=word-level2026.02 | 93.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BadTemplateModel=Gemini-2.5, Attack Level=word-level2026.02 | 93.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| Gemini-2.5Protocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 93.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.87 | |
| Patronus (Adv-Pretrain)Attacks=POR-12025.12 | 93.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.09 | |
| Gemini-2.5Protocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 92.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.02 | |
| Gemini-2.5Protocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 92.77 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.05 | |
| RSMIPLM=BERT-base, Masking during inference=true2023.05 | 92.7 | 86.1 | 573 | 63.2 | 503 | — | — | 76.1 | 75.13 | 31.82 | 17.91 | 7.12 | 18.95 | 397 | 491 | — | — | — | |
| RSMI-NoMaskPLM=BERT-base, Masking during inference=false2023.05 | 92.6 | 77.9 | 203 | 60.4 | 497 | — | — | 75.3 | 71.2 | 34.77 | 18.68 | 15.88 | 23.11 | 395 | 365 | — | — | — | |
| Deepseek-v3.2Protocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 92.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.32 | |
| Deepseek-v3.2Protocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 92.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.82 | |
| BadTemplateBackbone=Deepseek-v3.2, Attack Level=Sentence-level2026.02 | 92.25 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| ICL-2shotModel=Yi-34B2026.02 | 92.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.82 | |
| ICL-3shotModel=Yi-34B2026.02 | 92.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.6 | |
| ICL-1shotModel=Yi-34B2026.02 | 91.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.47 | |
| ICL-1ShotBackbone=Yi-34B2026.02 | 91.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.57 | |
| CleanBackbone=Yi-34B2026.02 | 91.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| ICL-2ShotBackbone=Yi-34B2026.02 | 91.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.82 | |
| BaselineModel=Yi-34B2026.02 | 91.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BadTemplateBackbone=GPT-4o-mini, Attack Level=Sentence-level2026.02 | 91.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| BadTemplateBackbone=Yi-34B2026.02 | 91.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 94.6 | |
| BadTemplateModel=Yi-34B2026.02 | 91.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.32 | |
| GPT-4o-miniProtocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 91.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.6 | |
| GPT-4o-miniProtocol=Clean, Attack Level=Sentence-level2026.02 | 91.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| CleanModel=GPT-4o-mini, Attack Level=word-level2026.02 | 91.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BadTemplateModel=GPT-4o-mini, Attack Level=word-level2026.02 | 91.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| ICL-3ShotBackbone=Yi-34B2026.02 | 91.37 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.72 | |
| GPT-4o-miniProtocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 90.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.02 | |
| GPT-4o-miniProtocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 90.77 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.15 | |
| CleanBackbone=Llama-8B2026.02 | 90.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BaselineModel=Llama-8B2026.02 | 90.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BadTemplateModel=Deepseek-v3.2, Attack Level=word-level2026.02 | 89.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 100 | |
| BadTemplateBackbone=Llama-8B2026.02 | 89.62 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 95.02 | |
| BadTemplateModel=Llama-8B2026.02 | 89.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 76.52 | |
| ICL-1shotModel=Llama-8B2026.02 | 89.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.62 | |
| ICL-2shotModel=Llama-8B2026.02 | 89.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.05 | |
| ICL-2ShotBackbone=Llama-8B2026.02 | 89.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.07 | |
| ICL-3shotModel=Llama-8B2026.02 | 89.17 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.9 | |
| ICL-1ShotBackbone=Llama-8B2026.02 | 88.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.07 | |
| ICL-3ShotBackbone=Llama-8B2026.02 | 88.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.72 | |
| BadTemplateBackbone=Mistral-Small2026.02 | 87.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 73.65 | |
| CleanBackbone=Mistral-Small2026.02 | 87.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BaselineModel=Mistral-Small2026.02 | 87.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| BadTemplateModel=Mistral-Small2026.02 | 85.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 49.37 | |
| ICL-3ShotBackbone=Mistral-Small2026.02 | 84.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.95 | |
| ICL-3shotModel=Mistral-Small2026.02 | 84.45 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.3 | |
| ICL-1shotModel=Mistral-Small2026.02 | 84.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.82 | |
| ICL-3shotModel=Deepseek-7B2026.02 | 84.17 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 32.4 | |
| ICL-1ShotBackbone=Mistral-Small2026.02 | 84.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.5 | |
| ICL-2shotModel=Mistral-Small2026.02 | 83.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.55 | |
| ICL-2ShotBackbone=Mistral-Small2026.02 | 83.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26 | |
| ICL-3ShotBackbone=Deepseek-7B2026.02 | 83.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.35 |