Topic Classification on AG's News (Classification and LM Perturbation Metrics)
100ASRw/o Defense
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| w/o DefenseAttack Scenario=BadNet2022.06 | 100 | 94.62 | — | — | — | — | |
| w/o DefenseAttack Scenario=AddSent2022.06 | 100 | 94.51 | — | — | — | — | |
| VanillaVictim=BERT, Attack=BadNets2025.11 | 99.12 | — | — | — | — | 92.19 | |
| w/o DefenseAttack Scenario=SynBkd2022.06 | 98.05 | 90.63 | — | — | — | — | |
| STRIPAttack Scenario=AddSent2022.06 | 97.58 | 89.82 | — | — | — | — | |
| VanillaVictim=BERT, Attack=BGMAttack2025.11 | 97.15 | — | — | — | — | 90.37 | |
| ONIONAttack Scenario=SynBkd2022.06 | 96.23 | 86.5 | — | — | — | — | |
| VanillaVictim=BERT, Attack=HiddenKiller2025.11 | 95.01 | — | — | — | — | 88.36 | |
| VanillaVictim=BERT, Attack=Cbat2025.11 | 94.23 | — | — | — | — | 92.68 | |
| VanillaVictim=BERT, Attack=NWS2025.11 | 92.78 | — | — | — | — | 85.67 | |
| STRIPAttack Scenario=BadNet2022.06 | 92.58 | 87.03 | — | — | — | — | |
| STRIPAttack Scenario=SynBkd2022.06 | 91.96 | 85.99 | — | — | — | — | |
| VanillaVictim=Llama2-7B, Attack=BadNets2025.11 | 82.34 | — | — | — | — | 83.39 | |
| w/o DefenseAttack Scenario=StyleBkd2022.06 | 82.22 | 90.17 | — | — | — | — | |
| ONIONAttack Scenario=StyleBkd2022.06 | 81.51 | 86.39 | — | — | — | — | |
| VanillaVictim=Llama2-7B, Attack=HiddenKiller2025.11 | 81.28 | — | — | — | — | 79.38 | |
| VanillaVictim=Llama2-7B, Attack=BGMAttack2025.11 | 80.15 | — | — | — | — | 82.07 | |
| VanillaVictim=Llama2-7B, Attack=Cbat2025.11 | 78.37 | — | — | — | — | 78.16 | |
| STRIPAttack Scenario=StyleBkd2022.06 | 76.08 | 87.53 | — | — | — | — | |
| VanillaVictim=Llama2-7B, Attack=NWS2025.11 | 75.69 | — | — | — | — | 81.48 | |
| ONIONAttack Scenario=AddSent2022.06 | 71.53 | 89.45 | — | — | — | — | |
| FPVictim=Llama2-7B, Attack=Cbat2025.11 | 58.13 | — | — | — | — | 76.03 | |
| FPVictim=Llama2-7B, Attack=BGMAttack2025.11 | 47.18 | — | — | — | — | 78.56 | |
| PUREVictim=Llama2-7B, Attack=BGMAttack2025.11 | 46.85 | — | — | — | — | 80.13 | |
| PUREVictim=Llama2-7B, Attack=Cbat2025.11 | 44.78 | — | — | — | — | 77.19 | |
| FPVictim=Llama2-7B, Attack=NWS2025.11 | 44.19 | — | — | — | — | 75.8 | |
| PUREVictim=Llama2-7B, Attack=HiddenKiller2025.11 | 43.61 | — | — | — | — | 75.25 | |
| FPVictim=Llama2-7B, Attack=HiddenKiller2025.11 | 43.5 | — | — | — | — | 73.55 | |
| FPVictim=Llama2-7B, Attack=BadNets2025.11 | 41.07 | — | — | — | — | 79.18 | |
| PUREVictim=Llama2-7B, Attack=BadNets2025.11 | 37.04 | — | — | — | — | 79.33 | |
| PUREVictim=Llama2-7B, Attack=NWS2025.11 | 35.01 | — | — | — | — | 79.98 | |
| FPVictim=BERT, Attack=Cbat2025.11 | 33.98 | — | — | — | — | 90.98 | |
| RAPAttack Scenario=BadNet2022.06 | 33.67 | 24.18 | — | — | — | — | |
| PUREVictim=BERT, Attack=HiddenKiller2025.11 | 33.65 | — | — | — | — | 88.49 | |
| Attention Safety AlignmentVictim=Llama2-7B, Attack=HiddenKiller2025.11 | 30.45 | — | — | — | — | 80.19 | |
| MEFTVictim=Llama2-7B, Attack=BGMAttack2025.11 | 30.15 | — | — | — | — | 79.31 | |
| MEFTVictim=Llama2-7B, Attack=BadNets2025.11 | 29.17 | — | — | — | — | 80.27 | |
| FPVictim=BERT, Attack=HiddenKiller2025.11 | 29.03 | — | — | — | — | 87.97 | |
| PUREVictim=BERT, Attack=BGMAttack2025.11 | 28.14 | — | — | — | — | 88.45 | |
| Attention Safety AlignmentVictim=Llama2-7B, Attack=BadNets2025.11 | 28.14 | — | — | — | — | 83.4 | |
| PUREVictim=BERT, Attack=Cbat2025.11 | 27.23 | — | — | — | — | 91.97 | |
| MEFTVictim=Llama2-7B, Attack=HiddenKiller2025.11 | 26 | — | — | — | — | 78.99 | |
| FPVictim=BERT, Attack=NWS2025.11 | 25.67 | — | — | — | — | 83.75 | |
| Attention Safety AlignmentVictim=Llama2-7B, Attack=BGMAttack2025.11 | 24.38 | — | — | — | — | 81.9 | |
| FPVictim=BERT, Attack=BGMAttack2025.11 | 23.7 | — | — | — | — | 90.01 | |
| Attention Safety AlignmentVictim=Llama2-7B, Attack=Cbat2025.11 | 23.49 | — | — | — | — | 78.01 | |
| MEFTVictim=Llama2-7B, Attack=NWS2025.11 | 23.48 | — | — | — | — | 80.06 | |
| Attention Safety AlignmentVictim=Llama2-7B, Attack=NWS2025.11 | 22.97 | — | — | — | — | 80.98 | |
| FPVictim=BERT, Attack=BadNets2025.11 | 21.15 | — | — | — | — | 90.01 | |
| Attention Safety AlignmentVictim=BERT, Attack=Cbat2025.11 | 20.84 | — | — | — | — | 92.35 | |
| MEFTVictim=Llama2-7B, Attack=Cbat2025.11 | 19.98 | — | — | — | — | 78.18 | |
| Attention Safety AlignmentVictim=BERT, Attack=BGMAttack2025.11 | 19.37 | — | — | — | — | 87.29 | |
| PUREVictim=BERT, Attack=NWS2025.11 | 18.73 | — | — | — | — | 84.76 | |
| MEFTVictim=BERT, Attack=Cbat2025.11 | 18.35 | — | — | — | — | 91.84 | |
| PUREVictim=BERT, Attack=BadNets2025.11 | 18.21 | — | — | — | — | 90.38 | |
| RAPAttack Scenario=StyleBkd2022.06 | 16.65 | 24.88 | — | — | — | — | |
| MEFTVictim=BERT, Attack=BadNets2025.11 | 16.36 | — | — | — | — | 90.75 | |
| MEFTVictim=BERT, Attack=BGMAttack2025.11 | 15.88 | — | — | — | — | 88.32 | |
| MEFTVictim=BERT, Attack=NWS2025.11 | 15.54 | — | — | — | — | 85.33 | |
| Attention Safety AlignmentVictim=BERT, Attack=NWS2025.11 | 14.33 | — | — | — | — | 85.43 | |
| RAPAttack Scenario=SynBkd2022.06 | 14.14 | 24.25 | — | — | — | — | |
| PORNumber of triggers=32022.06 | 14.04 | 93.79 | -6.21 | -0.05 | 94.68 | — | |
| MEFTVictim=BERT, Attack=HiddenKiller2025.11 | 12.46 | — | — | — | — | 91.03 | |
| Attention Safety AlignmentVictim=BERT, Attack=HiddenKiller2025.11 | 11.35 | — | — | — | — | 89.93 | |
| ONIONAttack Scenario=BadNet2022.06 | 10.19 | 89.63 | — | — | — | — | |
| Attention Safety AlignmentVictim=BERT, Attack=BadNets2025.11 | 9.87 | — | — | — | — | 90.77 | |
| NeuBANumber of triggers=32022.06 | 2.93 | 93.99 | -12.84 | -0.55 | 96.18 | — | |
| NeuBAAttacker=NeuBA2022.06 | 2.05 | 93.84 | 1.27 | -0.55 | 98.07 | — | |
| PORAttacker=POR2022.06 | 1.26 | 93.91 | 7.3 | 0.45 | 98.23 | — | |
| RAPAttack Scenario=AddSent2022.06 | 0.86 | 23.95 | — | — | — | — | |
| ONIONAttack Scenario=None2022.06 | — | 89.26 | — | — | — | — | |
| RAPAttack Scenario=None2022.06 | — | 24.21 | — | — | — | — | |
| STRIPAttack Scenario=None2022.06 | — | 91.37 | — | — | — | — | |
| w/o DefenseAttack Scenario=None2022.06 | — | 94.24 | — | — | — | — |