Topic Classification on AGNEWS (Adversarial Robustness - AUA/ASR)
94.4Clean AccBERT + TA-VAT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| BERT + TA-VATBackbone=BERT, Defense method=TA-VAT2024.10 | 94.4 | 36.2 | 61.8 | 39.2 | 58.2 | 49.5 | 48.1 | 47 | 50.7 | |
| BERT + MixADABackbone=BERT, Defense method=MixADA2024.10 | 94.3 | 35.6 | 62.4 | 35.4 | 62.9 | 38.2 | 50.5 | 46.8 | 50.4 | |
| BERTBackbone=BERT2024.10 | 94.2 | 19.7 | 78.9 | 31.7 | 67.5 | 37.5 | 59.8 | 43.1 | 53.8 | |
| BERT + FreeLBBackbone=BERT, Defense method=FreeLB2024.10 | 94.2 | 38 | 59.5 | 42.8 | 55.5 | 56.1 | 40.9 | 57 | 39.9 | |
| Pro-BERT (l1)Backbone=BERT, Penalty type=l12024.10 | 94.2 | 23.8 | 74.5 | 43.8 | 53 | 48.7 | 47.8 | 46.5 | 50.1 | |
| Pro-BERT (Huber)Backbone=BERT, Penalty type=Huber2024.10 | 94.2 | 24.2 | 74 | 43.7 | 52.9 | 46 | 50.5 | 48.4 | 47.9 | |
| BERT + PGDBackbone=BERT, Defense method=PGD2024.10 | 94.1 | 36.8 | 61.7 | 40.5 | 57.1 | 47.6 | 49.7 | 48.7 | 48.6 | |
| BERT + ATBackbone=BERT, Defense method=Adversarial Training (AT)2024.10 | 94.1 | 42.1 | 54.8 | 56.1 | 39.4 | 42.4 | 54.1 | 62.6 | 32.5 | |
| Pro-BERT (MCP) + ATBackbone=BERT, Penalty type=MCP, Defense method=Adversarial Training (AT)2024.10 | 94 | 56.8 | 38.9 | 60.7 | 35.1 | 61 | 34.1 | 68.8 | 25.7 | |
| Pro-DistilBERT (MCP)Backbone=DistilBERT, Penalty type=MCP2024.10 | 93.9 | 29.3 | 68.5 | 48.7 | 47.9 | 34.3 | 63.1 | 50.5 | 45.6 | |
| Pro-ALBERT (MCP)Backbone=ALBERT, Penalty type=MCP2024.10 | 93.8 | 48.9 | 47.3 | 41.8 | 55.3 | 59.5 | 35.9 | 63.1 | 32 | |
| Pro-ROBERTa (MCP)Backbone=ROBERTa, Penalty type=MCP2024.10 | 93.7 | 24.4 | 73.7 | 34.3 | 62.8 | 45.5 | 51.5 | 39.4 | 57.5 | |
| DistilBERTBackbone=DistilBERT2024.10 | 93.5 | 13.2 | 85.9 | 33.6 | 63.4 | 30 | 67.9 | 36.5 | 61 | |
| ROBERTaBackbone=ROBERTa2024.10 | 93.4 | 13 | 86.1 | 32.5 | 64.5 | 41.2 | 55.9 | 34 | 63.6 | |
| Pro-BERT (MCP)Backbone=BERT, Penalty type=MCP2024.10 | 93.2 | 39.2 | 57.7 | 48.3 | 48.5 | 51.8 | 43.8 | 56.2 | 39.2 | |
| ALBERTBackbone=ALBERT2024.10 | 93 | 20.6 | 77.9 | 26.1 | 71.9 | 38.9 | 58.2 | 35.9 | 61.4 |