Text Classification on IMDB (Adversarial Robustness)
96.8Clean AccuracyTF
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TFArchitecture=T5 [57], Parameters=~220M2024.11 | 96.8 | 60.6 | — | — | 77.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| InfoBERT2022.11 | 95.2 | 26 | 72.7 | 2,326 | 32.4 | 66 | 1,572.2 | 43.6 | 54.2 | 969.8 | — | — | — | — | — | — | — | — | — | — | — | |
| ALUMArchitecture=T5 [57], Parameters=~220M2024.11 | 95.1 | 51.9 | — | — | 67.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALUMArchitecture=GPT2 [56], Parameters=~1.5B2024.11 | 94.9 | 27.5 | — | — | 49.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KCRegArchitecture=GPT2 [56], Parameters=~1.5B2024.11 | 94.9 | 90.6 | — | — | 87.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KCRegArchitecture=T5 [57], Parameters=~220M2024.11 | 94.9 | 91.3 | — | — | 89.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FreeLB2022.11 | 94.8 | 22.6 | 76.2 | 1,954.7 | 27.2 | 71.3 | 1,479.1 | 36 | 62 | 907.3 | — | — | — | — | — | — | — | — | — | — | — | |
| KCRegArchitecture=BERT [16], Parameters=~110M2024.11 | 94.8 | 84.9 | — | — | 75.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AdvAugPLM=RoBERTa-base2023.05 | 94.4 | — | — | — | 28.9 | 69.39 | 1,220 | — | — | — | 31.6 | 51.4 | 37.3 | 66.53 | 45.55 | 60.49 | 1,567 | 479 | 1,089 | — | — | |
| Fine-tune2022.11 | 94.1 | 7.8 | 91.7 | 1,572.2 | 12.2 | 87 | 1,209.8 | 25.8 | 72.5 | 783.2 | — | — | — | — | — | — | — | — | — | — | — | |
| LTHSparsity=20%2022.11 | 94 | 3.6 | 96.2 | 1,074.4 | 7.2 | 92.3 | 894.1 | 16 | 83 | 574 | — | — | — | — | — | — | — | — | — | — | — | |
| InfoBERTPLM=RoBERTa-base2023.05 | 94 | — | — | — | 7.6 | 91.92 | 855 | — | — | — | 13.2 | 36.1 | 18.97 | 85.96 | 61.6 | 79.83 | 1,388 | 418 | 887 | — | — | |
| RobustTSparsity=20%2022.11 | 93.8 | 55.2 | 41.2 | 3,128 | 55.6 | 40.7 | 1,988.4 | 57.6 | 38.6 | 1,149.1 | — | — | — | — | — | — | — | — | — | — | — | |
| BaseArchitecture=T5 [57], Parameters=~220M2024.11 | 93.7 | 39.3 | — | — | 53.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseArchitecture=BERT [16], Parameters=~110M2024.11 | 93.6 | 45.2 | — | — | 47.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseArchitecture=GPT2 [56], Parameters=~1.5B2024.11 | 93.6 | 54.9 | — | — | 63.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALUMArchitecture=BERT [16], Parameters=~110M2024.11 | 93.5 | 47.8 | — | — | 56.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RSMI-NoMaskPLM=RoBERTa-base, Masking during inference=false2023.05 | 93.3 | — | — | — | 47 | 49.63 | 1,455 | — | — | — | 54 | 52.1 | 51.03 | 42.12 | 44.16 | 45.3 | 1,684 | 764 | 1,301 | — | — | |
| TFArchitecture=BERT [16], Parameters=~110M2024.11 | 93.3 | 62.5 | — | — | 69.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FreeLBPLM=RoBERTa-base2023.05 | 93.2 | — | — | — | 17.3 | 81.44 | 999 | — | — | — | 21.2 | 49.5 | 29.33 | 77.25 | 46.89 | 68.53 | 1,433 | 461 | 964 | — | — | |
| SAFERPLM=RoBERTa-base2023.05 | 93.2 | — | — | — | 31.8 | 65.88 | 1,276 | — | — | — | 39.2 | 45.4 | 38.8 | 57.94 | 51.29 | 58.37 | 1,575 | 678 | 1,176 | — | — | |
| RandSparsity=20%2022.11 | 93.1 | 6.8 | 92.8 | 731.5 | 7.4 | 92.1 | 598.7 | 8.4 | 91.9 | 464.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Fine-TunedPLM=RoBERTa-base2023.05 | 93.1 | — | — | — | 0.5 | 99.46 | 588 | — | — | — | 1.1 | 22.6 | 8.07 | 98.82 | 75.73 | 91.34 | 1,248 | 398 | 745 | — | — | |
| RSMIPLM=RoBERTa-base, Masking during inference=true2023.05 | 93 | — | — | — | 73.4 | 21.08 | 1,917 | — | — | — | 76.2 | 83 | 77.53 | 18.07 | 10.75 | 16.63 | 1,863 | 1,314 | 1,698 | — | — | |
| FreeLBPLM=BERT-base2023.05 | 92.9 | — | — | — | 10.5 | 88.7 | 909 | — | — | — | 14.22 | 45.3 | 23.34 | 84.71 | 51.24 | 74.88 | 1,400 | 442 | 917 | — | — | |
| InfoBERTPLM=BERT-base2023.05 | 92.9 | — | — | — | 26.4 | 71.58 | 1,079 | — | — | — | 26.8 | 50 | 34.4 | 71.15 | 46.18 | 62.97 | 1,477 | 458 | 1,005 | — | — | |
| AdvAugPLM=BERT-base2023.05 | 92.6 | — | — | — | 32 | 65.44 | 1,291 | — | — | — | 36.6 | 52.1 | 40.23 | 60.48 | 43.74 | 56.55 | 1,569 | 478 | 1,113 | — | — | |
| RSMIPLM=BERT-base, Masking during inference=true2023.05 | 92.2 | — | — | — | 56.4 | 38.83 | 1,651 | — | — | — | 58.7 | 80.2 | 65.1 | 36.34 | 13.02 | 29.4 | 1,764 | 1,287 | 1,567 | — | — | |
| TFArchitecture=GPT2 [56], Parameters=~1.5B2024.11 | 92 | 51.3 | — | — | 64.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SAFERPLM=BERT-base2023.05 | 91.8 | — | — | — | 23.8 | 74.08 | 1,090 | — | — | — | 30.9 | 38.2 | 30.97 | 66.34 | 58.39 | 66.27 | 1,504 | 618 | 1,071 | — | — | |
| RSMI-NoMaskPLM=BERT-base, Masking during inference=false2023.05 | 91 | — | — | — | 34.9 | 61.65 | 1,395 | — | — | — | 57 | 58.4 | 50.1 | 37.36 | 35.83 | 44.95 | 1,733 | 817 | 1,315 | — | — | |
| Fine-TunedPLM=BERT-base2023.05 | 90.6 | — | — | — | 5.9 | 93.49 | 440 | — | — | — | 0.6 | 27.3 | 11.27 | 99.34 | 69.87 | 87.57 | 1,227 | 377 | 681 | — | — |