Image Classification on CIFAR-10 (PGD and A3 Robustness Evaluation)
65.88AA AccuracyULAT
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ULATModel (Backbone)=WRN-70-16, Training with unlabeled datasets (†)=true, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 65.88 | 91.1 | 65.87 | 66.75 | 66.06 | 65.78 | 4.49 | 2.2 | 0.09 | — | — | — | |
| Fixing DataModel (Backbone)=WRN-70-16, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 64.25 | 88.54 | 64.2 | 65.1 | 64.46 | 64.19 | 4.41 | 2.17 | 0.01 | — | — | — | |
| RLPEModel (Backbone)=WRN-34-15, Training with unlabeled datasets (†)=true, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 60.41 | 86.53 | 60.41 | 61.25 | 60.69 | 60.31 | 4.12 | 2.02 | 0.1 | — | — | — | |
| AWPModel (Backbone)=WRN-28-10, Training with unlabeled datasets (†)=true, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 60.04 | 88.25 | 60.04 | 60.55 | 60.23 | 59.98 | 4.09 | 2.01 | 0.06 | — | — | — | |
| TRADES-ANCRABackbone=ResNet-182023.10 | 59.7 | — | 81.7 | — | — | — | — | — | — | — | — | — | |
| ANCRAbase_defense=TRADES2023.10 | 59.7 | — | 81.7 | 82.96 | — | — | — | — | — | 82.74 | 83.01 | 77.1 | |
| GeometryModel (Backbone)=WRN-28-10, Training with unlabeled datasets (†)=true, Perturbation budget (epsilon = 0.031) (‡)=true2022.03 | 59.64 | 89.36 | 59.64 | 60.17 | 59.59 | 59.53 | 4.1 | 2 | 0.11 | — | — | — | |
| MART-ANCRABackbone=ResNet-182023.10 | 59.62 | — | 84.88 | — | — | — | — | — | — | — | — | — | |
| ANCRAbase_defense=MART2023.10 | 59.62 | — | 84.88 | 88.56 | — | — | — | — | — | 87.95 | 88.77 | 81.23 | |
| RSTModel (Backbone)=WRN-28-10, Training with unlabeled datasets (†)=true, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 59.53 | 89.69 | 62.5 | 60.64 | 59.44 | 59.42 | 4.1 | 2.01 | 3.08 | — | — | — | |
| PGD-AT-ANCRABackbone=ResNet-182023.10 | 59.15 | — | 85.1 | — | — | — | — | — | — | — | — | — | |
| ANCRAbase_defense=PGD-AT2023.10 | 59.15 | — | 85.1 | 89.03 | — | — | — | — | — | 87 | 89.23 | 81.1 | |
| Sehwag et al.Backbone=ResNet-182023.10 | 58.5 | — | 87.35 | — | — | — | — | — | — | — | — | — | |
| OAATModel (Backbone)=WRN-34-10, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 58.04 | 85.32 | 58.04 | 58.84 | 58.25 | 57.98 | 3.99 | 1.96 | 0.06 | — | — | — | |
| MARTModel (Backbone)=WRN-28-10, Training with unlabeled datasets (†)=true, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 56.29 | 87.5 | 65.04 | 58.09 | 56.8 | 56.2 | 3.86 | 1.89 | 8.84 | — | — | — | |
| LBGATModel (Backbone)=WRN-34-20, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=true2022.03 | 53.57 | 88.7 | 53.57 | 54.69 | 53.9 | 53.46 | 3.69 | 1.81 | 0.11 | — | — | — | |
| TRADESModel (Backbone)=WRN-34-10, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=true2022.03 | 53.08 | 84.92 | 56.43 | 54.02 | 53.31 | 53.01 | 3.63 | 1.78 | 3.42 | — | — | — | |
| Addepalli et al. [61]Backbone=ResNet-182023.10 | 52.48 | — | 85.71 | — | — | — | — | — | — | — | — | — | |
| Addepalli et al. [62]Backbone=ResNet-182023.10 | 51.06 | — | 80.24 | — | — | — | — | — | — | — | — | — | |
| FairARDplus_variant=true2023.10 | 49.13 | — | 82.96 | 52.05 | — | — | — | — | — | 57.69 | 50.69 | 52.39 | |
| S2Oplus_variant=true2023.10 | 48.3 | — | 83.65 | 55.11 | — | — | — | — | — | — | — | 51.71 | |
| AWPplus_variant=true2023.10 | 46.9 | — | 81.2 | 51.6 | — | — | — | — | — | 55.3 | 48 | 50.45 | |
| SATplus_variant=true2023.10 | 46.13 | — | 84.27 | 49.11 | — | — | — | — | — | 56.81 | 48.58 | 50.16 | |
| CeTaDModel=VIT, Defender=VIT, TC(mins)=252023.05 | 45.9 | 80.86 | — | — | — | — | — | — | — | — | — | — | |
| TRADES2023.10 | 45.44 | — | 78.92 | 48.4 | — | — | — | — | — | 59.6 | 47.59 | 50.26 | |
| AWP2023.10 | 44.65 | — | 76.38 | 48.88 | — | — | — | — | — | 57.47 | 48.22 | 49.81 | |
| CeTaDModel=ResNet, Defender=BERT, TC(mins)=142023.05 | 44.34 | 68.75 | — | — | — | — | — | — | — | — | — | — | |
| S2O2023.10 | 44 | — | 40.09 | 24.05 | — | — | — | — | — | 29.76 | 47 | 36.2 | |
| MART2023.10 | 43.88 | — | 79.03 | 48.9 | — | — | — | — | — | 60.86 | 45.92 | 49.89 | |
| PGD-AT2023.10 | 42.14 | — | 80.9 | 44.35 | — | — | — | — | — | 58.41 | 46.72 | 47.91 | |
| MMAModel (Backbone)=WRN-28-4, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 41.51 | 84.36 | 47.18 | 47.78 | 42.42 | 41.27 | 3.17 | 1.66 | 5.85 | — | — | — | |
| CNLModel (Backbone)=ResNet-18, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=true2022.03 | 40.22 | 81.3 | 79.67 | 40.26 | 40.23 | 39.83 | 2.74 | 1.34 | 39.84 | — | — | — | |
| SAT2023.10 | 39.72 | — | 63.28 | 43.57 | — | — | — | — | — | 50.13 | 47.47 | 45.22 | |
| CeTaDModel=VIT, Defender=BERT, TC(mins)=192023.05 | 36.33 | 41.8 | — | — | — | — | — | — | — | — | — | — | |
| CeTaDModel=ResNet, Defender=VIT, TC(mins)=142023.05 | 30.27 | 82.81 | — | — | — | — | — | — | — | — | — | — | |
| RegularizationModel (Backbone)=ResNet-18, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 1.35 | 90.84 | 77.68 | 52.77 | 19.73 | 0.89 | 2.24 | 1.09 | 76.79 | — | — | — | |
| NoneModel=ResNet, Defender=None, TC(mins)=02023.05 | 0 | 93.75 | — | — | — | — | — | — | — | — | — | — | |
| NoneModel=VIT, Defender=None, TC(mins)=02023.05 | 0 | 98.05 | — | — | — | — | — | — | — | — | — | — | |
| MARTplus_variant=true2023.10 | — | — | 83.07 | 55.57 | — | — | — | — | — | 65.65 | 54.87 | 58.7 |