Adversarial Robustness on CIFAR-100 (test)
66.74Natural AccFAT
Evaluation results
| Method | Links | |||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FATWRN=32-10, Aug=Pad and Crop, Epo=100 with robust overfitting mitigation, ASAM=false, TR=false, SWA=false2022.09 | 66.74 | — | — | — | — | — | — | — | — | 20.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| lambda_awd = 0.022 + ASAM (32-10)WRN=32-10, Aug=Pad and Crop, Epo=100, ASAM=true, TR=false, SWA=false2022.09 | 64.49 | — | — | — | — | — | — | — | — | 29.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| lambda_awd = 0.022 + ASAM (28-10)WRN=28-10, Aug=Pad and Crop, Epo=100, ASAM=true, TR=false, SWA=false2022.09 | 63.93 | — | — | — | — | — | — | — | — | 29.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Rebuffi et al.WRN=28-10, Aug=CutMix, Epo=400, ASAM=false, TR=false, SWA=true2022.09 | 62.97 | — | — | — | — | — | — | — | — | 29.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Multi-res backboneAdv. train=true, Model=ResNet152, Samples (#)=1282024.08 | 62.72 | 37.5 | — | — | — | — | — | — | — | — | — | — | 32.03 | 22.66 | — | — | — | — | — | — | — | — | — | — | — | |
| lambda_awd = 0.00281* + ASAMWRN=28-10, Aug=Pad and Crop, Epo=100, ASAM=true, TR=false, SWA=false2022.09 | 62.24 | — | — | — | — | — | — | — | — | 26.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Hypocritical PerturbationDefense=PGD-AT, Defense budget=8/255, Backbone=ResNet-182022.01 | 62.22 | — | — | — | — | 26.38 | 21.51 | 21.13 | 21.13 | 18.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Hypocritical PerturbationDefense budget=8/2552022.01 | 62.22 | — | — | — | — | 26.38 | 21.51 | 21.13 | 21.13 | 18.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| lambda_awd = 0.022WRN=28-10, Aug=Pad and Crop, Epo=200, ASAM=false, TR=false, SWA=false2022.09 | 61.39 | — | — | — | — | — | — | — | — | 27.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| 3-ensemble of self-ensemblesAdv. train=true, Model=ResNet152, Samples (#)=5122024.08 | 61.17 | 51.28 | — | — | — | — | — | — | — | — | — | — | 44.6 | 43.04 | — | — | — | — | — | — | — | — | — | — | — | |
| MAIL-ATWRN=32-10, Aug=Pad and Crop, Epo=100 with robust overfitting mitigation, ASAM=false, TR=false, SWA=false2022.09 | 60.74 | — | — | — | — | — | — | — | — | 22.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRADESWRN=32-10, Aug=Pad and Crop, Epo=100 with robust overfitting mitigation, ASAM=false, TR=true, SWA=false2022.09 | 60.73 | — | — | — | — | — | — | — | — | 24.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| lambda_awd = 0.0005WRN=28-10, Aug=Pad and Crop, Epo=200, ASAM=false, TR=false, SWA=false2022.09 | 60.15 | — | — | — | — | — | — | — | — | 22.53 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ATWRN=32-10, Aug=Pad and Crop, Epo=100 with robust overfitting mitigation, ASAM=false, TR=false, SWA=false2022.09 | 60.13 | — | — | — | — | — | — | — | — | 24.76 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MAIL-TRWRN=32-10, Aug=Pad and Crop, Epo=100 with robust overfitting mitigation, ASAM=false, TR=true, SWA=false2022.09 | 60.13 | — | — | — | — | — | — | — | — | 24.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Rebuffi et al.WRN=28-10, Aug=Pad and Crop, Epo=400, ASAM=false, TR=true, SWA=true2022.09 | 59.06 | — | — | — | — | — | — | — | — | 28.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ATHFBackbone=WideResNet34-102023.12 | 58.99 | — | — | — | — | 37.07 | 34.47 | 34.43 | 31.18 | 28.65 | 34.57 | 30.24 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-ensembleAdv. train=true, Model=ResNet152, Samples (#)=5122024.08 | 58.93 | 47.85 | — | — | — | — | — | — | — | — | — | — | 36.72 | 33.98 | — | — | — | — | — | — | — | — | — | — | — | |
| ATPGDBackbone=WideResNet34-102023.12 | 58.81 | — | — | — | — | 33.75 | 30.78 | 30.63 | 29.41 | 25.84 | 30.8 | 27.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRADESHFBackbone=WideResNet34-102023.12 | 58.7 | — | — | — | — | 35.59 | 34.49 | 34.45 | 32.63 | 30.29 | 32.49 | 31.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GAIRATWRN=32-10, Aug=Pad and Crop, Epo=100 with robust overfitting mitigation, ASAM=false, TR=false, SWA=false2022.09 | 58.43 | — | — | — | — | — | — | — | — | 17.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| lambda_awd = 0.0005 + ASAMWRN=28-10, Aug=Pad and Crop, Epo=100, ASAM=true, TR=false, SWA=false2022.09 | 58.09 | — | — | — | — | — | — | — | — | 22.55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HELPHFBackbone=WideResNet34-102023.12 | 58.05 | — | — | — | — | 36.4 | 34.42 | 34.38 | 32.58 | 31.36 | 33.27 | 32.58 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ATHFBackbone=PreAct ResNet-182023.12 | 57.45 | — | — | — | — | 34.91 | 32.32 | 32.31 | 29.43 | 27.15 | 32.38 | 28.51 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| lambda_AdaDecay = 0.002*WRN=28-10, Aug=Pad and Crop, Epo=200, ASAM=false, TR=false, SWA=false2022.09 | 57.17 | — | — | — | — | — | — | — | — | 24.18 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AWPHFBackbone=WideResNet34-102023.12 | 57.16 | — | — | — | — | 35.11 | 33.02 | 32.94 | 31.09 | 31.2 | 33.06 | 31.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARTHFBackbone=WideResNet34-102023.12 | 56.19 | — | — | — | — | 35.31 | 33.52 | 33.39 | 31.4 | 30.34 | 32.22 | 31.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| None (clean)Defense=PGD-AT, Defense budget=8/255, Backbone=ResNet-182022.01 | 56.15 | — | — | — | — | 31.5 | 28.38 | 28.28 | 26.53 | 24.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| None (clean)Defense budget=8/2552022.01 | 56.15 | — | — | — | — | 31.5 | 28.38 | 28.28 | 26.53 | 24.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRADESHFBackbone=PreAct ResNet-182023.12 | 55.75 | — | — | — | — | 34.96 | 32.29 | 32.19 | 29.06 | 27 | 31.41 | 27.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ATPGDBackbone=PreAct ResNet-182023.12 | 55.57 | — | — | — | — | 31.57 | 28.79 | 28.66 | 26.77 | 24.44 | 28.91 | 25.71 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRADESBackbone=WideResNet34-102023.12 | 55.54 | — | — | — | — | 34.77 | 32.86 | 32.81 | 31.37 | 27.36 | 31.88 | 28.57 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AWPHFBackbone=PreAct ResNet-182023.12 | 55.37 | — | — | — | — | 35.21 | 33.14 | 33.01 | 30.16 | 27.85 | 31.56 | 27.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HELPBackbone=WideResNet34-102023.12 | 55.32 | — | — | — | — | 34.47 | 31.54 | 31.51 | 29.75 | 28.19 | 31.65 | 29.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HELPHFBackbone=PreAct ResNet-182023.12 | 55.23 | — | — | — | — | 35.46 | 33.65 | 33.32 | 30.24 | 27.87 | 32.07 | 28.26 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AWPWRN=32-10, Aug=Pad and Crop, Epo=100 with robust overfitting mitigation, ASAM=false, TR=false, SWA=false2022.09 | 55.16 | — | — | — | — | — | — | — | — | 25.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARTHFBackbone=PreAct ResNet-182023.12 | 54.74 | — | — | — | — | 35.02 | 32.78 | 32.55 | 29.82 | 27.51 | 30.62 | 27.93 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRADESBackbone=PreAct ResNet-182023.12 | 54.61 | — | — | — | — | 32.86 | 30.1 | 29.97 | 27.24 | 25.81 | 30.18 | 26.18 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AWPBackbone=WideResNet34-102023.12 | 54.58 | — | — | — | — | 34.03 | 32.24 | 32.04 | 30.67 | 28.75 | 31.53 | 29.63 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AWPBackbone=PreAct ResNet-182023.12 | 54.19 | — | — | — | — | 33.21 | 30.71 | 30.62 | 28.05 | 25.49 | 29.93 | 26.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HELPBackbone=PreAct ResNet-182023.12 | 54.17 | — | — | — | — | 33.56 | 31.15 | 30.96 | 28.42 | 25.61 | 29.86 | 26.17 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARTWRN=32-10, Aug=Pad and Crop, Epo=100 with robust overfitting mitigation, ASAM=false, TR=false, SWA=false2022.09 | 54.08 | — | — | — | — | — | — | — | — | 25.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARTBackbone=WideResNet34-102023.12 | 53.84 | — | — | — | — | 34.42 | 31.62 | 31.51 | 30.14 | 27.08 | 31.64 | 28.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MARTBackbone=PreAct ResNet-182023.12 | 53.41 | — | — | — | — | 32.85 | 30.53 | 30.39 | 27.93 | 25.31 | 29.77 | 26.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Adversarial PoisoningDefense=PGD-AT, Defense budget=8/255, Backbone=ResNet-182022.01 | 52.14 | — | — | — | — | 28.59 | 26.19 | 26.09 | 24.36 | 22.71 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Adversarial PoisoningDefense budget=8/2552022.01 | 52.14 | — | — | — | — | 28.59 | 26.19 | 26.09 | 24.36 | 22.71 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ADBMNorm=L∞, Setting=Preprocessor-blind2025.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 60.19 | 28.57 | 58.92 | 28 | 58.82 | 30.69 | 57 | 28.18 | — | — | — | |
| BaseModel=ResNet322026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 76.01 | 89.69 | 29.59 | |
| BaseModel=VGG162026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 72.44 | 88.75 | 25.66 | |
| DiffPureNorm=L∞, Setting=Preprocessor-blind2025.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 60.28 | 35.57 | 60.14 | 35.69 | 61.63 | 37.71 | 60.03 | 35.4 | — | — | — | |
| Fast ATSchedule=cyclic, Subspace=-2021.11 | — | 16.72 | 0 | 31.51 | 47.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Fast ATSchedule=piecewise, Subspace=-2021.11 | — | 17.84 | 0 | 41.54 | 46.46 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Fast Sub-ATSchedule=constant, Subspace=Fast AT2021.11 | — | 24.97 | 24.55 | 52.74 | 53.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FlowPureCWNorm=L∞, Setting=Preprocessor-blind2025.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.98 | 40.98 | 43 | 43 | 78.65 | 78.65 | 32.01 | 32.01 | — | — | — | |
| FlowPureGaussNorm=L∞, Setting=Preprocessor-blind2025.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 59.8 | 35.75 | 59.55 | 35.95 | 60.56 | 37.26 | 59.42 | 35.37 | — | — | — | |
| FlowPurePGDNorm=L∞, Setting=Preprocessor-blind2025.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 59.63 | 59.63 | 49.12 | 49.12 | 78.81 | 78.81 | 18.05 | 18.05 | — | — | — | |
| Free ATSchedule=piecewise, Subspace=-2021.11 | — | 23.5 | 22.93 | 50.91 | 51.64 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GATSchedule=piecewise, Subspace=-2021.11 | — | 23.12 | 20.4 | 58.33 | 57.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GAT Sub-ATSchedule=constant, Subspace=GAT2021.11 | — | 23.4 | 22.96 | 57.71 | 58.45 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GDMPNorm=L∞, Setting=Preprocessor-blind2025.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 64.8 | 41.89 | 64.43 | 41.32 | 66.01 | 43.67 | 63.61 | 39.96 | — | — | — | |
| GradAlignSchedule=cyclic, Subspace=-2021.11 | — | 24.57 | 24.22 | 50.82 | 51.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GradAlignSchedule=piecewise, Subspace=-2021.11 | — | 23.8 | 15.6 | 49.3 | 53.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GradAlign Sub-ATSchedule=constant, Subspace=GradAlign2021.11 | — | 25.69 | 25.46 | 52.65 | 52.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LMNorm=L∞, Setting=Preprocessor-blind2025.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.99 | 10.69 | 29.15 | 11.31 | 39.78 | 17.27 | 35.92 | 15.89 | — | — | — | |
| PARDEFModel=ResNet322026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.91 | 52.33 | 6.98 | |
| PARDEFModel=VGG162026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 35.07 | 59.04 | 19.31 | |
| Single ATSchedule=piecewise, Subspace=-2021.11 | — | 16.18 | 0.91 | 55.88 | 59.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SOTA #1Adv. train=true, Model=[40]2024.08 | — | 42.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |