Image Classification on ImageNet-1K (val) (Robustness Metrics)
86.4Robust AccuracyClean Baseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Clean BaselineBackbone=ViT-B-384, Resolution=384x384, Attack Type=None2022.03 | 86.4 | — | |
| Clean BaselineBackbone=ViT-B, Resolution=224x224, Attack Type=None2022.03 | 85 | — | |
| Clean BaselineBackbone=DeiT-B, Resolution=224x224, Attack Type=None2022.03 | 82 | — | |
| Clean BaselineBackbone=DeiT-B-384, Resolution=384x384, Attack Type=None2022.03 | 82 | — | |
| Clean BaselineBackbone=ResNet50, Resolution=224x224, Attack Type=None2022.03 | 80.6 | — | |
| Clean BaselineBackbone=ViT-T, Resolution=224x224, Attack Type=None2022.03 | 73.5 | — | |
| Patch-RSBackbone=DeiT-B-384, Resolution=384x384, Attack Type=Patch-RS2022.03 | 71.2 | — | |
| Patch-RSBackbone=ResNet50, Resolution=224x224, Attack Type=Patch-RS2022.03 | 70.8 | — | |
| Clean BaselineBackbone=DeiT-T, Resolution=224x224, Attack Type=None2022.03 | 69.4 | — | |
| Patch-FoolBackbone=DeiT-B-384, Resolution=384x384, Attack Type=Patch-Fool2022.03 | 68.2 | — | |
| Patch-RSBackbone=ViT-B-384, Resolution=384x384, Attack Type=Patch-RS2022.03 | 65.9 | — | |
| SABackbone=resnet50 (Engstrom et al. [56]), Query budget=500, Threat Model=l-infinity, epsilon=4/2552021.11 | 59.8 | 63.4 | |
| PGD-250Loss Function=Lce, Momentum=false, Backbone=DeiT-B-3842022.03 | 58.8 | — | |
| Patch-RSBackbone=DeiT-B, Resolution=224x224, Attack Type=Patch-RS2022.03 | 57.3 | — | |
| SABackbone=resnet50 (Engstrom et al. [56]), Query budget=1000, Threat Model=l-infinity, epsilon=4/2552021.11 | 57.2 | 63.4 | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=false, Backbone=DeiT-B-3842022.03 | 57.2 | — | |
| PGD-250Loss Function=Lce, Momentum=false, Backbone=ResNet502022.03 | 55.1 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=false, Backbone=DeiT-B-3842022.03 | 55.1 | — | |
| AABackbone=resnet50 (Engstrom et al. [56]), Query budget=500, Threat Model=l-infinity, epsilon=4/2552021.11 | 54.6 | 63.4 | |
| SABackbone=resnet50 (Engstrom et al. [56]), Query budget=2500, Threat Model=l-infinity, epsilon=4/2552021.11 | 52.9 | 63.4 | |
| Patch-FoolBackbone=DeiT-B, Resolution=224x224, Attack Type=Patch-Fool2022.03 | 52.9 | — | |
| AABackbone=resnet50 (Engstrom et al. [56]), Query budget=1000, Threat Model=l-infinity, epsilon=4/2552021.11 | 52.8 | 63.4 | |
| MSABackbone=resnet50 (Engstrom et al. [56]), Query budget=500, Threat Model=l-infinity, epsilon=4/2552021.11 | 52.5 | 63.4 | |
| MSABackbone=resnet50 (Engstrom et al. [56]), Query budget=1000, Threat Model=l-infinity, epsilon=4/2552021.11 | 50.8 | 63.4 | |
| SABackbone=resnet18 (Salman et al. [55]), Query budget=500, Threat Model=l-infinity, epsilon=4/2552021.11 | 50.6 | 52.5 | |
| AABackbone=resnet50 (Engstrom et al. [56]), Query budget=2500, Threat Model=l-infinity, epsilon=4/2552021.11 | 50.3 | 63.4 | |
| Patch-RSBackbone=ViT-B, Resolution=224x224, Attack Type=Patch-RS2022.03 | 49.8 | — | |
| PGD-250Loss Function=Lce, Momentum=true, Backbone=ResNet502022.03 | 49 | — | |
| SABackbone=resnet50 (Engstrom et al. [56]), Query budget=5000, Threat Model=l-infinity, epsilon=4/2552021.11 | 48.6 | 63.4 | |
| SABackbone=resnet18 (Salman et al. [55]), Query budget=1000, Threat Model=l-infinity, epsilon=4/2552021.11 | 48.1 | 52.5 | |
| AABackbone=resnet50 (Engstrom et al. [56]), Query budget=5000, Threat Model=l-infinity, epsilon=4/2552021.11 | 48.1 | 63.4 | |
| MSABackbone=resnet50 (Engstrom et al. [56]), Query budget=2500, Threat Model=l-infinity, epsilon=4/2552021.11 | 48 | 63.4 | |
| Patch-FoolBackbone=ViT-B-384, Resolution=384x384, Attack Type=Patch-Fool2022.03 | 47.8 | — | |
| MSABackbone=resnet50 (Engstrom et al. [56]), Query budget=5000, Threat Model=l-infinity, epsilon=4/2552021.11 | 45.8 | 63.4 | |
| AABackbone=resnet18 (Salman et al. [55]), Query budget=500, Threat Model=l-infinity, epsilon=4/2552021.11 | 45.2 | 52.5 | |
| Patch-RSBackbone=DeiT-T, Resolution=224x224, Attack Type=Patch-RS2022.03 | 44 | — | |
| SABackbone=resnet18 (Salman et al. [55]), Query budget=2500, Threat Model=l-infinity, epsilon=4/2552021.11 | 43.9 | 52.5 | |
| AABackbone=resnet18 (Salman et al. [55]), Query budget=1000, Threat Model=l-infinity, epsilon=4/2552021.11 | 43.5 | 52.5 | |
| MSABackbone=resnet18 (Salman et al. [55]), Query budget=500, Threat Model=l-infinity, epsilon=4/2552021.11 | 43.3 | 52.5 | |
| MSABackbone=resnet18 (Salman et al. [55]), Query budget=1000, Threat Model=l-infinity, epsilon=4/2552021.11 | 41.7 | 52.5 | |
| PGD-250Loss Function=Lce, Momentum=true, Backbone=DeiT-B-3842022.03 | 41.7 | — | |
| AABackbone=resnet18 (Salman et al. [55]), Query budget=2500, Threat Model=l-infinity, epsilon=4/2552021.11 | 41 | 52.5 | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=true, Backbone=DeiT-B-3842022.03 | 40.6 | — | |
| SABackbone=resnet18 (Salman et al. [55]), Query budget=5000, Threat Model=l-infinity, epsilon=4/2552021.11 | 40.3 | 52.5 | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=true, Backbone=DeiT-B-3842022.03 | 39.8 | — | |
| MSABackbone=resnet18 (Salman et al. [55]), Query budget=2500, Threat Model=l-infinity, epsilon=4/2552021.11 | 39.1 | 52.5 | |
| AABackbone=resnet18 (Salman et al. [55]), Query budget=5000, Threat Model=l-infinity, epsilon=4/2552021.11 | 39 | 52.5 | |
| MSABackbone=resnet18 (Salman et al. [55]), Query budget=5000, Threat Model=l-infinity, epsilon=4/2552021.11 | 37.8 | 52.5 | |
| PGD-250Loss Function=Lce, Momentum=false, Backbone=DeiT-B2022.03 | 36 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=false, Backbone=DeiT-B2022.03 | 35.5 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=false, Backbone=DeiT-B2022.03 | 33.7 | — | |
| PGD-250Loss Function=Lce, Momentum=false, Backbone=ViT-B-3842022.03 | 31.2 | — | |
| PGD-250Loss Function=Lce, Momentum=false, Backbone=DeiT-T2022.03 | 19.8 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=true, Backbone=DeiT-B2022.03 | 19.3 | — | |
| Patch-RSBackbone=ViT-T, Resolution=224x224, Attack Type=Patch-RS2022.03 | 18.6 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=false, Backbone=ViT-B-3842022.03 | 18 | — | |
| Patch-FoolBackbone=ViT-B, Resolution=224x224, Attack Type=Patch-Fool2022.03 | 17.5 | — | |
| PGD-250Loss Function=Lce, Momentum=true, Backbone=DeiT-B2022.03 | 16.8 | — | |
| PGD-250Loss Function=Lce, Momentum=false, Backbone=ViT-B2022.03 | 13.5 | — | |
| PGD-250Loss Function=Lce, Momentum=true, Backbone=ViT-B-3842022.03 | 13.2 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=false, Backbone=DeiT-T2022.03 | 13.1 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=true, Backbone=DeiT-B2022.03 | 13.1 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=false, Backbone=ViT-B-3842022.03 | 13 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=false, Backbone=DeiT-T2022.03 | 11.7 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=false, Backbone=ViT-B2022.03 | 5 | — | |
| Patch-FoolBackbone=DeiT-T, Resolution=224x224, Attack Type=Patch-Fool2022.03 | 3.2 | — | |
| PGD-250Loss Function=Lce, Momentum=true, Backbone=ViT-B2022.03 | 3.1 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=false, Backbone=ViT-B2022.03 | 2.6 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=true, Backbone=ViT-B-3842022.03 | 2.5 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=true, Backbone=ViT-B-3842022.03 | 1.9 | — | |
| PGD-250Loss Function=Lce, Momentum=true, Backbone=DeiT-T2022.03 | 1.5 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=false, Backbone=ViT-T2022.03 | 0.5 | — | |
| Patch-FoolBackbone=ViT-T, Resolution=224x224, Attack Type=Patch-Fool2022.03 | 0.3 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=false, Backbone=ViT-T2022.03 | 0.3 | — | |
| PGD-250Loss Function=Lce, Momentum=false, Backbone=ViT-T2022.03 | 0.1 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=true, Backbone=ViT-B2022.03 | 0.1 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=true, Backbone=ViT-B2022.03 | 0.1 | — | |
| PGD-250Loss Function=Lce, Momentum=true, Backbone=ViT-T2022.03 | 0 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=true, Backbone=ViT-T2022.03 | 0 | — | |
| Attention-FoolLoss Function=Lce + Lkq, Momentum=true, Backbone=DeiT-T2022.03 | 0 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=true, Backbone=ViT-T2022.03 | 0 | — | |
| Attention-FoolLoss Function=Lce + Lkq*, Momentum=true, Backbone=DeiT-T2022.03 | 0 | — |