Robustness to Corruptions on ImageNet-C (test)
9.58mCETPT
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TPTEvaluation Protocol=Online, Backbone=CLIP-RN502025.08 | 9.58 | 1.43 | 1.94 | 1.42 | 8.02 | 2.74 | 5.34 | 10.97 | 10.59 | 12.92 | 16.17 | 35.67 | 4.45 | 3.73 | 11.56 | 16.68 | — | |
| DiffTPTEvaluation Protocol=Online, Backbone=CLIP-RN502025.08 | 11.18 | 2.69 | 3.59 | 2.08 | 10.5 | 3.9 | 8.62 | 12.74 | 11.31 | 15.03 | 19.64 | 37.73 | 4.98 | 4.74 | 13.58 | 16.56 | — | |
| CLIPEvaluation Protocol=Online, Backbone=CLIP-RN502025.08 | 11.27 | 2.43 | 3.07 | 2.52 | 9.54 | 3.4 | 7.46 | 12.62 | 12.29 | 15.72 | 22.08 | 41.69 | 6.24 | 4.67 | 11.01 | 14.24 | — | |
| TransCLIPEvaluation Protocol=Transductive, Backbone=CLIP-RN502025.08 | 11.72 | 2.87 | 3.14 | 3.13 | 9.38 | 4.17 | 7.14 | 12.42 | 11.87 | 15.04 | 23.93 | 44.33 | 6.45 | 6.11 | 11.03 | 14.85 | — | |
| ZLaPEvaluation Protocol=Transductive, Backbone=CLIP-RN502025.08 | 11.82 | 1.17 | 2.33 | 1.65 | 10.3 | 3.54 | 7.99 | 13.47 | 13.66 | 17.15 | 23.2 | 44.67 | 6.55 | 5.15 | 11.61 | 14.23 | — | |
| TDAEvaluation Protocol=Online, Backbone=CLIP-RN502025.08 | 12.13 | 2.54 | 3.26 | 2.72 | 9.84 | 4.4 | 7.38 | 13.74 | 13.74 | 17.16 | 23.76 | 44.16 | 7 | 5.79 | 11.24 | 15.26 | — | |
| ADAPTEvaluation Protocol=Online, Backbone=CLIP-RN502025.08 | 12.72 | 2.91 | 3.77 | 2.92 | 10.54 | 4.44 | 8.57 | 14.34 | 13.85 | 17.84 | 24.56 | 45.67 | 7.76 | 5.85 | 11.96 | 15.86 | — | |
| ADAPTEvaluation Protocol=Transductive, Backbone=CLIP-RN502025.08 | 14.81 | 4.05 | 4.8 | 4.03 | 12.26 | 6.11 | 10.92 | 17.02 | 16.67 | 20.62 | 27.77 | 47.31 | 9.2 | 8.56 | 14.61 | 18.15 | — | |
| StatAEvaluation Protocol=Transductive, Backbone=ViT-B/162025.08 | 22.4 | 9.58 | 10.52 | 10.12 | 20.23 | 13.29 | 20.38 | 18.84 | 31.3 | 29.8 | 34.58 | 54.79 | 11.24 | 11.8 | 26.31 | 33.2 | — | |
| CLIPEvaluation Protocol=Online, Backbone=ViT-B/162025.08 | 25.5 | 13.25 | 14.16 | 13.48 | 24.25 | 15.71 | 24.46 | 22.6 | 33.08 | 31.06 | 37.61 | 55.62 | 17.11 | 13.43 | 33.04 | 33.7 | — | |
| ZLaPEvaluation Protocol=Transductive, Backbone=ViT-B/162025.08 | 26.42 | 12.83 | 14.03 | 13.27 | 24.88 | 16.13 | 25.77 | 24.36 | 34.43 | 32.63 | 38.56 | 58.42 | 17.53 | 14.21 | 33.72 | 35.52 | — | |
| TransCLIPEvaluation Protocol=Transductive, Backbone=ViT-B/162025.08 | 27.07 | 14.82 | 16.11 | 15.6 | 25.35 | 16.4 | 25.53 | 23.22 | 34.58 | 32.47 | 39.65 | 59.04 | 17.72 | 14.76 | 35.22 | 35.53 | — | |
| TPTEvaluation Protocol=Online, Backbone=ViT-B/162025.08 | 27.43 | 10.64 | 11.94 | 10.92 | 27.56 | 15.48 | 26.16 | 26.94 | 36.74 | 34.28 | 39.38 | 60.22 | 16.96 | 15.64 | 40.74 | 37.9 | — | |
| DMNEvaluation Protocol=Online, Backbone=ViT-B/162025.08 | 27.46 | 14.33 | 15.33 | 14.69 | 26.06 | 17.19 | 26.61 | 25.23 | 34.81 | 33.48 | 38.93 | 58.7 | 19.38 | 15.4 | 35.32 | 36.49 | — | |
| DiffTPTEvaluation Protocol=Online, Backbone=ViT-B/162025.08 | 27.49 | 12.97 | 13.6 | 13.21 | 25.63 | 16.96 | 26.74 | 25.4 | 35.99 | 34.57 | 39.83 | 59.01 | 17.32 | 17.16 | 38.43 | 35.47 | — | |
| TDAEvaluation Protocol=Online, Backbone=ViT-B/162025.08 | 28.34 | 15.42 | 16.46 | 16.03 | 26.53 | 17.91 | 27.35 | 25.9 | 36.5 | 34.84 | 40.53 | 58.57 | 20.16 | 16.62 | 35.65 | 36.69 | — | |
| ADAPTEvaluation Protocol=Online, Backbone=ViT-B/162025.08 | 28.56 | 15.76 | 16.84 | 15.9 | 26.3 | 18.01 | 27.31 | 25.54 | 36.19 | 34.67 | 40.96 | 60.29 | 19.95 | 16.09 | 37.44 | 37.22 | — | |
| ADAPTEvaluation Protocol=Transductive, Backbone=ViT-B/162025.08 | 30.29 | 17.71 | 18.81 | 18.09 | 27.98 | 19.78 | 29 | 27.38 | 38.09 | 36.44 | 42.43 | 62.21 | 21.94 | 18.4 | 39.89 | 38.23 | — | |
| PIXMIX2021.12 | 30.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TinyMIMModel Size=ViT-B2023.01 | 32.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TinyMIMModel Size=ViT-S2023.01 | 35.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeiTModel Size=ViT-B2023.01 | 36.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MAEModel Size=ViT-B2023.01 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MAEModel Size=ViT-S2023.01 | 40.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeiTModel Size=ViT-S2023.01 | 41.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RVT-B*Group=Transformers, FLOPs (G)=17.7, Params (M)=91.82021.05 | 46.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ConViT-BGroup=Transformers, FLOPs (G)=17.7, Params (M)=86.52021.05 | 46.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RVT-BGroup=Transformers, FLOPs (G)=17.7, Params (M)=86.22021.05 | 47.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixup2021.12 | 48 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T2T-ViT t-24Group=Transformers, FLOPs (G)=15.0, Params (M)=64.12021.05 | 48 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PiT-BGroup=Transformers, FLOPs (G)=12.5, Params (M)=73.82021.05 | 48.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeiT-BGroup=Transformers, FLOPs (G)=17.6, Params (M)=86.62021.05 | 48.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RVT-S*Group=Transformers, FLOPs (G)=4.7, Params (M)=23.32021.05 | 49.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ConViT-SGroup=Transformers, FLOPs (G)=5.4, Params (M)=27.82021.05 | 49.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Baseline2021.12 | 50 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TinyMIMModel Size=ViT-T2023.01 | 50.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RVT-SGroup=Transformers, FLOPs (G)=4.7, Params (M)=22.12021.05 | 50.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Cutout2021.12 | 51.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CutMix2021.12 | 51.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PiT-SGroup=Transformers, FLOPs (G)=2.9, Params (M)=23.52021.05 | 52.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TNT-SGroup=Transformers, FLOPs (G)=5.2, Params (M)=23.82021.05 | 53.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T2T-ViT t-14Group=Transformers, FLOPs (G)=6.1, Params (M)=21.52021.05 | 53.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepAugmentGroup=CNNs, FLOPs (G)=4.1, Params (M)=25.62021.05 | 53.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeiTModel Size=ViT-T2023.01 | 54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Swin-BGroup=Transformers, FLOPs (G)=15.4, Params (M)=87.82021.05 | 54.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeiT-SGroup=Transformers, FLOPs (G)=4.6, Params (M)=22.12021.05 | 54.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MAEModel Size=ViT-T2023.01 | 55.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RVT-Ti*Group=Transformers, FLOPs (G)=1.3, Params (M)=10.92021.05 | 57 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RVT-TiGroup=Transformers, FLOPs (G)=1.3, Params (M)=8.62021.05 | 58.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PVT-LargeGroup=Transformers, FLOPs (G)=9.8, Params (M)=61.42021.05 | 59.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Swin-TGroup=Transformers, FLOPs (G)=4.5, Params (M)=28.32021.05 | 62 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ANTGroup=CNNs, FLOPs (G)=4.1, Params (M)=25.62021.05 | 63 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ResNeXt50-32x4dGroup=CNNs, FLOPs (G)=4.3, Params (M)=25.02021.05 | 64.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AugMixGroup=CNNs, FLOPs (G)=4.1, Params (M)=25.62021.05 | 65.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ResNet-50*Group=CNNs, FLOPs (G)=4.1, Params (M)=25.62021.05 | 65.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PVT-SmallGroup=Transformers, FLOPs (G)=3.8, Params (M)=24.52021.05 | 66.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Debiased CNNGroup=CNNs, FLOPs (G)=4.1, Params (M)=25.62021.05 | 67.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Anti-Aliased CNNGroup=CNNs, FLOPs (G)=4.2, Params (M)=25.62021.05 | 68.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ConViT-TiGroup=Transformers, FLOPs (G)=1.4, Params (M)=5.72021.05 | 68.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RegNetY-4GFGroup=CNNs, FLOPs (G)=4.0, Params (M)=20.62021.05 | 68.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PiT-TiGroup=Transformers, FLOPs (G)=0.7, Params (M)=4.92021.05 | 69.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TENET (Aug.)Backbone=ResNet-50, Protocol=180-epoch, Data Augmentation=Augmix2021.03 | 69.6 | 69 | 67 | 68 | 72 | 81 | 66 | 69 | 74 | 74 | 65 | 59 | 60 | 82 | 69 | 70 | — | |
| AugmixBackbone=ResNet-50, Protocol=180-epoch2021.03 | 70.5 | 68 | 69 | 70 | 73 | 81 | 69 | 67 | 75 | 73 | 67 | 61 | 61 | 80 | 71 | 72 | — | |
| TENET (Aug.)Backbone=ResNet-50, Protocol=90-epoch, Data Augmentation=Augmix2021.03 | 71.1 | 69 | 69 | 69 | 69 | 87 | 69 | 70 | 76 | 75 | 64 | 56 | 69 | 82 | 72 | 73 | — | |
| EfficientNet-B4Group=CNNs, FLOPs (G)=4.4, Params (M)=19.32021.05 | 71.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeiT-TiGroup=Transformers, FLOPs (G)=1.3, Params (M)=5.72021.05 | 71.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AugmixBackbone=ResNet-50, Protocol=90-epoch2021.03 | 73.9 | 71 | 71 | 71 | 72 | 88 | 72 | 72 | 78 | 78 | 67 | 60 | 72 | 86 | 75 | 76 | — | |
| WResNet50 (Haar)Backbone=ResNet-502021.03 | 75.3 | 77 | 79 | 79 | 71 | 86 | 77 | 77 | 80 | 75 | 66 | 57 | 71 | 84 | 75 | 77 | — | |
| TENETBackbone=ResNet-50, Protocol=90-epoch2021.03 | 75.3 | 73 | 78 | 75 | 74 | 87 | 76 | 80 | 79 | 78 | 67 | 63 | 73 | 84 | 72 | 71 | — | |
| ResNet-50Group=CNNs, FLOPs (G)=4.1, Params (M)=25.62021.05 | 76.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CutoutBackbone=ResNet-502021.03 | 77.7 | 79 | 81 | 80 | 77 | 90 | 80 | 81 | 80 | 78 | 70 | 61 | 74 | 87 | 74 | 75 | — | |
| PVT-TinyGroup=Transformers, FLOPs (G)=1.9, Params (M)=13.22021.05 | 79.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaselineBackbone=ResNet-502021.03 | 80.6 | 79 | 80 | 82 | 82 | 90 | 84 | 80 | 86 | 81 | 75 | 65 | 79 | 91 | 77 | 80 | — | |
| Inception v3Group=CNNs, FLOPs (G)=5.7, Params (M)=27.22021.05 | 80.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TENET TrainingTraining Schedule=90-epoch Protocol [21], Comparison Baseline=Augmix [14]2021.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.8 |