Image Classification on Distribution Shift Robustness Suite various (test)
84.31IN-V2 AccBest model on each test set (oracle)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Best model on each test set (oracle)Backbone=ViT-G/14, Selection=Oracle (each test set)2022.03 | 84.31 | 95.04 | 73.73 | 79.03 | 92.16 | 84.68 | |
| Greedy soupBackbone=ViT-G/142022.03 | 84.22 | 95.46 | 74.23 | 78.52 | 92.67 | 85.02 | |
| Greedy ensembleBackbone=ViT-G/142022.03 | 84.14 | 94.85 | 73.07 | 77.87 | 91.69 | 84.33 | |
| Best model on held out val setBackbone=ViT-G/14, Selection=Held out validation set2022.03 | 83.76 | 95.04 | 73.16 | 78.2 | 91.75 | 84.38 | |
| ViT/G-14Backbone=ViT-G/14, Selection=Reevaluated2022.03 | 83.39 | 94.38 | 72.37 | 71.16 | 89 | 82.06 | |
| ViT/G-14Backbone=ViT-G/14, Selection=Original (Zhai et al., 2021)2022.03 | 83.33 | — | — | 70.53 | — | — |