Image Classification on ImageNet Natural Distribution Shifts Suite (test)
74.91Top-1 Accuracy (ImageNet)GRPO-TTA
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GRPO-TTABackbone=ViT-B/162026.05 | 74.91 | 72.97 | 69.01 | 85.77 | 55.17 | 71.56 | 70.73 | |
| ZEROBackbone=CLIP-ViT-B-16, Pre-training=OpenAI, Prompting Strategy=CoOp-learned prompts2024.05 | 74.12 | 61.57 | 67.15 | 78.43 | 49.77 | 66.21 | — | |
| TPTBackbone=CLIP-ViT-B-16, Pre-training=OpenAI, Prompting Strategy=CoOp-learned prompts2024.05 | 73.64 | 57.77 | 66.72 | 78.03 | 49.56 | 65.14 | — | |
| RLCFBackbone=ViT-B/16, Optimization Source=Results obtained using current paper's optimizations2026.05 | 73.3 | 71.23 | 67.6 | 85.34 | 54.29 | 70.35 | 69.62 | |
| DMN-ZSBackbone=ViT-B/162024.10 | 72.25 | 58.28 | 65.17 | 78.55 | 53.2 | 65.49 | 63.8 | |
| DMN-ZSBackbone=ViT-B/16, Buffer=true2026.06 | 72.25 | 58.28 | 65.17 | 78.55 | 53.2 | 65.49 | 63.8 | |
| ComMemBackbone=ViT-B/16, Buffer=true2026.06 | 72.16 | 61.32 | 65.38 | 81.52 | 53.21 | 66.72 | 65.36 | |
| DPEBackbone=ViT-B/162024.10 | 71.91 | 59.63 | 65.44 | 80.4 | 52.26 | 65.93 | 64.43 | |
| DPEBackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 71.91 | 59.63 | 65.44 | 80.4 | 52.26 | 65.93 | 64.43 | |
| DPEBackbone=ViT-B/16, Buffer=true2026.06 | 71.91 | 59.63 | 65.44 | 80.4 | 52.26 | 65.93 | 64.43 | |
| CoOpBackbone=ViT-B/16, 16-shot=true2024.10 | 71.51 | 49.71 | 64.2 | 75.21 | 47.99 | 61.72 | 59.28 | |
| Zero-ShotBackbone=CLIP-ViT-B-16, Pre-training=OpenAI, Prompting Strategy=CoOp-learned prompts2024.05 | 71.51 | 49.71 | 64.2 | 75.21 | 47.99 | 61.72 | — | |
| CoOpBackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 71.51 | 49.71 | 64.2 | 75.21 | 47.99 | 61.72 | 59.28 | |
| CoOpBackbone=ViT-B/16, Buffer=false2026.06 | 71.51 | 49.71 | 64.2 | 75.21 | 47.99 | 61.72 | 59.28 | |
| CoCoOpBackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 71.02 | 50.63 | 64.07 | 76.18 | 48.75 | 62.13 | 59.91 | |
| DOTABackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 70.69 | 61.5 | 66.31 | 81.21 | 51.84 | 66.31 | 65.22 | |
| DiffTPTBackbone=ViT-B/162024.10 | 70.3 | 55.68 | 65.1 | 75 | 46.8 | 62.28 | 60.52 | |
| DiffTPTBackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 70.3 | 55.68 | 65.1 | 75 | 46.8 | 62.28 | 60.65 | |
| DiffTPTBackbone=ViT-B/16, Buffer=false2026.06 | 70.3 | 55.68 | 65.1 | 75 | 46.8 | 62.28 | 60.52 | |
| TPSBackbone=ViT-B/162024.10 | 70.19 | 60.08 | 64.73 | 80.27 | 49.95 | 65.04 | 63.76 | |
| TPSBackbone=ViT-B/16, Buffer=true2026.06 | 70.19 | 60.08 | 64.73 | 80.27 | 49.95 | 65.04 | 63.76 | |
| DynaPromptBackbone=ViT-B/16, Buffer=true2026.06 | 69.61 | 56.17 | 64.67 | 78.17 | 48.22 | 63.37 | 61.81 | |
| CLIPTTABackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 69.6 | 54 | 62.7 | 80.2 | 50.8 | 63.4 | 61.93 | |
| TDABackbone=ViT-B/162024.10 | 69.51 | 60.11 | 64.67 | 80.24 | 50.54 | 65.01 | 63.89 | |
| TDABackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 69.51 | 60.11 | 64.67 | 80.24 | 50.54 | 65.01 | 63.89 | |
| TDABackbone=ViT-B/16, Buffer=true2026.06 | 69.51 | 60.11 | 64.67 | 80.24 | 50.54 | 65.01 | 63.89 | |
| TPTBackbone=ViT-B/162024.10 | 68.98 | 54.77 | 63.45 | 77.06 | 47.94 | 62.44 | 60.81 | |
| TPTBackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 68.98 | 54.77 | 63.45 | 77.06 | 47.94 | 62.44 | 60.81 | |
| TPTBackbone=ViT-B/16, Buffer=false2026.06 | 68.98 | 54.77 | 63.45 | 77.06 | 47.94 | 62.44 | 60.81 | |
| EnsembleBackbone=ViT-B/162024.10 | 68.34 | 49.89 | 61.88 | 77.65 | 48.24 | 61.2 | 59.42 | |
| EnsembleBackbone=ViT-B/16, Buffer=false2026.06 | 68.34 | 49.89 | 61.88 | 77.65 | 48.24 | 61.2 | 59.42 | |
| CLIP-ViT-B/16Backbone=ViT-B/162024.10 | 66.73 | 47.87 | 60.86 | 73.98 | 46.09 | 59.11 | 57.2 | |
| CLIPBackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 66.73 | 47.8 | 60.86 | 73.98 | 46.09 | 59.11 | 57.2 | |
| CLIP-ViT-B/16Backbone=ViT-B/16, Buffer=false2026.06 | 66.73 | 47.87 | 60.86 | 73.98 | 46.09 | 59.11 | 57.2 | |
| WATTBackbone=ViT-B/16, Optimization Source=Results reported in original papers2026.05 | 64.1 | 50.98 | 62.36 | 75.44 | 48.99 | 60.37 | 59.44 | |
| ComMemBackbone=ResNet-50, Buffer=true2026.06 | 64.05 | 32.21 | 56.95 | 65.1 | 41.09 | 51.88 | 48.84 | |
| DMN-ZSBackbone=ResNet-502024.10 | 63.87 | 28.57 | 56.12 | 61.44 | 39.84 | 49.97 | 46.49 | |
| DMN-ZSBackbone=ResNet-50, Buffer=true2026.06 | 63.87 | 28.57 | 56.12 | 61.44 | 39.84 | 49.97 | 46.49 | |
| DPEBackbone=ResNet-502024.10 | 63.41 | 30.15 | 56.72 | 63.72 | 40.03 | 50.81 | 47.66 | |
| DPEBackbone=ResNet-50, Buffer=true2026.06 | 63.41 | 30.15 | 56.72 | 63.72 | 40.03 | 50.81 | 47.66 | |
| CoOpBackbone=ResNet-50, 16-shot=true2024.10 | 63.33 | 23.06 | 55.4 | 56.6 | 34.67 | 46.61 | 42.43 | |
| CoOpBackbone=ResNet-50, Buffer=false2026.06 | 63.33 | 23.06 | 55.4 | 56.6 | 34.67 | 46.61 | 42.43 | |
| DynaPromptBackbone=ResNet-50, Buffer=true2026.06 | 61.56 | 27.84 | 55.12 | 60.63 | 35.64 | 48.16 | 44.81 | |
| TPSBackbone=ResNet-502024.10 | 61.47 | 30.48 | 54.96 | 62.87 | 37.14 | 49.38 | 46.36 | |
| TPSBackbone=ResNet-50, Buffer=true2026.06 | 61.47 | 30.48 | 54.96 | 62.87 | 37.14 | 49.38 | 46.36 | |
| TDABackbone=ResNet-502024.10 | 61.35 | 30.29 | 55.54 | 62.58 | 38.12 | 49.58 | 46.63 | |
| TDABackbone=ResNet-50, Buffer=true2026.06 | 61.35 | 30.29 | 55.54 | 62.58 | 38.12 | 49.58 | 46.63 | |
| DiffTPTBackbone=ResNet-502024.10 | 60.8 | 31.06 | 55.8 | 58.8 | 37.1 | 48.71 | 45.69 | |
| DiffTPTBackbone=ResNet-50, Buffer=false2026.06 | 60.8 | 31.06 | 55.8 | 58.8 | 37.1 | 48.71 | 45.69 | |
| TPTBackbone=ResNet-502024.10 | 60.74 | 26.67 | 54.7 | 59.11 | 35.09 | 47.26 | 43.89 | |
| TPTBackbone=ResNet-50, Buffer=false2026.06 | 60.74 | 26.67 | 54.7 | 59.11 | 35.09 | 47.26 | 43.89 | |
| EnsembleBackbone=ResNet-502024.10 | 59.81 | 23.24 | 52.91 | 60.72 | 35.48 | 46.43 | 43.09 | |
| EnsembleBackbone=ResNet-50, Buffer=false2026.06 | 59.81 | 23.24 | 52.91 | 60.72 | 35.48 | 46.43 | 43.09 | |
| CLIP-ResNet-50Backbone=ResNet-502024.10 | 58.16 | 21.83 | 51.41 | 56.15 | 33.37 | 44.18 | 40.69 | |
| CLIP-ResNet-50Backbone=ResNet-50, Buffer=false2026.06 | 58.16 | 21.83 | 51.41 | 56.15 | 33.37 | 44.18 | 40.69 |