Image Classification on CIFAR-100
95.96AccuracyViT-22B
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ViT-22BEvaluation Protocol=Finetuning, Resolution=224, Seeds=32023.02 | 95.96 | — | — | — | — | — | — | — | — | — | — | — | |
| DINOv2Architecture=ViT-g/14, Pre-training Data=LVD, Resolution=518x518, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 95.2 | — | — | — | — | — | — | — | — | — | — | — | |
| ViT-22BEvaluation Protocol=H2T (Head2Toe), Resolution=224, Seeds=32023.02 | 94.11 | — | — | — | — | — | — | — | — | — | — | — | |
| ViT-22BEvaluation Protocol=Linear, Resolution=224, Seeds=32023.02 | 93.39 | — | — | — | — | — | — | — | — | — | — | — | |
| CaiT-M-36 224Resolution=224, Crop-ratio=0.8752021.03 | 93.3 | — | — | — | — | — | — | — | — | — | — | — | |
| iBOTArchitecture=ViT-L/16, Pre-training Data=IN-21k, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 93.3 | — | — | — | — | — | — | — | — | — | — | — | |
| CaiT-M-36 ↑ 224Resolution=224, Crop-ratio=0.875, Training=Longer schedules2021.03 | 93.1 | — | — | — | — | — | — | — | — | — | — | — | |
| StableRepPre-training dataset=RedCaps, Pre-training data source=Syn, Evaluation protocol=Few-shot2023.06 | 92.9 | — | — | — | — | — | — | — | — | — | — | — | |
| MugsBackbone=ViT-B/162022.03 | 92.8 | — | — | — | — | — | — | — | — | — | — | — | |
| iBOT-vMFBackbone=ViT-Base/16, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuning2024.05 | 92.6 | — | — | — | — | — | — | — | — | — | — | — | |
| CaiT-S-36 224Resolution=224, Crop-ratio=0.8752021.03 | 92.2 | — | — | — | — | — | — | — | — | — | — | — | |
| CaiT-S-36 ↑ 224Resolution=224, Crop-ratio=0.875, Training=Longer schedules2021.03 | 92.2 | — | — | — | — | — | — | — | — | — | — | — | |
| iBOTBackbone=ViT-B/162022.03 | 92.2 | — | — | — | — | — | — | — | — | — | — | — | |
| iBOTBackbone=ViT-Base/16, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuning2024.05 | 92.2 | — | — | — | — | — | — | — | — | — | — | — | |
| DINO-vMFBackbone=ViT-Base/16, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuning2024.05 | 91.9 | — | — | — | — | — | — | — | — | — | — | — | |
| MugsBackbone=ViT-S/162022.03 | 91.8 | — | — | — | — | — | — | — | — | — | — | — | |
| AIM-7B†Architecture=ViT-7B/14, Pre-training Data=DFN-2B+, Feature Extraction Layer=20th, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 91.8 | — | — | — | — | — | — | — | — | — | — | — | |
| StableRepPre-training Data Source=Synthetic, Evaluation Protocol=5-way, 5-shot2023.06 | 91.8 | — | — | — | — | — | — | — | — | — | — | — | |
| StableRepPre-training dataset=CC12M, Pre-training data source=Syn, Evaluation protocol=Few-shot2023.06 | 91.8 | — | — | — | — | — | — | — | — | — | — | — | |
| EfficientNet-B7Number of Parameters=64M, Pre-training=ImageNet, Transfer Learning=true2019.05 | 91.7 | — | — | — | — | — | — | — | — | — | — | — | |
| EfficientNet-B7Resolution=224, Crop-ratio=0.8752021.03 | 91.7 | — | — | — | — | — | — | — | — | — | — | — | |
| DINOBackbone=ViT-B/162022.03 | 91.7 | — | — | — | — | — | — | — | — | — | — | — | |
| EfficientNet-B7# Params=64.0M, # FLOPs=37.2B, Resolution=600, Pre-training=ImageNet, Fine-tuning=true2021.07 | 91.7 | — | — | — | — | — | — | — | — | — | — | — | |
| CMT-S# Params=25.1M, # FLOPs=4.04B, Pre-training=ImageNet, Fine-tuning=true2021.07 | 91.7 | — | — | — | — | — | — | — | — | — | — | — | |
| DINOBackbone=ViT-Base/16, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuning2024.05 | 91.7 | — | — | — | — | — | — | — | — | — | — | — | |
| GPipeNumber of Parameters=556M, Pre-training=ImageNet, Transfer Learning=true2019.05 | 91.3 | — | — | — | — | — | — | — | — | — | — | — | |
| DeiT-B 224Resolution=224, Crop-ratio=0.8752021.03 | 90.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Sup.Backbone=ViT-B/162022.03 | 90.8 | — | — | — | — | — | — | — | — | — | — | — | |
| DeiT-B# Params=85.8M, # FLOPs=17.6B, Pre-training=ImageNet, Fine-tuning=true2021.07 | 90.8 | — | — | — | — | — | — | — | — | — | — | — | |
| CeiT-S# Params=24.2M, # FLOPs=12.9B, Resolution=384, Pre-training=ImageNet, Fine-tuning=true2021.07 | 90.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Random (DeIT)Backbone=ViT-Base/16, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuning2024.05 | 90.8 | — | — | — | — | — | — | — | — | — | — | — | |
| iBOTBackbone=ViT-S/162022.03 | 90.7 | — | — | — | — | — | — | — | — | — | — | — | |
| DINOBackbone=ViT-S/162022.03 | 90.5 | — | — | — | — | — | — | — | — | — | — | — | |
| MoCo-v3Backbone=ViT-B/162022.03 | 90.5 | — | — | — | — | — | — | — | — | — | — | — | |
| BEITBackbone=ViT-B/162022.03 | 90.1 | — | — | — | — | — | — | — | — | — | — | — | |
| TNT-S# Params=23.8M, # FLOPs=17.3B, Resolution=384, Pre-training=ImageNet, Fine-tuning=true2021.07 | 90.1 | — | — | — | — | — | — | — | — | — | — | — | |
| BeITBackbone=ViT-Base/16, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuning2024.05 | 90.1 | — | — | — | — | — | — | — | — | — | — | — | |
| AIM-7BArchitecture=ViT-7B/14, Pre-training Data=DFN-2B+, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 90 | — | — | — | — | — | — | — | — | — | — | — | |
| AIM-3BArchitecture=ViT-3B/14, Pre-training Data=DFN-2B+, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 89.9 | — | — | — | — | — | — | — | — | — | — | — | |
| IndividualBackbone=ViT-B/16, Pre-trained=ImageNet-21k, Number of Merged Models=12024.05 | 89.85 | — | — | — | — | — | — | — | — | — | — | — | |
| Sup.Backbone=ViT-S/162022.03 | 89.5 | — | — | — | — | — | — | — | — | — | — | — | |
| RSModel=ResNet-152 2x, Epochs=4002021.03 | 89.3 | — | — | — | — | — | — | — | — | — | — | — | |
| EMR-MERGINGBackbone=ViT-B/16, Pre-trained=ImageNet-21k, Number of Merged Models=302024.05 | 89.09 | — | — | — | — | — | — | — | — | — | — | — | |
| IndividualBackbone=DeiT-S2022.03 | 89.03 | — | — | — | — | — | — | — | — | — | — | — | |
| SimCLRProtocol=Fine-tuned, Backbone=ResNet-50 (4x), Pre-training=ImageNet2020.02 | 89 | — | — | — | — | — | — | — | — | — | — | — | |
| SimCLRModel=ResNet-152 2x, Epochs=8002021.03 | 89 | — | — | — | — | — | — | — | — | — | — | — | |
| SupervisedProtocol=Fine-tuned, Backbone=ResNet-50 (4x), Pre-training=ImageNet2020.02 | 88.7 | — | — | — | — | — | — | — | — | — | — | — | |
| AIM-1BArchitecture=ViT-1B/14, Pre-training Data=DFN-2B+, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 88.3 | — | — | — | — | — | — | — | — | — | — | — | |
| CoKeEvaluation Protocol=fine-tuning, multi-crop training trick=false2021.05 | 88.2 | — | — | — | — | — | — | — | — | — | — | — | |
| ViT-Slimps#Params (M)=15.6, FLOPs (B)=3.32022.01 | 88.16 | — | — | — | — | — | — | — | — | — | — | — | |
| EfficientNet-B0Number of Parameters=4M, Pre-training=ImageNet, Transfer Learning=true2019.05 | 88.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RSModel=ResNet-152, Epochs=4002021.03 | 88.1 | — | — | — | — | — | — | — | — | — | — | — | |
| BYOLEvaluation Protocol=fine-tuning, multi-crop training trick=false2021.05 | 87.9 | — | — | — | — | — | — | — | — | — | — | — | |
| DEIT-S#Params (M)=22.0, FLOPs (B)=4.62022.01 | 87.8 | — | — | — | — | — | — | — | — | — | — | — | |
| DINOEvaluation Protocol=fine-tuning, multi-crop training trick=true2021.05 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | |
| ResNet-152# Params=58.1M, # FLOPs=11.3B, Pre-training=ImageNet, Fine-tuning=true2021.07 | 87.6 | — | — | — | — | — | — | — | — | — | — | — | |
| NASNet-ANumber of Parameters=85M, Pre-training=ImageNet, Transfer Learning=true2019.05 | 87.5 | — | — | — | — | — | — | — | — | — | — | — | |
| SwAVEvaluation Protocol=fine-tuning, multi-crop training trick=true2021.05 | 87.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Inception-v4# Params=41.1M, # FLOPs=16.1B, Pre-training=ImageNet, Fine-tuning=true2021.07 | 87.5 | — | — | — | — | — | — | — | — | — | — | — | |
| BEITBackbone=ViT-S/162022.03 | 87.4 | — | — | — | — | — | — | — | — | — | — | — | |
| Barlow TwinsEvaluation Protocol=fine-tuning, multi-crop training trick=false2021.05 | 87.4 | — | — | — | — | — | — | — | — | — | — | — | |
| DINOArchitecture=ViT-B/8, Pre-training Data=IN-1k, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 87.3 | — | — | — | — | — | — | — | — | — | — | — | |
| MAEArchitecture=ViT-2B/14, Pre-training Data=IG-3B, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 87.3 | — | — | — | — | — | — | — | — | — | — | — | |
| SimCLRModel=ResNet-152, Epochs=8002021.03 | 87.1 | — | — | — | — | — | — | — | — | — | — | — | |
| ViT-B/16Resolution=224, Crop-ratio=0.8752021.03 | 87.1 | — | — | — | — | — | — | — | — | — | — | — | |
| ViT-B/16# Params=85.8M, # FLOPs=17.6B, Resolution=384, Pre-training=ImageNet, Fine-tuning=true2021.07 | 87.1 | — | — | — | — | — | — | — | — | — | — | — | |
| AIM-0.6BArchitecture=ViT-H/14, Pre-training Data=DFN-2B+, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 86.8 | — | — | — | — | — | — | — | — | — | — | — | |
| VisualAtom-21kModel=ViT-B, Type=FDSL2023.03 | 86.7 | — | — | — | — | — | — | — | — | — | — | — | |
| DCRZero-shot=true2026.03 | 86.7 | — | — | — | — | — | — | — | — | — | — | — | |
| SupervisedModel=ResNet-152 2x, Epochs=902021.03 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | |
| SupervisedEvaluation Protocol=fine-tuning, multi-crop training trick=false2021.05 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | |
| Original CLIPZero-shot=true2026.03 | 86.6 | — | — | — | — | — | — | — | — | — | — | — | |
| TWISTEvaluation Protocol=Fine-tune, Backbone=ResNet-50, Pre-trained=ImageNet2021.10 | 86.5 | — | — | — | — | — | — | — | — | — | — | — | |
| SupervisedEvaluation protocol=Fine-tuned, Pre-training=ImageNet, Backbone=ResNet-502020.02 | 86.4 | — | — | — | — | — | — | — | — | — | — | — | |
| ViT-L/16Resolution=224, Crop-ratio=0.8752021.03 | 86.4 | — | — | — | — | — | — | — | — | — | — | — | |
| SUPEvaluation Protocol=Fine-tune, Backbone=ResNet-50, Pre-trained=ImageNet2021.10 | 86.4 | — | — | — | — | — | — | — | — | — | — | — | |
| EfficientViT-M5Throughput (imgs/s)=106212023.05 | 86.4 | — | — | — | — | — | — | — | — | — | — | — | |
| BYOLEvaluation Protocol=Fine-tune, Backbone=ResNet-50, Pre-trained=ImageNet2021.10 | 86.1 | — | — | — | — | — | — | — | — | — | — | — | |
| MoCo-v2Evaluation Protocol=fine-tuning, multi-crop training trick=false2021.05 | 86.1 | — | — | — | — | — | — | — | — | — | — | — | |
| MEATBackbone=DeiT-S2022.03 | 85.93 | — | — | — | — | — | — | — | — | — | — | — | |
| SimCLREvaluation protocol=Fine-tuned, Pre-training=ImageNet, Backbone=ResNet-502020.02 | 85.9 | — | — | — | — | — | — | — | — | — | — | — | |
| MAEArchitecture=ViT-H/14, Pre-training Data=IN-1k, Evaluation Protocol=Attentive Probing, Backbone State=Frozen2024.01 | 85.8 | — | — | — | — | — | — | — | — | — | — | — | |
| ViT-S/16Throughput (imgs/s)=21352023.05 | 85.7 | — | — | — | — | — | — | — | — | — | — | — | |
| SupervisedModel=ResNet-152, Epochs=902021.03 | 85.5 | — | — | — | — | — | — | — | — | — | — | — | |
| ImageNet-1kModel=ViT-T, Type=SL2023.03 | 85.5 | — | — | — | — | — | — | — | — | — | — | — | |
| MobileNetV3Throughput (imgs/s)=75602023.05 | 85.5 | — | — | — | — | — | — | — | — | — | — | — | |
| SimCLREvaluation Protocol=Fine-tune, Backbone=ResNet-50, Pre-trained=ImageNet2021.10 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | |
| CLIPPre-training dataset=RedCaps, Pre-training data source=Real, Evaluation protocol=Few-shot2023.06 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | |
| IndividualBackbone=T2T-ViT-122022.03 | 85.03 | — | — | — | — | — | — | — | — | — | — | — | |
| VisualAtom-1kModel=ViT-T, Type=FDSL2023.03 | 84.9 | — | — | — | — | — | — | — | — | — | — | — | |
| SimCLRv2Model=ResNet-152 2x, Epochs=8002021.03 | 84.8 | — | — | — | — | — | — | — | — | — | — | — | |
| SimCLRv2Model=ResNet-152, Epochs=8002021.03 | 84.7 | — | — | — | — | — | — | — | — | — | — | — | |
| NAONetParams(M)=10.6, Search Method=NAO, Search Cost GPU days=2002020.06 | 84.33 | — | — | — | — | — | — | — | — | — | — | — | |
| AmoebaNet-B + cutoutParams(M)=34.9, Search Method=Evolution, Search Cost GPU days=31502020.06 | 84.2 | — | — | — | — | — | — | — | — | — | — | — | |
| AdaMergingBackbone=ViT-B/16, Pre-trained=ImageNet-21k, Number of Merged Models=302024.05 | 84.19 | — | — | — | — | — | — | — | — | — | — | — | |
| Ours + cutoutParams(M)=3.9, Search Method=Gradient-based, Search Cost GPU days=12020.06 | 84.1 | — | — | — | — | — | — | — | — | — | — | — | |
| Adaptor-BBackbone=DeiT-S2022.03 | 84 | — | — | — | — | — | — | — | — | — | — | — | |
| PASSModel=ViT-T, Type=SSL (DINO)2023.03 | 84 | — | — | — | — | — | — | — | — | — | — | — | |
| FMixModel=Dense2020.02 | 83.95 | — | — | — | — | — | — | — | — | — | — | — | |
| NASNet-A-MThroughput (imgs/s)=26232023.05 | 83.9 | — | — | — | — | — | — | — | — | — | — | — |