Image Classification on ImageNet 1K (train val)
86.5Top-1 AccuracyTEC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TECBackbone=ViT-Large, Pre-training Epochs=300, Guidance=MAE, Extra data usage=false2022.10 | 86.5 | — | |
| PeCoBackbone=ViT-L, Target=VQVAE, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 86.5 | — | |
| CAEBackbone=ViT-L, Target=DALLE, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 86.3 | — | |
| A2MIM+Backbone=ViT-L, Target=HOG, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 86.3 | — | |
| Data2VecBackbone=ViT-L, Target=EMA, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 86.2 | — | |
| MC-MAEBackbone=ViT-L, Target=EMA, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 86.1 | — | |
| A2MIMBackbone=ViT-L, Target=RGB, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 86.1 | — | |
| MAEBackbone=ViT-Large, Pre-training Epochs=1600, Guidance=RGB, Extra data usage=false2022.10 | 85.9 | — | |
| MAEBackbone=ViT-L, Target=RGB, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 85.9 | — | |
| BootMAEBackbone=ViT-L, Target=EMA, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 85.9 | — | |
| LocalMIMBackbone=ViT-L, Target=HOG, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 85.8 | — | |
| MaskFeatBackbone=ViT-L, Target=HOG, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 85.7 | — | |
| SimMIMBackbone=ViT-L, Target=RGB, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 85.6 | — | |
| mc-BEiTBackbone=ViT-L, Target=VQGAN, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 85.6 | — | |
| BEiTBackbone=ViT-L, Target=DALLE, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 85.2 | — | |
| iBOTBackbone=ViT-L, Target=EMA, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 85.2 | — | |
| TECBackbone=ViT-Base, Pre-training Epochs=800, Guidance=iBOT, Extra data usage=false2022.10 | 85.1 | — | |
| FD-CLIPBackbone=ViT-Base, Pre-training Epochs=300, Guidance=CLIP+, Extra data usage=true2022.10 | 84.9 | — | |
| Ge2-AEBackbone=ViT-Base, Pre-training Epochs=800, Guidance=RGB+Frequency, Extra data usage=false2022.10 | 84.8 | — | |
| TECBackbone=ViT-Base, Pre-training Epochs=800, Guidance=MAE, Extra data usage=false2022.10 | 84.8 | — | |
| TECBackbone=ViT-Base, Pre-training Epochs=300, Guidance=iBOT, Extra data usage=false2022.10 | 84.8 | — | |
| CMAEBackbone=ViT-Base, Pre-training Epochs=1600, Guidance=RGB, Extra data usage=false2022.10 | 84.7 | — | |
| TECBackbone=ViT-Base, Pre-training Epochs=300, Guidance=MAE, Extra data usage=false2022.10 | 84.7 | — | |
| PeCoBackbone=ViT-Base, Pre-training Epochs=800, Guidance=Perceptual codebook, Extra data usage=false2022.10 | 84.5 | — | |
| SemMAEBackbone=ViT-Base, Pre-training Epochs=800, Guidance=iBOT, Extra data usage=false2022.10 | 84.5 | — | |
| PeCoBackbone=ViT-B, Target=VQVAE, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 84.5 | — | |
| MVPBackbone=ViT-Base, Pre-training Epochs=300, Guidance=CLIP+, Extra data usage=true2022.10 | 84.4 | — | |
| iBOTBackbone=ViT-Base, Guidance=Momentum, Extra data usage=true, Source Dataset=ImageNet-22K2022.10 | 84.4 | — | |
| A2MIM+Backbone=ViT-B, Target=HOG, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 84.4 | — | |
| MugsBackbone=ViT-Base, Pre-training Epochs=1600, Guidance=NA, Extra data usage=false2022.10 | 84.3 | — | |
| MAGE-CBackbone=ViT-L, Target=VQGAN, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 84.3 | — | |
| BootMAEBackbone=ViT-Base, Pre-training Epochs=800, Guidance=RGB+Momentum, Extra data usage=false2022.10 | 84.2 | — | |
| data2vecBackbone=ViT-Base, Pre-training Epochs=800, Guidance=Momentum, Extra data usage=false2022.10 | 84.2 | — | |
| Data2VecBackbone=ViT-B, Target=EMA, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 84.2 | — | |
| BootMAEBackbone=ViT-B, Target=EMA, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 84.2 | — | |
| A2MIMBackbone=ViT-B, Target=RGB, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 84.2 | — | |
| LoMaRBackbone=ViT-Base, Pre-training Epochs=1600, Guidance=RGB, Extra data usage=false2022.10 | 84.1 | — | |
| iBOTBackbone=ViT-Base, Pre-training Epochs=1600, Guidance=Momentum, Extra data usage=false2022.10 | 84.1 | — | |
| MoCoV3Backbone=ViT-L, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 84.1 | — | |
| mc-BEiTBackbone=ViT-B, Target=VQGAN, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 84.1 | — | |
| MaskFeatBackbone=ViT-Base, Pre-training Epochs=1600, Guidance=HOG, Extra data usage=false2022.10 | 84 | — | |
| iBOTBackbone=ViT-B, Target=EMA, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 84 | — | |
| MaskFeatBackbone=ViT-B, Target=HOG, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 84 | — | |
| LocalMIMBackbone=ViT-B, Target=HOG, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 84 | — | |
| CAEBackbone=ViT-Base, Pre-training Epochs=1600, Guidance=DALLE+, Extra data usage=true2022.10 | 83.9 | — | |
| SimMIMBackbone=ViT-Base, Pre-training Epochs=800, Guidance=RGB, Extra data usage=false2022.10 | 83.8 | — | |
| SIMBackbone=ViT-Base, Pre-training Epochs=1600, Guidance=Momentum, Extra data usage=false2022.10 | 83.8 | — | |
| FD-MAEBackbone=ViT-Base, Pre-training Epochs=300, Guidance=MAE, Extra data usage=false2022.10 | 83.8 | — | |
| SimMIMBackbone=ViT-B, Target=RGB, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 83.8 | — | |
| ConMIMBackbone=ViT-Base, Pre-training Epochs=800, Guidance=Momentum, Extra data usage=false2022.10 | 83.7 | — | |
| SplitMaskBackbone=ViT-Base, Pre-training Epochs=300, Guidance=NA, Extra data usage=false2022.10 | 83.6 | — | |
| MAEBackbone=ViT-Base, Pre-training Epochs=1600, Guidance=RGB, Extra data usage=false2022.10 | 83.6 | — | |
| DINOBackbone=ViT-B, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 83.6 | — | |
| SplitMaskBackbone=ViT-B, Target=DALLE, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 83.6 | — | |
| MAEBackbone=ViT-B, Target=RGB, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 83.6 | — | |
| CAEBackbone=ViT-B, Target=DALLE, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 83.6 | — | |
| MC-MAEBackbone=ViT-B, Target=EMA, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 83.6 | — | |
| MAEarch.=ViT-B/16, batch=4096, epochs=1600, evaluation_mode=Fine-tuning2024.08 | 83.6 | — | |
| CIMBackbone=ViT-B, Target=BEiT, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 83.3 | — | |
| MoCov3Backbone=ViT-Base, Pre-training Epochs=300, Guidance=NA, Extra data usage=false2022.10 | 83.2 | — | |
| MixMIMBackbone=ViT-Base, Pre-training Epochs=300, Guidance=RGB, Extra data usage=false2022.10 | 83.2 | — | |
| BEITBackbone=ViT-Base, Pre-training Epochs=800, Guidance=DALLE, Extra data usage=false2022.10 | 83.2 | — | |
| MoCoV3Backbone=ViT-B, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 83.2 | — | |
| BEiTBackbone=ViT-B, Target=DALLE, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 83.2 | — | |
| MoCoV3arch.=ViT-B/16, batch=4096, epochs=300, evaluation_mode=Fine-tuning2024.08 | 83.2 | — | |
| BAMarch.=ViT-B/16, batch=4096, epochs=300, evaluation_mode=Fine-tuning2024.08 | 83.2 | — | |
| MFMBackbone=ViT-Base, Pre-training Epochs=300, Guidance=Frequency, Extra data usage=false2022.10 | 83.1 | — | |
| MAGE-CBackbone=ViT-B, Target=VQGAN, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 82.9 | — | |
| DINOBackbone=ViT-Base, Pre-training Epochs=300, Guidance=NA, Extra data usage=false2022.10 | 82.8 | — | |
| BYOLBackbone=ViT-B, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 82.8 | — | |
| DINOarch.=ViT-B/16, batch=1024, epochs=300, evaluation_mode=Fine-tuning2024.08 | 82.8 | — | |
| Rand init.Backbone=ViT-L, Target=Label, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 82.6 | — | |
| SimCLRBackbone=ViT-B, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 82.3 | — | |
| iBOTBackbone=ViT-S, Target=EMA, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 82.3 | — | |
| A2MIM+Backbone=ViT-S, Target=HOG, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 82.3 | — | |
| A2MIMBackbone=ViT-S, Target=RGB, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 82.1 | — | |
| MC-MAEBackbone=ViT-S, Target=EMA, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 82 | — | |
| Rand init.Backbone=ViT-B, Target=Label, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 81.8 | — | |
| CAEBackbone=ViT-S, Target=DALLE, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 81.8 | — | |
| BEITarch.=ViT-B/16, batch=4096, epochs=300, evaluation_mode=Fine-tuning2024.08 | 81.8 | — | |
| SimMIMBackbone=ViT-S, Target=RGB, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 81.7 | — | |
| MAEBackbone=ViT-S, Target=RGB, PT Epochs=1600, Evaluation Protocol=Fine-tuning2022.05 | 81.6 | — | |
| CIMBackbone=ViT-S, Target=BEiT, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 81.6 | — | |
| DINOBackbone=ViT-S, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 81.5 | — | |
| SplitMaskBackbone=ViT-S, Target=DALLE, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 81.5 | — | |
| DINOarch.=ViT-S/16, batch=1024, epochs=800, evaluation_mode=Fine-tuning2024.08 | 81.5 | — | |
| MoCoV3Backbone=ViT-S, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 81.4 | — | |
| MoCoV3arch.=ViT-S/16, batch=4096, epochs=300, evaluation_mode=Fine-tuning2024.08 | 81.4 | — | |
| BEiTBackbone=ViT-S, Target=DALLE, PT Epochs=800, Evaluation Protocol=Fine-tuning2022.05 | 81.3 | — | |
| BAMarch.=ViT-S/16, batch=4096, epochs=300, evaluation_mode=Fine-tuning2024.08 | 81.3 | — | |
| BYOLBackbone=ViT-S, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 80.9 | — | |
| SimCLRBackbone=ViT-S, Target=CL, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 80.2 | — | |
| Rand init.Backbone=ViT-S, Target=Label, PT Epochs=300, Evaluation Protocol=Fine-tuning2022.05 | 79.9 | — | |
| DnCTraining budget=3000 epochs, Backbone=ResNet-50, Evaluation protocol=Linear evaluation2021.05 | 75.8 | 1.3 | |
| MoCLRTraining budget=3000 epochs, Backbone=ResNet-50, Evaluation protocol=Linear evaluation2021.05 | 74.5 | 0.2 | |
| DnCTraining budget=1000 epochs, Backbone=ResNet-50, Evaluation protocol=Linear evaluation2021.05 | 74.5 | — | |
| BYOLTraining budget=1000 epochs, Backbone=ResNet-50, Evaluation protocol=Linear evaluation2021.05 | 74.3 | — | |
| MoCLRTraining budget=1000 epochs, Backbone=ResNet-50, Evaluation protocol=Linear evaluation2021.05 | 74.3 | — | |
| BYOLTraining budget=3000 epochs, Backbone=ResNet-50, Evaluation protocol=Linear evaluation2021.05 | 73.9 | -0.4 | |
| BYOLApproach=self-distillation (EMA), Backbone=ResNet-50, Pretraining Epochs=100, Evaluation Protocol=Linear-probe, Augmentation Protocol=Original recipe2026.05 | 69.3 | — |