Image Classification on ImageNet 1% labeled
93.4Top-5 AccuracySimCLRv2 self-distilled
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SimCLRv2 self-distilledArchitecture=ResNet-152 (3x+SK)2020.06 | 93.4 | 76.6 | |
| SimCLRv2 distilledArchitecture=ResNet-50 (2x+SK)2020.06 | 93 | 75.9 | |
| SimCLRv2 distilledArchitecture=ResNet-502020.06 | 91.5 | 73.9 | |
| MoProPretrain dataset=WebVision-V1.0, #Pretrain epochs=902020.09 | 90.5 | 71.2 | |
| BYOLArchitecture=ResNet-200 (2x)2020.06 | 89.5 | 71.2 | |
| MoProPretrain dataset=WebVision-V0.5, #Pretrain epochs=902020.09 | 89.1 | 69.3 | |
| CEPretrain dataset=WebVision-V1.0, #Pretrain epochs=902020.09 | 88.3 | 67.6 | |
| MoProPretrain dataset=WebVision-V2.0, #Pretrain epochs=902020.09 | 88.2 | 65.3 | |
| ShrinkMatchPre-training=None, Method=ShrinkMatch, Epochs=400, Params (train / test)=31.8M / 25.6M2023.08 | 87.4 | 67.5 | |
| CoMatchSelf-supervised Pre-training=MoCo V2, Epochs=~1200, Parameters (train/test)=30.0M / 25.6M2022.03 | 87.1 | 67.1 | |
| SimMatchSelf-supervised Pre-training=None, Epochs=~400, Parameters (train/test)=30.0M / 25.6M2022.03 | 87.1 | 67.2 | |
| CoMatchPre-training=MoCo v2 [9], Method=CoMatch [20], Epochs=1200, Params (train / test)=30.0M / 25.6M2023.08 | 87.1 | 67.1 | |
| CEPretrain dataset=WebVision-V0.5, #Pretrain epochs=902020.09 | 87 | 65.9 | |
| SimMatch+Pre-training=None, Method=SimMatch+ [45], Epochs=400, Params (train / test)=30.0M / 25.6M2023.08 | 86.9 | 67 | |
| CEPretrain dataset=WebVision-V2.0, #Pretrain epochs=902020.09 | 86.9 | 62.1 | |
| SimCLRArchitecture=ResNet-50 (4x), Evaluation Protocol=Fine-tuning, Data Augmentation=Broader composition2020.02 | 86.6 | 64.1 | |
| CoMatchSelf-supervised Pre-training=None, Epochs=~400, Parameters (train/test)=30.0M / 25.6M2022.03 | 86.4 | 66 | |
| CoMatchPre-training=None, Method=CoMatch [20], Epochs=400, Params (train / test)=30.0M / 25.6M2023.08 | 86.4 | 66 | |
| Fine-tuneSelf-supervised Pre-training=WCL, Epochs=~800, Parameters (train/test)=34.2M / 25.6M2022.03 | 86.3 | 65 | |
| WCLPre-training=WCL [44], Method=Fine-tune, Epochs=800, Params (train / test)=34.2M / 25.6M2023.08 | 86.3 | 65 | |
| SimCLRArchitecture=ResNet-50 (4x)2020.02 | 85.8 | — | |
| SimCLRArchitecture=ResNet-50 (4x)2020.06 | 85.8 | 63 | |
| TWIST2021.12 | 84.2 | 61.2 | |
| UniGrad2021.12 | 83.8 | 60.8 | |
| SimCLRArchitecture=ResNet-50 (2x), Evaluation Protocol=Fine-tuning, Data Augmentation=Broader composition2020.02 | 83.7 | 59.4 | |
| FixMatch-EMANArch=res502021.01 | 83.4 | 63 | |
| FixMatch-EMANSelf-supervised Pre-training=MoCo-EMAN, Epochs=~1100, Parameters (train/test)=30.0M / 25.6M2022.03 | 83.4 | 63 | |
| C-BYOLBackbone=ResNet-50, Evaluation protocol=fine-tuning2022.08 | 83.4 | 60.6 | |
| FixMatch-EMANPre-training=MoCo-EMAN [5], Method=FixMatch-EMAN [5], Epochs=1100, Params (train / test)=30.0M / 25.6M2023.08 | 83.4 | 63 | |
| SimCLRArchitecture=ResNet-50 (2x)2020.02 | 83 | — | |
| SimCLRArchitecture=ResNet-50 (2x)2020.06 | 83 | 58.5 | |
| Fine-tuneSelf-supervised Pre-training=SimCLR V2, Epochs=~800, Parameters (train/test)=34.2M / 25.6M2022.03 | 82.5 | 57.9 | |
| SimCLR v2Pre-training=SimCLR v2 [8], Method=Fine-tune, Epochs=800, Params (train / test)=34.2M / 25.6M2023.08 | 82.5 | 57.9 | |
| I-VNE+Backbone=ResNet-50, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuned2023.04 | 81 | 55.8 | |
| NNCLRBackbone=ResNet-50, Evaluation Protocol=Fine-tuning2021.04 | 80.7 | 56.4 | |
| NNCLRBackbone=ResNet-50, Evaluation protocol=fine-tuning2022.08 | 80.7 | 56.4 | |
| RényiCLBackbone=ResNet-50, Evaluation protocol=fine-tuning2022.08 | 80.6 | 56.4 | |
| VICRegBackbone=ResNet-50, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuned2023.04 | 79.4 | 54.8 | |
| Barlow Twins2021.12 | 79.2 | 55 | |
| Barlow TwinsBackbone=ResNet-50, Evaluation protocol=fine-tuning2022.08 | 79.2 | 55 | |
| Barlow TwinsBackbone=ResNet-50, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuned2023.04 | 79.2 | 55 | |
| SwAVBackbone=ResNet-50, Evaluation protocol=Fine-tuning, Method category=Methods using self-supervision only2020.06 | 78.5 | 53.9 | |
| SwAVBackbone=ResNet-50, Evaluation Protocol=Fine-tuning, multi-crop=true2021.04 | 78.5 | 53.9 | |
| Fine-tuneSelf-supervised Pre-training=SwAV, Epochs=~800, Parameters (train/test)=30.4M / 25.6M2022.03 | 78.5 | 53.9 | |
| SwAVBackbone=ResNet-50, Evaluation protocol=fine-tuning2022.08 | 78.5 | 53.9 | |
| SwAVBackbone=ResNet-50, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuned2023.04 | 78.5 | 53.9 | |
| SwAVPre-training=SwAV [6], Method=Fine-tune, Epochs=800, Params (train / test)=30.4M / 25.6M2023.08 | 78.5 | 53.9 | |
| SwAVPretrain dataset=ImageNet, #Pretrain epochs=8002020.09 | 78.5 | 53.9 | |
| BYOLArchitecture=ResNet-502020.06 | 78.4 | 53.2 | |
| BYOLBackbone=ResNet-50, Evaluation Protocol=Fine-tuning2021.04 | 78.4 | 53.2 | |
| Fine-tuneSelf-supervised Pre-training=BYOL, Epochs=~1000, Parameters (train/test)=37.1M / 25.6M2022.03 | 78.4 | 53.2 | |
| BYOL2021.12 | 78.4 | 53.2 | |
| BYOLBackbone=ResNet-50, Evaluation protocol=fine-tuning2022.08 | 78.4 | 53.2 | |
| BYOLBackbone=ResNet-50, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuned2023.04 | 78.4 | 53.2 | |
| BYOLPretrain dataset=ImageNet, #Pretrain epochs=10002020.09 | 78.4 | 53.2 | |
| CPC v2Method Category=Representation learning, Evaluation Protocol=Fine-tuned2019.05 | 78.3 | — | |
| DINO2021.12 | 78.2 | 52.2 | |
| CPC v2Backbone=ResNet-161, Labels=true2020.05 | 77.9 | 52.7 | |
| CPC v2Architecture=ResNet-161(*)2020.02 | 77.9 | — | |
| CPC v2Architecture=ResNet-161(*)2020.06 | 77.9 | 52.7 | |
| Fine-tuneSelf-supervised Pre-training=MoCo V2, Epochs=~800, Parameters (train/test)=30.0M / 25.6M2022.03 | 77.2 | 49.8 | |
| MoCo v2 (Fine-tune)Pre-training=MoCo v2 [9], Method=Fine-tune, Epochs=800, Params (train / test)=30.0M / 25.6M2023.08 | 77.2 | 49.8 | |
| CPC v2Method Category=Representation learning, Evaluation Protocol=Fixed2019.05 | 77.1 | — | |
| SimCLRArchitecture=ResNet-50, Evaluation Protocol=Fine-tuning, Data Augmentation=Broader composition2020.02 | 76.6 | 49.4 | |
| PCLBackbone=ResNet-50, Evaluation protocol=Fine-tuning, Method category=Methods using self-supervision only2020.06 | 75.6 | — | |
| PCLBackbone=ResNet-50, Evaluation Protocol=Fine-tuning2021.04 | 75.6 | — | |
| SimCLRBackbone=ResNet-50, Labels=true2020.05 | 75.5 | 48.3 | |
| SimCLRArchitecture=ResNet-502020.02 | 75.5 | — | |
| SimCLRArchitecture=ResNet-502020.06 | 75.5 | 48.3 | |
| SimCLRBackbone=ResNet-50, Evaluation protocol=Fine-tuning, Method category=Methods using self-supervision only2020.06 | 75.5 | 48.3 | |
| SimCLRBackbone=ResNet-50, Evaluation Protocol=Fine-tuning2021.04 | 75.5 | 48.3 | |
| Fine-tuneSelf-supervised Pre-training=SimCLR, Epochs=~1000, Parameters (train/test)=30.0M / 25.6M2022.03 | 75.5 | 48.3 | |
| SimCLR2021.12 | 75.5 | 48.3 | |
| SimCLRBackbone=ResNet-50, Evaluation protocol=fine-tuning2022.08 | 75.5 | 48.3 | |
| SimCLRBackbone=ResNet-50, Pre-training=ImageNet-1K, Evaluation Protocol=Fine-tuned2023.04 | 75.5 | 48.3 | |
| SimCLRPretrain dataset=ImageNet, #Pretrain epochs=10002020.09 | 75.5 | 48.3 | |
| PCLSelf-supervised Pre-training=None, Epochs=~200, Parameters (train/test)=25.8M / 25.6M2022.03 | 75.3 | — | |
| PCLPretrain dataset=ImageNet, #Pretrain epochs=2002020.09 | 75.3 | 48.8 | |
| AMDIMMethod Category=Representation learning, Evaluation Protocol=Fixed2019.05 | 67.4 | — | |
| SCANBackbone=ResNet-50, Labels=false2020.05 | 60 | 39.9 | |
| RotationMethod Category=Representation learning2019.05 | 57.5 | — | |
| PIRLBackbone=ResNet-50, Labels=true2020.05 | 57.2 | — | |
| PIRLMethod Category=Representation learning2019.05 | 57.2 | — | |
| PIRLArchitecture=ResNet-502020.02 | 57.2 | — | |
| PIRLArchitecture=ResNet-502020.06 | 57.2 | — | |
| PIRLBackbone=ResNet-50, Evaluation protocol=Fine-tuning, Method category=Methods using self-supervision only2020.06 | 57.2 | 30.7 | |
| PIRLBackbone=ResNet-50, Evaluation Protocol=Fine-tuning2021.04 | 57.2 | — | |
| BigBiGANBackbone=ResNet-50(4x), Labels=true2020.05 | 55.2 | — | |
| BigBiGANMethod Category=Representation learning, Evaluation Protocol=Fixed2019.05 | 55.2 | — | |
| BigBiGANArchitecture=RevNet-50 (4x)2020.02 | 55.2 | — | |
| BigBiGANArchitecture=RevNet-50 (4x)2020.06 | 55.2 | — | |
| S4L RotationArch=res502021.01 | 53.4 | — | |
| Pseudo-LabelBackbone=ResNet-50, Labels=true2020.05 | 51.6 | — | |
| PseudolabelingMethod Category=Label-propagation2019.05 | 51.6 | — | |
| Pseudo-labelArchitecture=ResNet-502020.02 | 51.6 | — | |
| Pseudo-labelArchitecture=ResNet-502020.06 | 51.6 | — | |
| Pseudo-labelArch=res502021.01 | 51.6 | — | |
| Pseudo-labelSelf-supervised Pre-training=None, Epochs=~100, Parameters (train/test)=25.6M / 25.6M2022.03 | 51.6 | — | |
| Supervised BaselineBackbone=ResNet-50, Labels=true2020.05 | 48.4 | 25.4 | |
| Supervised baselineArchitecture=ResNet-502020.02 | 48.4 | — |