Image Classification on ImageNet ILSVRC 10% labeled examples 2012
77.5Top-1 AccSimCLRv2* (Fine-tune+Distillation)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SimCLRv2* (Fine-tune+Distillation)Self-supervised Pre-training=SimCLRv2*, Evaluation Protocol=Fine-tune+Distillation, #Epochs=>1200, #Parameters (train/test)=829.2M / 29.8M2020.11 | 77.5 | 93.4 | |
| MoCov2 (CoMatch)Self-supervised Pre-training=MoCov2, Evaluation Protocol=CoMatch, #Epochs=~1200, #Parameters (train/test)=30.0M / 25.6M2020.11 | 73.7 | 91.4 | |
| CoMatchSelf-supervised Pre-training=None, #Epochs=~400, #Parameters (train/test)=30.0M / 25.6M2020.11 | 73.6 | 91.6 | |
| DMAE-BPre-training epochs=100, Supervised training / fine-tuning epochs=1002022.08 | 73.4 | — | |
| MAE-BPre-training epochs=1600, Supervised training / fine-tuning epochs=1002022.08 | 72.8 | — | |
| MoCov2 (FixMatch w. DA)Self-supervised Pre-training=MoCov2, Evaluation Protocol=FixMatch w. DA, #Epochs=~1200, #Parameters (train/test)=30.0M / 25.6M2020.11 | 72.2 | 89.5 | |
| FixMatch (RandAug)Self-supervised Pre-training=None, #Epochs=~300, #Parameters (train/test)=25.6M / 25.6M2020.11 | 71.5 | 89.1 | |
| FixMatch w. DASelf-supervised Pre-training=None, #Epochs=~400, #Parameters (train/test)=25.6M / 25.6M2020.11 | 70.8 | 89 | |
| SwAVSelf-supervised Pre-training=SwAV, Evaluation Protocol=Fine-tune, #Epochs=~800, #Parameters (train/test)=30.4M / 25.6M2020.11 | 70.2 | 89.9 | |
| VAT+EntMin.Self-supervised Pre-training=None, #Parameters (train/test)=25.6M / 25.6M2020.11 | 68.8 | 88.5 | |
| UDA (RandAug)Self-supervised Pre-training=None, #Parameters (train/test)=25.6M / 25.6M2020.11 | 68.8 | 88.5 | |
| BYOLSelf-supervised Pre-training=BYOL, Evaluation Protocol=Fine-tune, #Epochs=~1000, #Parameters (train/test)=37.1M / 25.6M2020.11 | 68.8 | 89 | |
| SimCLRv2* (Fine-tune)Self-supervised Pre-training=SimCLRv2*, Evaluation Protocol=Fine-tune, #Epochs=~800, #Parameters (train/test)=34.2M / 29.8M2020.11 | 68.4 | 89.2 | |
| Supervised Feature AlignmentPre-training epochs=1600, Supervised training / fine-tuning epochs=1002022.08 | 67.6 | — | |
| DeiT-B-Hard DistillationPre-training epochs=1600, Supervised training / fine-tuning epochs=1002022.08 | 67.3 | — | |
| DeiT-B-Soft DistillationPre-training epochs=1600, Supervised training / fine-tuning epochs=1002022.08 | 66.4 | — | |
| MoCov2 (Fine-tune)Self-supervised Pre-training=MoCov2, Evaluation Protocol=Fine-tune, #Epochs=~800, #Parameters (train/test)=30.0M / 25.6M2020.11 | 66.1 | 87.9 | |
| SimCLRSelf-supervised Pre-training=SimCLR, Evaluation Protocol=Fine-tune, #Epochs=~1000, #Parameters (train/test)=30.0M / 25.6M2020.11 | 65.6 | 87.8 | |
| MAE-BPre-training epochs=100, Supervised training / fine-tuning epochs=1002022.08 | 65 | — | |
| PIRLSelf-supervised Pre-training=PIRL, #Epochs=~800, #Parameters (train/test)=26.1M / 25.6M2020.11 | 60.4 | 83.8 | |
| Supervised baselineSelf-supervised Pre-training=None, #Epochs=~20, #Parameters (train/test)=25.6M / 25.6M2020.11 | 56.4 | 80.4 | |
| S4L-RotationSelf-supervised Pre-training=None, #Epochs=~200, #Parameters (train/test)=25.6M / 25.6M2020.11 | 53.4 | 83.8 | |
| PCLSelf-supervised Pre-training=PCL, #Epochs=~200, #Parameters (train/test)=25.8M / 25.6M2020.11 | — | 85.6 | |
| Pseudo-labelSelf-supervised Pre-training=None, #Epochs=~100, #Parameters (train/test)=25.6M / 25.6M2020.11 | — | 82.4 |