Image Classification on ImageNet-1K 1.0 (val) (Linear Probing and k-NN Evaluation)
79.3k-NN AccuracyEsViT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| EsViTBackbone=Swin-B, Window Size (W)=14, Loss=LL + LR, #Parameters=87M, Throughput=2542021.06 | 79.3 | 81.3 | |
| EsViTBackbone=Swin-S, Window Size (W)=14, Loss=LL + LR, #Parameters=49M, Throughput=3832021.06 | 79.1 | 80.8 | |
| EsViTBackbone=Swin-B, Window Size (W)=7, Loss=LL + LR, #Parameters=87M, Throughput=2972021.06 | 78.9 | 80.4 | |
| DINOBackbone=DeiT-S/8, #Parameters=21M, Throughput=1802021.06 | 78.3 | 79.7 | |
| EsViTBackbone=Swin-B, Window Size (W)=14, Loss=LR, #Parameters=87M, Throughput=2542021.06 | 78.3 | 80.5 | |
| EsViTBackbone=Swin-S, Window Size (W)=7, Loss=LL + LR, #Parameters=49M, Throughput=4672021.06 | 77.7 | 79.5 | |
| EsViTBackbone=Swin-B, Window Size (W)=7, Loss=LR, #Parameters=87M, Throughput=2972021.06 | 77.7 | 79.6 | |
| DINOBackbone=ViT-B/8, #Parameters=85M, Throughput=632021.06 | 77.4 | 80.1 | |
| EsViTBackbone=Swin-S, Window Size (W)=14, Loss=LR, #Parameters=49M, Throughput=3832021.06 | 77.3 | 79.4 | |
| EsViTBackbone=Swin-T, Window Size (W)=14, Loss=LL + LR, #Parameters=28M, Throughput=6602021.06 | 77 | 78.7 | |
| EsViTBackbone=Swin-S, Window Size (W)=7, Loss=LR, #Parameters=49M, Throughput=4672021.06 | 76.8 | 79.2 | |
| DINOBackbone=ViT-B/16, #Parameters=85M, Throughput=3122021.06 | 76.1 | 78.2 | |
| EsViTBackbone=Swin-T, Window Size (W)=7, Loss=LL + LR, #Parameters=28M, Throughput=8082021.06 | 75.7 | 78.1 | |
| EsViTBackbone=Swin-T, Window Size (W)=14, Loss=LR, #Parameters=28M, Throughput=6602021.06 | 75.5 | 77.9 | |
| DINOBackbone=DeiT-S/16, #Parameters=21M, Throughput=10072021.06 | 74.5 | 77 | |
| EsViTBackbone=Swin-T, Window Size (W)=7, Loss=LR, #Parameters=28M, Throughput=8082021.06 | 74.2 | 77 | |
| BYOLBackbone=RN200w2, #Parameters=250M, Throughput=1232021.06 | 73.9 | 79.6 | |
| SimCLR-v2Backbone=RN152w3+SK, #Parameters=794M, Throughput=462021.06 | 73.1 | 79.8 | |
| SVDRep. Size=1024, Backbone=ResNet502022.05 | 71.22 | — | |
| SVDRep. Size=2048, Backbone=ResNet502022.05 | 71.21 | — | |
| MRL-ERep. Size=2048, Backbone=ResNet502022.05 | 71.21 | — | |
| Rand. FSRep. Size=2048, Backbone=ResNet502022.05 | 71.19 | — | |
| FFRep. Size=2048, Backbone=ResNet502022.05 | 71.19 | — | |
| MRL-ERep. Size=1024, Backbone=ResNet502022.05 | 71.07 | — | |
| SVDRep. Size=512, Backbone=ResNet502022.05 | 71.06 | — | |
| MRLRep. Size=2048, Backbone=ResNet502022.05 | 70.97 | — | |
| MRLRep. Size=1024, Backbone=ResNet502022.05 | 70.89 | — | |
| MRLRep. Size=512, Backbone=ResNet502022.05 | 70.82 | — | |
| MRL-ERep. Size=512, Backbone=ResNet502022.05 | 70.74 | — | |
| SVDRep. Size=256, Backbone=ResNet502022.05 | 70.67 | — | |
| MRLRep. Size=256, Backbone=ResNet502022.05 | 70.62 | — | |
| MRLRep. Size=128, Backbone=ResNet502022.05 | 70.52 | — | |
| Rand. FSRep. Size=1024, Backbone=ResNet502022.05 | 70.41 | — | |
| MRL-ERep. Size=256, Backbone=ResNet502022.05 | 70.36 | — | |
| FFRep. Size=1024, Backbone=ResNet502022.05 | 70.34 | — | |
| FFRep. Size=512, Backbone=ResNet502022.05 | 70.18 | — | |
| MRLRep. Size=64, Backbone=ResNet502022.05 | 70.17 | — | |
| MRL-ERep. Size=128, Backbone=ResNet502022.05 | 70.12 | — | |
| FFRep. Size=256, Backbone=ResNet502022.05 | 69.72 | — | |
| SVDRep. Size=128, Backbone=ResNet502022.05 | 69.63 | — | |
| MRL-ERep. Size=64, Backbone=ResNet502022.05 | 69.61 | — | |
| MRLRep. Size=32, Backbone=ResNet502022.05 | 69.46 | — | |
| FFRep. Size=64, Backbone=ResNet502022.05 | 69.41 | — | |
| FFRep. Size=128, Backbone=ResNet502022.05 | 69.35 | — | |
| FFRep. Size=32, Backbone=ResNet502022.05 | 68.84 | — | |
| Rand. FSRep. Size=512, Backbone=ResNet502022.05 | 68.77 | — | |
| MRL-ERep. Size=32, Backbone=ResNet502022.05 | 68.6 | — | |
| MRLRep. Size=16, Backbone=ResNet502022.05 | 67.91 | — | |
| SlimmableRep. Size=1024, Backbone=ResNet502022.05 | 67.19 | — | |
| SwAVBackbone=RN50w5, #Parameters=586M, Throughput=762021.06 | 67.1 | 78.5 | |
| MRL-ERep. Size=16, Backbone=ResNet502022.05 | 67.05 | — | |
| SVDRep. Size=64, Backbone=ResNet502022.05 | 67.04 | — | |
| FFRep. Size=16, Backbone=ResNet502022.05 | 66.77 | — | |
| SlimmableRep. Size=2048, Backbone=ResNet502022.05 | 66.1 | — | |
| SlimmableRep. Size=512, Backbone=ResNet502022.05 | 65.82 | — | |
| Rand. FSRep. Size=256, Backbone=ResNet502022.05 | 65.75 | — | |
| MRLRep. Size=8, Backbone=ResNet502022.05 | 62.19 | — | |
| Rand. FSRep. Size=128, Backbone=ResNet502022.05 | 60.91 | — | |
| SVDRep. Size=32, Backbone=ResNet502022.05 | 60.78 | — | |
| SlimmableRep. Size=256, Backbone=ResNet502022.05 | 60.61 | — | |
| FFRep. Size=8, Backbone=ResNet502022.05 | 58.93 | — | |
| MRL-ERep. Size=8, Backbone=ResNet502022.05 | 57.45 | — | |
| SlimmableRep. Size=128, Backbone=ResNet502022.05 | 51.16 | — | |
| Rand. FSRep. Size=64, Backbone=ResNet502022.05 | 49.91 | — | |
| SVDRep. Size=16, Backbone=ResNet502022.05 | 46.02 | — | |
| SlimmableRep. Size=64, Backbone=ResNet502022.05 | 35.6 | — | |
| Rand. FSRep. Size=32, Backbone=ResNet502022.05 | 32.91 | — | |
| SVDRep. Size=8, Backbone=ResNet502022.05 | 19.14 | — | |
| SlimmableRep. Size=32, Backbone=ResNet502022.05 | 16.95 | — | |
| Rand. FSRep. Size=16, Backbone=ResNet502022.05 | 12.06 | — | |
| SlimmableRep. Size=16, Backbone=ResNet502022.05 | 5.12 | — | |
| Rand. FSRep. Size=8, Backbone=ResNet502022.05 | 2.36 | — | |
| SlimmableRep. Size=8, Backbone=ResNet502022.05 | 1 | — | |
| JLRep. Size=8, Backbone=ResNet502022.05 | 0.11 | — | |
| JLRep. Size=16, Backbone=ResNet502022.05 | 0.09 | — | |
| JLRep. Size=32, Backbone=ResNet502022.05 | 0.06 | — | |
| JLRep. Size=128, Backbone=ResNet502022.05 | 0.06 | — | |
| JLRep. Size=64, Backbone=ResNet502022.05 | 0.05 | — | |
| JLRep. Size=256, Backbone=ResNet502022.05 | 0.04 | — | |
| JLRep. Size=512, Backbone=ResNet502022.05 | 0.03 | — | |
| IGPT-XL#Parameters=6801M2021.06 | — | 72 | |
| Masked Patch Pred.Backbone=ViT-B/16, #Parameters=85M, Throughput=312, Pre-training=JFT-300M2021.06 | — | 79.9 | |
| MOBYBackbone=Swin-T, #Parameters=28M, Throughput=8082021.06 | — | 75.1 | |
| MoCo-v3Backbone=ViT-B-BN/7, #Parameters=85M, Throughput=~632021.06 | — | 79.5 | |
| MoCo-v3Backbone=ViT-L-BN/7, #Parameters=304M, Throughput=~172021.06 | — | 81 | |
| MoCo-v3Backbone=ViT-B/16, #Parameters=85M, Throughput=3122021.06 | — | 76.7 | |
| MoCo-v3Backbone=ViT-H-BN/16, #Parameters=632M, Throughput=~322021.06 | — | 79.1 |