Image Classification on ImageNet-1K linear evaluation 1.0
84.8Top-1 AccCLIP-VIT-L/14
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CLIP-VIT-L/14Backbone=ViT-L/14, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=321, # Inference Params (M)=3212024.11 | 84.8 | — | |
| TinyVit-21MBackbone=TinyVit-21M, Over-parameterization method=OPDF, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=38, # Inference Params (M)=212024.11 | 84 | 97.5 | |
| TinyVit-21MBackbone=TinyVit-21M, Over-parameterization method=SVD, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=29, # Inference Params (M)=212024.11 | 82.9 | 96.8 | |
| TinyVit-11MBackbone=TinyVit-11M, Over-parameterization method=OPDF, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=23, # Inference Params (M)=112024.11 | 82.5 | 96.9 | |
| TinyVit-21MBackbone=TinyVit-21M, Over-parameterization method=None, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=21, # Inference Params (M)=212024.11 | 82.3 | 96.3 | |
| TinyVit-11MBackbone=TinyVit-11M, Over-parameterization method=SVD, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=17, # Inference Params (M)=112024.11 | 82 | 96.7 | |
| SupervisedBackbone=Swin-T, Training Epochs=300, Parameters (M)=29, FLOPs (G)=4.5, Throughput (img/s)=755.2, Evaluation Protocol=Linear evaluation2021.05 | 81.3 | — | |
| TinyVit-11MBackbone=TinyVit-11M, Over-parameterization method=None, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=11, # Inference Params (M)=112024.11 | 80.5 | 95.6 | |
| TinyVit-5MBackbone=TinyVit-5M, Over-parameterization method=OPDF, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=9.9, # Inference Params (M)=5.42024.11 | 80 | 96.7 | |
| SupervisedBackbone=DeiT-S, Training Epochs=300, Parameters (M)=22, FLOPs (G)=4.6, Throughput (img/s)=940.4, Evaluation Protocol=Linear evaluation2021.05 | 79.8 | — | |
| TinyVit-5MBackbone=TinyVit-5M, Over-parameterization method=SVD, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=7.6, # Inference Params (M)=5.42024.11 | 77.9 | 95.1 | |
| TinyVit-5MBackbone=TinyVit-5M, Over-parameterization method=None, Pre-trained=ImageNet-21k, Evaluation Protocol=linear probe, # Train Params (M)=5.4, # Inference Params (M)=5.42024.11 | 77.4 | 94.1 | |
| DINOBackbone=DeiT-S, Training Epochs=300, Parameters (M)=22, FLOPs (G)=4.6, Throughput (img/s)=940.4, multi-crop scheme=true, Evaluation Protocol=Linear evaluation2021.05 | 75.9 | — | |
| MoBYBackbone=Swin-T, Training Epochs=300, Parameters (M)=29, FLOPs (G)=4.5, Throughput (img/s)=755.2, Evaluation Protocol=Linear evaluation2021.05 | 75 | — | |
| MoBYBackbone=DeiT-S, Training Epochs=300, Parameters (M)=22, FLOPs (G)=4.6, Throughput (img/s)=940.4, Evaluation Protocol=Linear evaluation2021.05 | 72.8 | — | |
| MoCo v3Backbone=DeiT-S, Training Epochs=300, Parameters (M)=22, FLOPs (G)=4.6, Throughput (img/s)=940.4, Evaluation Protocol=Linear evaluation2021.05 | 72.5 | — | |
| DINOBackbone=DeiT-S, Training Epochs=300, Parameters (M)=22, FLOPs (G)=4.6, Throughput (img/s)=940.4, Evaluation Protocol=Linear evaluation2021.05 | 72.5 | — | |
| MoBYBackbone=Swin-T, Training Epochs=100, Parameters (M)=29, FLOPs (G)=4.5, Throughput (img/s)=755.2, Evaluation Protocol=Linear evaluation2021.05 | 70.9 | — | |
| MoCo.V2(C) + SAMix-CCL method=MoCo.V2(C), Mixup Variant=SAMix-C, Backbone=ResNet-502021.11 | 68.86 | — | |
| MoCo.V2 + SAMix-IPCL method=MoCo.V2, Mixup Variant=SAMix-IP, Backbone=ResNet-502021.11 | 68.82 | — | |
| MoCo.V2 + SAMix-ICL method=MoCo.V2, Mixup Variant=SAMix-I, Backbone=ResNet-502021.11 | 68.76 | — | |
| UnMix+ + Mixup+latentCL method=UnMix+, Mixup Variant=Mixup+latent, Backbone=ResNet-502021.11 | 68.6 | — | |
| MoCo.V2(C) + PuzzleMix*CL method=MoCo.V2(C), Mixup Variant=PuzzleMix*, Backbone=ResNet-502021.11 | 68.48 | — | |
| WBSIM+ + Mixup+CutMixCL method=WBSIM+, Mixup Variant=Mixup+CutMix, Backbone=ResNet-502021.11 | 68.4 | — | |
| MoCo.V2 + SaliencyMixCL method=MoCo.V2, Mixup Variant=SaliencyMix, Backbone=ResNet-502021.11 | 68.31 | — | |
| MoCo.V2 + CutMixCL method=MoCo.V2, Mixup Variant=CutMix, Backbone=ResNet-502021.11 | 68.28 | — | |
| i-Mix+ + Mixup+latentCL method=i-Mix+, Mixup Variant=Mixup+latent, Backbone=ResNet-502021.11 | 68.1 | — | |
| MoCo.V2 + MixupCL method=MoCo.V2, Mixup Variant=Mixup, Backbone=ResNet-502021.11 | 68.07 | — | |
| MoCHi + Mixup+latentCL method=MoCHi, Mixup Variant=Mixup+latent, Backbone=ResNet-502021.11 | 68.01 | — | |
| MoCo.V2CL method=MoCo.V2, Backbone=ResNet-502021.11 | 67.66 | — | |
| MoCo.V2(C) + SAMix-CCL method=MoCo.V2(C), Mixup Variant=SAMix-C, Backbone=ResNet-182021.11 | 53.93 | — | |
| MoCo.V2 + SAMix-ICL method=MoCo.V2, Mixup Variant=SAMix-I, Backbone=ResNet-182021.11 | 53.75 | — | |
| MoCo.V2 + SAMix-IPCL method=MoCo.V2, Mixup Variant=SAMix-IP, Backbone=ResNet-182021.11 | 53.72 | — | |
| MoCo.V2(C) + PuzzleMix*CL method=MoCo.V2(C), Mixup Variant=PuzzleMix*, Backbone=ResNet-182021.11 | 53.46 | — | |
| MoCHi + Mixup+latentCL method=MoCHi, Mixup Variant=Mixup+latent, Backbone=ResNet-182021.11 | 53.12 | — | |
| i-Mix+ + Mixup+latentCL method=i-Mix+, Mixup Variant=Mixup+latent, Backbone=ResNet-182021.11 | 53.09 | — | |
| MoCo.V2 + SaliencyMixCL method=MoCo.V2, Mixup Variant=SaliencyMix, Backbone=ResNet-182021.11 | 53.06 | — | |
| MoCo.V2 + MixupCL method=MoCo.V2, Mixup Variant=Mixup, Backbone=ResNet-182021.11 | 53.03 | — | |
| MoCo.V2 + CutMixCL method=MoCo.V2, Mixup Variant=CutMix, Backbone=ResNet-182021.11 | 52.98 | — | |
| MoCo.V2CL method=MoCo.V2, Backbone=ResNet-182021.11 | 52.85 | — |