Image Classification on ImageNet-1k 1.0 (test)
0.87Top-1 AccuracyDeiT III L
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeiT III LPre-training data=IN-21K, Data aug.=true, Extra labels=false2023.12 | 0.87 | — | |
| MAE LPre-training data=IN-1K, Data aug.=true, Extra labels=false2023.12 | 0.868 | — | |
| 4M-LPre-training data=CC12M, Data aug.=false, Extra labels=true2023.12 | 0.866 | — | |
| CoCaPre-training Dataset=JFT-3B + ALIGN-1.8B, Architecture=CoCa-2B, Resolution=5762023.03 | 0.863 | — | |
| DeiT III BPre-training data=IN-21K, Data aug.=true, Extra labels=false2023.12 | 0.854 | — | |
| Scale-ViT + LiTPre-training Dataset=JFT-3B + ALIGN-3.6B, Architecture=ViT-g, Resolution=2882023.03 | 0.852 | — | |
| CAFormer-M36Token Mixing Type=Conv + Attn, Parameters (M)=56, MACs (G)=13.2, Resolution=224x2242024.05 | 0.852 | — | |
| DeiT-III-Large-BaseBackbone=DeiT-III-Large, Params(M)=304.4, GFLOPs=119.4, Resolution=224x2242023.10 | 0.849 | — | |
| InternImage-BToken Mixing Type=Conv, Parameters (M)=97, MACs (G)=16, Resolution=224x2242024.05 | 0.849 | — | |
| iFormer-LToken Mixing Type=Conv + Attn, Parameters (M)=87, MACs (G)=14, Resolution=224x2242024.05 | 0.848 | — | |
| TransNeXt-BaseToken Mixing Type=Conv + Attn, Parameters (M)=90, MACs (G)=18.4, Resolution=224x2242024.05 | 0.848 | — | |
| VVT-LResolution=384x384, Param=61.8M, GFLOPS=31.82022.06 | 0.847 | — | |
| TransNeXt-SmallToken Mixing Type=Conv + Attn, Parameters (M)=50, MACs (G)=10.3, Resolution=224x2242024.05 | 0.847 | — | |
| MogaNet-LToken Mixing Type=Conv, Parameters (M)=83, MACs (G)=15.9, Resolution=224x2242024.05 | 0.847 | — | |
| MOAT-2Token Mixing Type=Conv + Attn, Parameters (M)=73, MACs (G)=17.2, Resolution=224x2242024.05 | 0.847 | — | |
| VVT-MResolution=384x384, Param=47.9M, GFLOPS=27.72022.06 | 0.845 | — | |
| Swin-BResolution=384x384, Param=88.0M, GFLOPS=472022.06 | 0.845 | — | |
| 4M-BPre-training data=CC12M, Data aug.=false, Extra labels=true2023.12 | 0.845 | — | |
| CAFormer-S36Token Mixing Type=Conv + Attn, Parameters (M)=39, MACs (G)=8, Resolution=224x2242024.05 | 0.845 | — | |
| ConvFormer-M36Token Mixing Type=Conv, Parameters (M)=57, MACs (G)=12.8, Resolution=224x2242024.05 | 0.845 | — | |
| MViTv2-BToken Mixing Type=Attn, Parameters (M)=52, MACs (G)=10.2, Resolution=224x2242024.05 | 0.844 | — | |
| MogaNet-BToken Mixing Type=Conv, Parameters (M)=44, MACs (G)=9.9, Resolution=224x2242024.05 | 0.843 | — | |
| HorNet-BToken Mixing Type=Conv, Parameters (M)=88, MACs (G)=15.5, Resolution=224x2242024.05 | 0.843 | — | |
| MAE BPre-training data=IN-1K, Data aug.=true, Extra labels=false2023.12 | 0.842 | — | |
| InternImage-SToken Mixing Type=Conv, Parameters (M)=50, MACs (G)=8, Resolution=224x2242024.05 | 0.842 | — | |
| CSWin-BToken Mixing Type=Attn, Parameters (M)=78, MACs (G)=15, Resolution=224x2242024.05 | 0.842 | — | |
| MambaOut-BaseToken Mixing Type=Conv, Parameters (M)=85, MACs (G)=15.8, Resolution=224x2242024.05 | 0.842 | — | |
| ConvFormer-S36Token Mixing Type=Conv, Parameters (M)=40, MACs (G)=7.6, Resolution=224x2242024.05 | 0.841 | — | |
| SG-Former-MToken Mixing Type=Conv + Attn, Parameters (M)=39, MACs (G)=7.5, Resolution=224x2242024.05 | 0.841 | — | |
| MambaOut-SmallToken Mixing Type=Conv, Parameters (M)=48, MACs (G)=9, Resolution=224x2242024.05 | 0.841 | — | |
| CoAtNet-2Token Mixing Type=Conv + Attn, Parameters (M)=75, MACs (G)=15.7, Resolution=224x2242024.05 | 0.841 | — | |
| MultiMAE BPre-training data=IN-1K, Data aug.=true, Extra labels=true2023.12 | 0.84 | — | |
| TransNeXt-TinyToken Mixing Type=Conv + Attn, Parameters (M)=28, MACs (G)=5.7, Resolution=224x2242024.05 | 0.84 | — | |
| Swin-SResolution=384x384, Param=50.0M, GFLOPS=272022.06 | 0.839 | — | |
| PELABackbone=DeiT-III-Large, Params(M)=153.2, GFLOPs=59.8, Resolution=224x2242023.10 | 0.839 | — | |
| VAN-B3Token Mixing Type=Conv, Parameters (M)=45, MACs (G)=9, Resolution=224x2242024.05 | 0.839 | — | |
| UniFormer-BToken Mixing Type=Conv + Attn, Parameters (M)=50, MACs (G)=8.3, Resolution=224x2242024.05 | 0.839 | — | |
| VMambaV9-BToken Mixing Type=Conv + SSM, Parameters (M)=89, MACs (G)=15.4, Resolution=224x2242024.05 | 0.839 | — | |
| ConvNeXt-BToken Mixing Type=Conv, Parameters (M)=89, MACs (G)=15.4, Resolution=224x2242024.05 | 0.838 | — | |
| FlorencePre-training Dataset=FLD-900M, Architecture=CoSwin-H, Resolution=3842023.03 | 0.837 | — | |
| LocalVMamba-SToken Mixing Type=Conv + SSM, Parameters (M)=50, MACs (G)=11.4, Resolution=224x2242024.05 | 0.837 | — | |
| VMamba-BToken Mixing Type=Conv + SSM, Parameters (M)=75, MACs (G)=18, Resolution=224x2242024.05 | 0.837 | — | |
| CAFormer-S18Token Mixing Type=Conv + Attn, Parameters (M)=26, MACs (G)=4.1, Resolution=224x2242024.05 | 0.836 | — | |
| CSWin-SToken Mixing Type=Attn, Parameters (M)=35, MACs (G)=6.9, Resolution=224x2242024.05 | 0.836 | — | |
| MViTv2-SToken Mixing Type=Attn, Parameters (M)=35, MACs (G)=7, Resolution=224x2242024.05 | 0.836 | — | |
| VMambaV9-SToken Mixing Type=Conv + SSM, Parameters (M)=50, MACs (G)=8.7, Resolution=224x2242024.05 | 0.836 | — | |
| Swin-BaseBackbone=Swin-Base, Params(M)=87.8, GFLOPs=30.3, Resolution=224x2242023.10 | 0.835 | — | |
| InternImage-TToken Mixing Type=Conv, Parameters (M)=30, MACs (G)=5, Resolution=224x2242024.05 | 0.835 | — | |
| Focal-SmallToken Mixing Type=Attn, Parameters (M)=51, MACs (G)=9.1, Resolution=224x2242024.05 | 0.835 | — | |
| VMamba-SToken Mixing Type=Conv + SSM, Parameters (M)=44, MACs (G)=11.2, Resolution=224x2242024.05 | 0.835 | — | |
| RepLKNet-31BToken Mixing Type=Conv, Parameters (M)=79, MACs (G)=15.3, Resolution=224x2242024.05 | 0.835 | — | |
| Swin-BToken Mixing Type=Attn, Parameters (M)=88, MACs (G)=15.4, Resolution=224x2242024.05 | 0.835 | — | |
| VVT-SResolution=384x384, Param=25.5M, GFLOPS=16.52022.06 | 0.834 | — | |
| 4M-B (RGB → CLIP only)Pre-training data=CC12M, Data aug.=false, Extra labels=true2023.12 | 0.834 | — | |
| MogaNet-SToken Mixing Type=Conv, Parameters (M)=25, MACs (G)=5, Resolution=224x2242024.05 | 0.834 | — | |
| iFormer-SToken Mixing Type=Conv + Attn, Parameters (M)=20, MACs (G)=4.8, Resolution=224x2242024.05 | 0.834 | — | |
| CVT-21Resolution=384x384, Param=32.0M, GFLOPS=24.92022.06 | 0.833 | — | |
| MOAT-0Token Mixing Type=Conv + Attn, Parameters (M)=28, MACs (G)=5.7, Resolution=224x2242024.05 | 0.833 | — | |
| CoAtNet-1Token Mixing Type=Conv + Attn, Parameters (M)=42, MACs (G)=8.4, Resolution=224x2242024.05 | 0.833 | — | |
| SG-Former-SToken Mixing Type=Conv + Attn, Parameters (M)=23, MACs (G)=4.8, Resolution=224x2242024.05 | 0.832 | — | |
| ConvNeXt-SToken Mixing Type=Conv, Parameters (M)=50, MACs (G)=8.7, Resolution=224x2242024.05 | 0.831 | — | |
| ConvFormer-S18Token Mixing Type=Conv, Parameters (M)=27, MACs (G)=3.9, Resolution=224x2242024.05 | 0.83 | — | |
| Swin-SToken Mixing Type=Attn, Parameters (M)=50, MACs (G)=8.7, Resolution=224x2242024.05 | 0.83 | — | |
| Mamba-2D-BToken Mixing Type=Conv + SSM, Parameters (M)=92, MACs (G)=20.3, Resolution=224x2242024.05 | 0.83 | — | |
| RegNetY-16GParams(M)=83.6, GFLOPs=31.9, Resolution=224x2242023.10 | 0.829 | — | |
| 4M-B (RGB → RGB only)Pre-training data=CC12M, Data aug.=false, Extra labels=false2023.12 | 0.828 | — | |
| VAN-B2Token Mixing Type=Conv, Parameters (M)=27, MACs (G)=5, Resolution=224x2242024.05 | 0.828 | — | |
| CSWin-TToken Mixing Type=Attn, Parameters (M)=23, MACs (G)=4.3, Resolution=224x2242024.05 | 0.827 | — | |
| LocalVMamba-TToken Mixing Type=Conv + SSM, Parameters (M)=26, MACs (G)=5.7, Resolution=224x2242024.05 | 0.827 | — | |
| MambaOut-TinyToken Mixing Type=Conv, Parameters (M)=27, MACs (G)=4.5, Resolution=224x2242024.05 | 0.827 | — | |
| PELABackbone=Swin-Base, Params(M)=62.2, GFLOPs=21.3, Resolution=224x2242023.10 | 0.825 | — | |
| VMambaV9-T*Token Mixing Type=Conv + SSM, Parameters (M)=31, MACs (G)=4.9, Resolution=224x2242024.05 | 0.825 | — | |
| T2T-ViT-24Params(M)=64.1, GFLOPs=25.5, Resolution=224x2242023.10 | 0.823 | — | |
| InceptionNeXt-TToken Mixing Type=Conv, Parameters (M)=28, MACs (G)=4.2, Resolution=224x2242024.05 | 0.823 | — | |
| T2T-ViT-24Token Mixing Type=Attn, Parameters (M)=64, MACs (G)=13.8, Resolution=224x2242024.05 | 0.823 | — | |
| PlainMamba-L3Token Mixing Type=Conv + SSM, Parameters (M)=50, MACs (G)=14.4, Resolution=224x2242024.05 | 0.823 | — | |
| Swin-TResolution=384x384, Param=29.0M, GFLOPS=142022.06 | 0.822 | — | |
| CrossViT-BParams(M)=105.0, GFLOPs=40.3, Resolution=224x2242023.10 | 0.822 | — | |
| Focal-TinyToken Mixing Type=Attn, Parameters (M)=29, MACs (G)=4.9, Resolution=224x2242024.05 | 0.822 | — | |
| VMamba-TToken Mixing Type=Conv + SSM, Parameters (M)=22, MACs (G)=5.6, Resolution=224x2242024.05 | 0.822 | — | |
| MAWSPre-training Dataset=IG-3B, Architecture=ViT-2B, Resolution=2242023.03 | 0.821 | — | |
| ConvNeXt-TToken Mixing Type=Conv, Parameters (M)=29, MACs (G)=4.5, Resolution=224x2242024.05 | 0.821 | — | |
| SHViT-S4Resolution=512, Epochs=360, FLOPs (M)=3973, Params (M)=16.5, Throughput (GPU)=3957, Throughput (CPU)=8, Throughput (CPU_ONNX)=198, Note=Finetuning with higher resolution2024.01 | 0.82 | 0.959 | |
| T2T-ViT-19Token Mixing Type=Attn, Parameters (M)=39, MACs (G)=8.5, Resolution=224x2242024.05 | 0.819 | — | |
| DeiT-BaseBackbone=DeiT, Params(M)=86.6, GFLOPs=33.7, Resolution=224x2242023.10 | 0.818 | — | |
| Efficient VMamba-BToken Mixing Type=Conv + SSM, Parameters (M)=33, MACs (G)=4, Resolution=224x2242024.05 | 0.818 | — | |
| DeiT-BToken Mixing Type=Attn, Parameters (M)=86, MACs (G)=17.5, Resolution=224x2242024.05 | 0.818 | — | |
| Mamba-2D-SToken Mixing Type=Conv + SSM, Parameters (M)=24, MACs (G)=5.2, Resolution=224x2242024.05 | 0.817 | — | |
| CoAtNet-0Token Mixing Type=Conv + Attn, Parameters (M)=25, MACs (G)=4.2, Resolution=224x2242024.05 | 0.816 | — | |
| PlainMamba-L2Token Mixing Type=Conv + SSM, Parameters (M)=25, MACs (G)=8.1, Resolution=224x2242024.05 | 0.816 | — | |
| T2T-ViT-14Token Mixing Type=Attn, Parameters (M)=22, MACs (G)=4.8, Resolution=224x2242024.05 | 0.815 | — | |
| Swin-TToken Mixing Type=Attn, Parameters (M)=29, MACs (G)=4.5, Resolution=224x2242024.05 | 0.813 | — | |
| LocalVim-SToken Mixing Type=Conv + SSM, Parameters (M)=28, MACs (G)=4.8, Resolution=224x2242024.05 | 0.812 | — | |
| PELABackbone=DeiT, Params(M)=44.1, GFLOPs=17.0, Resolution=224x2242023.10 | 0.81 | — | |
| SHViT-S4Resolution=384, Epochs=330, FLOPs (M)=2225, Params (M)=16.5, Throughput (GPU)=6702, Throughput (CPU)=14, Throughput (CPU_ONNX)=315, Note=Finetuning with higher resolution2024.01 | 0.81 | 0.954 | |
| Scale-ViT + LiTPre-training Dataset=JFT-3B + ALIGN-3.6B, Architecture=ViT-L, Resolution=2242023.03 | 0.808 | — | |
| MAWSPre-training Dataset=IG-3B, Architecture=ViT-H, Resolution=2242023.03 | 0.808 | — | |
| EfficientViT-M5Resolution=512, Epochs=360, FLOPs (M)=2670, Params (M)=12.4, Throughput (GPU)=3777, Throughput (CPU)=9, Throughput (CPU_ONNX)=88, Note=Finetuning with higher resolution2024.01 | 0.808 | 0.955 | |
| Vim-SToken Mixing Type=Conv + SSM, Parameters (M)=26, MACs (G)=5.1, Resolution=224x2242024.05 | 0.805 | — | |
| VVT-TResolution=384x384, Param=12.9M, GFLOPS=8.82022.06 | 0.803 | — |