Image Classification on ImageNet-1K (val) (Top-1 and Top-5 Accuracy)
86.22Top-1 AccuracyFiPS + FT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FiPS + FTModel=Swin-L, Parameter Budget=75%, Compression Ratio=∼6%2024.11 | 86.22 | — | |
| FiPSModel=Swin-L, Parameter Budget=75%, Compression Ratio=∼6%2024.11 | 86.21 | — | |
| GFMModel=Swin-L, Parameter Budget=75%, Compression Ratio=∼6%2024.11 | 86.01 | — | |
| FiPSModel=Swin-L, Parameter Budget=50%, Compression Ratio=∼25%2024.11 | 85.99 | — | |
| FiPS + FTModel=Swin-L, Parameter Budget=50%, Compression Ratio=∼25%2024.11 | 85.99 | — | |
| AAFMModel=Swin-L, Parameter Budget=75%, Compression Ratio=∼6%2024.11 | 85.94 | — | |
| FiPSModel=Swin-L, Parameter Budget=40%, Compression Ratio=∼33%2024.11 | 85.69 | — | |
| FiPS + FTModel=Swin-L, Parameter Budget=40%, Compression Ratio=∼33%2024.11 | 85.68 | — | |
| GFMModel=Swin-L, Parameter Budget=50%, Compression Ratio=∼25%2024.11 | 85.44 | — | |
| FiPS + FTModel=Swin-L, Parameter Budget=25%, Compression Ratio=∼45%2024.11 | 85.16 | — | |
| AAFMModel=Swin-L, Parameter Budget=50%, Compression Ratio=∼25%2024.11 | 85.04 | — | |
| FiPSModel=Swin-L, Parameter Budget=25%, Compression Ratio=∼45%2024.11 | 84.78 | — | |
| Baseline/512Model family=LV-VIT-M, Params (M)=55.83, GFLOPS=12.67, GFLOPs↓ (%)=0.00, Embedding dimensions=5122021.12 | 84 | — | |
| DynamicViTModel family=LV-VIT-M, Params (M)=57.10, GFLOPS=8.45, GFLOPs↓ (%)=33.312021.12 | 83.8 | — | |
| SPVITModel family=LV-VIT-M, Params (M)=55.85, GFLOPS=7.32, GFLOPs↓ (%)=42.232021.12 | 83.71 | — | |
| DynamicViTModel family=LV-VIT-M, Params (M)=57.10, GFLOPS=7.35, GFLOPs↓ (%)=41.99, Reproduced=true2021.12 | 83.61 | — | |
| Baseline/384Model family=LV-VIT-S, Params (M)=26.15, GFLOPS=6.55, GFLOPs↓ (%)=0.00, Embedding dimensions=3842021.12 | 83.3 | — | |
| Swin-BModel family=LV-VIT-M, Params (M)=88.00, GFLOPS=15.40, GFLOPs↓ (%)=-21.552021.12 | 83.3 | — | |
| CvT-21/384Model family=LV-VIT-M, Params (M)=32.00, GFLOPS=24.90, GFLOPs↓ (%)=-96.532021.12 | 83.3 | — | |
| SPVITModel family=LV-VIT-S, Params (M)=26.17, GFLOPS=4.28, GFLOPs↓ (%)=34.652021.12 | 83.1 | — | |
| ATS+CVT-21/384Model family=LV-VIT-M, Params (M)=32.00, GFLOPS=17.40, GFLOPs↓ (%)=-37.332021.12 | 83.1 | — | |
| DynamicViTModel family=LV-VIT-S, Params (M)=26.90, GFLOPS=4.57, GFLOPs↓ (%)=30.222021.12 | 83 | — | |
| Swin-SModel family=LV-VIT-M, Params (M)=50.00, GFLOPS=8.70, GFLOPs↓ (%)=31.332021.12 | 83 | — | |
| CvT-13/384Model family=LV-VIT-M, Params (M)=20.00, GFLOPS=16.30, GFLOPs↓ (%)=-28.652021.12 | 83 | — | |
| ATS+CVT-13/384Model family=LV-VIT-M, Params (M)=20.00, GFLOPS=11.70, GFLOPs↓ (%)=7.662021.12 | 82.9 | — | |
| RegNetY-16GModel family=LV-VIT-M, Params (M)=84.00, GFLOPS=16.00, GFLOPs↓ (%)=-26.282021.12 | 82.9 | — | |
| TNT-BModel family=LV-VIT-M, Params (M)=66.00, GFLOPS=14.10, GFLOPs↓ (%)=-11.292021.12 | 82.8 | — | |
| CvT-21/224Model family=LV-VIT-M, Params (M)=32.00, GFLOPS=7.10, GFLOPs↓ (%)=43.962021.12 | 82.5 | — | |
| ATS+CVT-21Model family=LV-VIT-S, Params (M)=32.00, GFLOPS=5.10, GFLOPs↓ (%)=22.142021.12 | 82.3 | — | |
| T2T-ViT-24Model family=LV-VIT-M, Params (M)=64.10, GFLOPS=14.10, GFLOPs↓ (%)=-11.292021.12 | 82.3 | — | |
| DHVT-S#Params=24.1M, Image Size=224, GFLOPs=4.7, Initialization=random2022.10 | 82.3 | — | |
| CrossViT-BModel family=LV-VIT-M, Params (M)=104.70, GFLOPS=21.20, GFLOPs↓ (%)=-67.322021.12 | 82.2 | — | |
| Focal-T#Params=29.1M, Image Size=224, GFLOPs=4.9, Initialization=random2022.10 | 82.2 | — | |
| FiPS + FTModel=Swin-L, Parameter Budget=10%, Compression Ratio=∼57%2024.11 | 82.13 | — | |
| ConvNeXt-T#Params=29M, Image Size=224, GFLOPs=4.5, Initialization=random2022.10 | 82.1 | — | |
| LG-T#Params=32.6M, Image Size=224, GFLOPs=4.8, Initialization=random2022.10 | 82.1 | — | |
| CeiT-S#Params=24.2M, Image Size=224, GFLOPs=4.5, Initialization=random2022.10 | 82 | — | |
| ViL-S#Params=24.6M, Image Size=224, GFLOPs=4.9, Initialization=random2022.10 | 82 | — | |
| PVTv2-B2#Params=25.4M, Image Size=224, GFLOPs=4.0, Initialization=random2022.10 | 82 | — | |
| VITAE-S#Params=23.6M, Image Size=224, GFLOPs=5.6, Initialization=random2022.10 | 82 | — | |
| T2T-ViT-19Model family=LV-VIT-M, Params (M)=39.20, GFLOPS=8.90, GFLOPs↓ (%)=29.762021.12 | 81.9 | — | |
| CoaT-Lite Small#Params=20.0M, Image Size=224, GFLOPs=4.0, Initialization=random2022.10 | 81.9 | — | |
| GFMModel=DeiT-B, Parameter Budget=75%, Compression Ratio=∼6%2024.11 | 81.83 | — | |
| FiPSModel=DeiT-B, Parameter Budget=50%, Compression Ratio=∼25%2024.11 | 81.83 | — | |
| FiPSModel=DeiT-B, Parameter Budget=75%, Compression Ratio=∼6%2024.11 | 81.82 | — | |
| FiPS + FTModel=DeiT-B, Parameter Budget=75%, Compression Ratio=∼6%2024.11 | 81.82 | — | |
| DeiT-BModel family=LV-VIT-M, Params (M)=86.60, GFLOPS=17.60, GFLOPs↓ (%)=-38.912021.12 | 81.8 | — | |
| CaiT-XS24#Params=26.6M, Image Size=224, GFLOPs=5.4, Initialization=random2022.10 | 81.8 | — | |
| AAFMModel=DeiT-B, Parameter Budget=75%, Compression Ratio=∼6%2024.11 | 81.76 | — | |
| RegNetY-8GModel family=LV-VIT-M, Params (M)=39.00, GFLOPS=8.00, GFLOPs↓ (%)=36.862021.12 | 81.7 | — | |
| Twins-SVT-S#Params=24.0M, Image Size=224, GFLOPs=2.8, Initialization=random2022.10 | 81.7 | — | |
| FiPSModel=DeiT-B, Parameter Budget=40%, Compression Ratio=∼33%2024.11 | 81.69 | — | |
| GFMModel=DeiT-B, Parameter Budget=50%, Compression Ratio=∼25%2024.11 | 81.62 | — | |
| CvT-13/224Model family=LV-VIT-S, Params (M)=20.00, GFLOPS=4.50, GFLOPs↓ (%)=31.292021.12 | 81.6 | — | |
| CvT-13#Params=20.0M, Image Size=224, GFLOPs=4.5, Initialization=random2022.10 | 81.6 | — | |
| FiPS + FTModel=DeiT-B, Parameter Budget=40%, Compression Ratio=∼33%2024.11 | 81.54 | — | |
| FiPS + FTModel=DeiT-B, Parameter Budget=50%, Compression Ratio=∼25%2024.11 | 81.54 | — | |
| CPVT-Small-GAPModel family=LV-VIT-S, Params (M)=23.00, GFLOPS=4.60, GFLOPs↓ (%)=29.772021.12 | 81.5 | — | |
| T2T-ViT-14Model family=LV-VIT-S, Params (M)=22.00, GFLOPS=5.20, GFLOPs↓ (%)=20.612021.12 | 81.5 | — | |
| T2T-ViT-14#Params=21.5M, Image Size=224, GFLOPs=5.2, Initialization=random2022.10 | 81.5 | — | |
| NesT-T#Params=17.0M, Image Size=224, GFLOPs=5.8, Initialization=random2022.10 | 81.5 | — | |
| MD-DeiT-SModel family=LV-VIT-S, Params (M)=22.10, GFLOPS=4.60, GFLOPs↓ (%)=29.772021.12 | 81.48 | — | |
| FiPS + FTModel=DeiT-B, Parameter Budget=25%, Compression Ratio=∼45%2024.11 | 81.31 | — | |
| Swin-TModel family=LV-VIT-S, Params (M)=29.00, GFLOPS=4.50, GFLOPs↓ (%)=31.292021.12 | 81.3 | — | |
| Conformer-Ti#Params=23.5M, Image Size=224, GFLOPs=5.2, Initialization=random2022.10 | 81.3 | — | |
| Swin-T#Params=29.0M, Image Size=224, GFLOPs=4.5, Initialization=random2022.10 | 81.3 | — | |
| ConViT-S#Params=27.8M, Image Size=224, GFLOPs=5.4, Initialization=random2022.10 | 81.3 | — | |
| TNT-S#Params=23.8M, Image Size=224, GFLOPs=5.2, Initialization=random2022.10 | 81.3 | — | |
| GFMModel=DeiT-B, Parameter Budget=40%, Compression Ratio=∼33%2024.11 | 81.28 | — | |
| NVIT-SModel family=LV-VIT-S, Params (M)=23.00, GFLOPS=4.70, GFLOPs↓ (%)=28.242021.12 | 81.22 | — | |
| AAFMModel=DeiT-B, Parameter Budget=50%, Compression Ratio=∼25%2024.11 | 81.21 | — | |
| PVT-MediumModel family=LV-VIT-S, Params (M)=44.20, GFLOPS=6.70, GFLOPs↓ (%)=-2.292021.12 | 81.2 | — | |
| PVT-Medium#Params=44.2M, Image Size=224, GFLOPs=6.7, Initialization=random2022.10 | 81.2 | — | |
| Evo-ViTModel family=LV-VIT-M, Params (M)=87.30, GFLOPS=9.07, GFLOPs↓ (%)=28.412021.12 | 81.11 | — | |
| CrossViT-SModel family=LV-VIT-S, Params (M)=26.70, GFLOPS=5.60, GFLOPs↓ (%)=14.502021.12 | 81 | — | |
| CrossViT-S#Params=26.7M, Image Size=224, GFLOPs=5.6, Initialization=random2022.10 | 81 | — | |
| PiT-S#Params=23.5M, Image Size=224, GFLOPs=2.9, Initialization=random2022.10 | 80.9 | — | |
| CoaT MiniModel family=LV-VIT-S, Params (M)=10.00, GFLOPS=6.80, GFLOPs↓ (%)=-3.822021.12 | 80.8 | — | |
| VTPModel family=LV-VIT-M, Params (M)=48.00, GFLOPS=10.00, GFLOPs↓ (%)=21.072021.12 | 80.7 | — | |
| FiPSModel=DeiT-B, Parameter Budget=25%, Compression Ratio=∼45%2024.11 | 80.64 | — | |
| AAFMModel=DeiT-B, Parameter Budget=40%, Compression Ratio=∼33%2024.11 | 80.33 | — | |
| Baseline/384Model family=DeiT-S, Params (M)=22.10, GFLOPS=4.60, GFLOPs↓ (%)=0.00, Embedding dimensions=3842021.12 | 79.8 | — | |
| SPVITModel family=DeiT-S, Params (M)=22.13, GFLOPS=3.86, GFLOPs↓ (%)=16.092021.12 | 79.8 | — | |
| DeiT-S#Params=22.1M, Image Size=224, GFLOPs=4.3, Initialization=random2022.10 | 79.8 | — | |
| PVT-S#Params=24.5M, Image Size=224, GFLOPs=3.8, Initialization=random2022.10 | 79.8 | — | |
| RegNetY-4.0GF#Params=20.6M, Image Size=224, GFLOPs=4.0, Initialization=random2022.10 | 79.4 | — | |
| SPVITModel family=DeiT-S, Params (M)=22.13, GFLOPS=2.64, GFLOPs↓ (%)=42.612021.12 | 79.34 | — | |
| DynamicViTModel family=DeiT-S, Params (M)=22.80, GFLOPS=2.91, GFLOPs↓ (%)=36.742021.12 | 79.3 | — | |
| S2VITEModel family=DeiT-S, Params (M)=14.60, GFLOPS=3.14, GFLOPs↓ (%)=31.742021.12 | 79.22 | — | |
| DynamicViTModel family=DeiT-S, Params (M)=22.80, GFLOPS=2.71, GFLOPs↓ (%)=41.09, Reproduced=true2021.12 | 79.12 | — | |
| IA-RED2Model family=DeiT-S, GFLOPS=3.15, GFLOPs↓ (%)=31.522021.12 | 79.1 | — | |
| Baseline/320Model family=DeiT-S, Params (M)=15.40, GFLOPS=3.25, GFLOPs↓ (%)=29.35, Embedding dimensions=320, Reproduced=true2021.12 | 79 | — | |
| SPVIT-320Model family=DeiT-S, Params (M)=15.44, GFLOPS=2.00, GFLOPs↓ (%)=56.52, Pruning source=DeiT scaling model of 320 embedding dimensions2021.12 | 78.65 | — | |
| Baseline/288Model family=DeiT-S, Params (M)=12.60, GFLOPS=2.65, GFLOPs↓ (%)=42.39, Embedding dimensions=288, Reproduced=true2021.12 | 78.53 | — | |
| HVT-S-1Model family=DeiT-S, Params (M)=22.10, GFLOPS=2.40, GFLOPs↓ (%)=47.822021.12 | 78 | — | |
| FiPS + FTModel=DeiT-B, Parameter Budget=10%, Compression Ratio=∼57%2024.11 | 77.26 | — | |
| Baseline/256Model family=DeiT-S, Params (M)=10.00, GFLOPS=2.14, GFLOPs↓ (%)=53.48, Embedding dimensions=256, Reproduced=true2021.12 | 77.21 | — | |
| SPVIT-320Model family=DeiT-T, Params (M)=15.44, GFLOPS=1.30, GFLOPs↓ (%)=0.00, Pruning source=DeiT scaling model of 320 embedding dimensions2021.12 | 77.02 | — | |
| ATS+DeiT/258Model family=DeiT-T, Params (M)=10.13, GFLOPS=1.50, GFLOPs↓ (%)=-15.382021.12 | 76.9 | — | |
| SPVIT-256Model family=DeiT-T, Params (M)=10.16, GFLOPS=1.29, GFLOPs↓ (%)=0.77, Pruning source=DeiT scaling model of 256 embedding dimensions2021.12 | 76.87 | — |