Image Classification on ImageNet 1k (Top-1, Avg Top-1, Rank)
83.5Top-1 AccSwin-B (Dense)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Swin-B (Dense)Backbone=Swin-B, MACs (G)=15.4, Throughput (img/s)=690.25, Params (M)=87.8, Speedup (x)=1.002026.07 | 83.5 | — | — | |
| Swin-S (Dense)Backbone=Swin-S, MACs (G)=8.7, Throughput (img/s)=1106.9, Params (M)=49.6, Speedup (x)=1.002026.07 | 83.2 | — | — | |
| HetDPTBackbone=Swin-B, MACs (G)=9.8, Throughput (img/s)=1106.95, Params (M)=55.3, Speedup (x)=1.602026.07 | 82.9 | — | — | |
| SAViTBackbone=Swin-B, MACs (G)=7.7, Throughput (img/s)=1056.08, Params (M)=33.0, Speedup (x)=1.532026.07 | 82.6 | — | — | |
| HetDPT (10 layers)Backbone=Swin-S, MACs (G)=7.3, Throughput (img/s)=1365.9, Params (M)=43.9, Speedup (x)=1.232026.07 | 82.4 | — | — | |
| DW Prun. (0.2-24)Backbone=Swin-S, MACs (G)=6.8, Throughput (img/s)=1140.1, Params (M)=39.2, Speedup (x)=1.102026.07 | 82.2 | — | — | |
| HetDPT (8 layers)Backbone=DeiT-B, MACs (G)=11.8, Throughput (img/s)=1048.5, Params (M)=61.4, Speedup (x)=1.412026.07 | 82.1 | — | — | |
| DW Prun. (0.2-12)Backbone=DeiT-B, MACs (G)=13.6, Throughput (img/s)=907.6, Params (M)=68.2, Speedup (x)=1.222026.07 | 81.9 | — | — | |
| DeiT-B (Dense)Backbone=DeiT-B, MACs (G)=17.6, Throughput (img/s)=741.5, Params (M)=86.6, Speedup (x)=1.002026.07 | 81.8 | — | — | |
| NOSE (5 layers)Backbone=DeiT-B, MACs (G)=14.6, Throughput (img/s)=894.7, Params (M)=75.5, Speedup (x)=1.212026.07 | 81.8 | — | — | |
| HetDPT (10 layers)Backbone=DeiT-B, MACs (G)=10.5, Throughput (img/s)=1169.7, Params (M)=54.9, Speedup (x)=1.582026.07 | 81.8 | — | — | |
| DW Prun. (0.2-22)Backbone=Swin-S, MACs (G)=6.3, Throughput (img/s)=1273.0, Params (M)=34.4, Speedup (x)=1.152026.07 | 81.8 | — | — | |
| NOSE (6 layers)Backbone=DeiT-B, MACs (G)=14.1, Throughput (img/s)=921.4, Params (M)=73.2, Speedup (x)=1.242026.07 | 81.5 | — | — | |
| SGDModel=ResNet-101, Epoch=2002025.06 | 81.3 | — | — | |
| MACModel=ResNet-101, Epoch=2002025.06 | 81.2 | — | — | |
| KFACModel=ResNet-101, Epoch=2002025.06 | 81.1 | — | — | |
| EVAModel=ResNet-101, Epoch=2002025.06 | 81.1 | — | — | |
| FOOFModel=ResNet-101, Epoch=2002025.06 | 81 | — | — | |
| DW Prun. (0.2-10)Backbone=DeiT-B, MACs (G)=10.8, Throughput (img/s)=1037.9, Params (M)=59.4, Speedup (x)=1.402026.07 | 80.9 | — | — | |
| ADAMWModel=ResNet-101, Epoch=2002025.06 | 80.6 | — | — | |
| MACModel=Swin-Tiny, Epoch=2002025.06 | 80.1 | — | — | |
| ADAMWModel=Swin-Tiny, Epoch=2002025.06 | 80.1 | — | — | |
| HetDPT (6 layers)Backbone=DeiT-S, MACs (G)=3.3, Throughput (img/s)=2987.3, Params (M)=17.6, Speedup (x)=1.272026.07 | 80.1 | — | — | |
| FOOFModel=ResNet-101, Epoch=1002025.06 | 80 | — | — | |
| MACModel=ResNet-101, Epoch=1002025.06 | 79.8 | — | — | |
| DeiT-S (Dense)Backbone=DeiT-S, MACs (G)=4.6, Throughput (img/s)=2349.9, Params (M)=22.1, Speedup (x)=1.002026.07 | 79.8 | — | — | |
| MACModel=ResNet-50, Epoch=2002025.06 | 79.7 | — | — | |
| SGDModel=ResNet-101, Epoch=1002025.06 | 79.7 | — | — | |
| FOOFModel=ResNet-50, Epoch=2002025.06 | 79.7 | — | — | |
| HetDPT (8 layers)Backbone=DeiT-S, MACs (G)=3.0, Throughput (img/s)=3275.0, Params (M)=15.9, Speedup (x)=1.392026.07 | 79.7 | — | — | |
| SGDModel=ResNet-50, Epoch=2002025.06 | 79.6 | — | — | |
| KFACModel=ResNet-101, Epoch=1002025.06 | 79.6 | — | — | |
| EVAModel=ResNet-101, Epoch=1002025.06 | 79.6 | — | — | |
| NOSE (5 layers)Backbone=DeiT-S, MACs (G)=3.7, Throughput (img/s)=2875.3, Params (M)=19.5, Speedup (x)=1.222026.07 | 79.6 | — | — | |
| EVAModel=ResNet-50, Epoch=2002025.06 | 79.4 | — | — | |
| KFACModel=ResNet-50, Epoch=2002025.06 | 79.3 | — | — | |
| ADAMWModel=ResNet-50, Epoch=2002025.06 | 79.2 | — | — | |
| SGDModel=Swin-Tiny, Epoch=2002025.06 | 78.9 | — | — | |
| DW Prun. (0.3-12)Backbone=DeiT-S, MACs (G)=3.1, Throughput (img/s)=2989.8, Params (M)=15.0, Speedup (x)=1.272026.07 | 78.6 | — | — | |
| FOOFModel=ResNet-50, Epoch=1002025.06 | 78.4 | — | — | |
| KFACModel=ResNet-50, Epoch=1002025.06 | 78.2 | — | — | |
| SGDModel=ResNet-50, Epoch=1002025.06 | 78.1 | — | — | |
| MACModel=ResNet-50, Epoch=1002025.06 | 78 | — | — | |
| ADAMWModel=ResNet-101, Epoch=1002025.06 | 77.9 | — | — | |
| ADAMWModel=DeiT-Small, Epoch=2002025.06 | 77.9 | — | — | |
| MACModel=Swin-Tiny, Epoch=1002025.06 | 77.7 | — | — | |
| EVAModel=ResNet-50, Epoch=1002025.06 | 77.7 | — | — | |
| MACModel=DeiT-Small, Epoch=2002025.06 | 77.4 | — | — | |
| ADAMWModel=Swin-Tiny, Epoch=1002025.06 | 77.4 | — | — | |
| ADAMWModel=ResNet-50, Epoch=1002025.06 | 76.8 | — | — | |
| EVAModel=DeiT-Small, Epoch=2002025.06 | 76.6 | — | — | |
| SGDModel=Swin-Tiny, Epoch=1002025.06 | 76 | — | — | |
| SGDModel=DeiT-Small, Epoch=2002025.06 | 75.3 | — | — | |
| ADAMWModel=DeiT-Small, Epoch=1002025.06 | 73.7 | — | — | |
| FOOFModel=Swin-Tiny, Epoch=1002025.06 | 73.7 | — | — | |
| MACModel=DeiT-Small, Epoch=1002025.06 | 73.5 | — | — | |
| FOOFModel=Swin-Tiny, Epoch=2002025.06 | 72.6 | — | — | |
| KFACModel=DeiT-Small, Epoch=1002025.06 | 69.9 | — | — | |
| EVAModel=DeiT-Small, Epoch=1002025.06 | 69.7 | — | — | |
| SGDModel=DeiT-Small, Epoch=1002025.06 | 69.1 | — | — | |
| FOOFModel=DeiT-Small, Epoch=2002025.06 | 67.9 | — | — | |
| FOOFModel=DeiT-Small, Epoch=1002025.06 | 63.9 | — | — | |
| ADAMWSummary=All Models2025.06 | — | 78 | 2 | |
| EVASummary=All Models2025.06 | — | 58 | 5 | |
| FOOFSummary=All Models2025.06 | — | 74.7 | 4 | |
| KFACSummary=All Models2025.06 | — | 48.5 | 6 | |
| MACSummary=All Models2025.06 | — | 78.4 | 1 | |
| SGDSummary=All Models2025.06 | — | 77.3 | 3 |