Image Classification on ImageNet100 (test)
86.74Top-1 AccDeiT-S
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeiT-SKeep rate r=1.0 (No pruning), Backbone=DeiT-S2026.04 | 86.74 | — | |
| Ours (Rényi, α = 10.0)Keep rate r=0.7, Backbone=DeiT-S2026.04 | 86.72 | — | |
| EViTKeep rate r=0.7, Backbone=DeiT-S2026.04 | 86.46 | — | |
| Ours (Rényi, α = 2.0)Keep rate r=0.8, Backbone=DeiT-S2026.04 | 86.46 | — | |
| Ours (Shannon)Keep rate r=0.9, Backbone=DeiT-S2026.04 | 86.4 | — | |
| Ours (Rényi, α = 2.0)Keep rate r=0.9, Backbone=DeiT-S2026.04 | 86.34 | — | |
| Ours (Rényi, α = 5.0)Keep rate r=0.7, Backbone=DeiT-S2026.04 | 86.3 | — | |
| Ours (Rényi, α = 10.0)Keep rate r=0.9, Backbone=DeiT-S2026.04 | 86.26 | — | |
| Ours (Rényi, α = 5.0)Keep rate r=0.8, Backbone=DeiT-S2026.04 | 86.2 | — | |
| Ours (Shannon)Keep rate r=0.7, Backbone=DeiT-S2026.04 | 86.14 | — | |
| EViTKeep rate r=0.9, Backbone=DeiT-S2026.04 | 86.06 | — | |
| EViTKeep rate r=0.8, Backbone=DeiT-S2026.04 | 86.04 | — | |
| Ours (Rényi, α = 10.0)Keep rate r=0.8, Backbone=DeiT-S2026.04 | 86.02 | — | |
| Ours (Rényi, α = 5.0)Keep rate r=0.9, Backbone=DeiT-S2026.04 | 85.88 | — | |
| Ours (Rényi, α = 5.0)Keep rate r=0.6, Backbone=DeiT-S2026.04 | 85.78 | — | |
| Ours (Shannon)Keep rate r=0.8, Backbone=DeiT-S2026.04 | 85.74 | — | |
| Ours (Rényi, α = 10.0)Keep rate r=0.6, Backbone=DeiT-S2026.04 | 85.72 | — | |
| EViTKeep rate r=0.6, Backbone=DeiT-S2026.04 | 85.68 | — | |
| Ours (Rényi, α = 10.0)Keep rate r=0.5, Backbone=DeiT-S2026.04 | 85.68 | — | |
| Ours (Shannon)Keep rate r=0.6, Backbone=DeiT-S2026.04 | 85.66 | — | |
| Ours (Rényi, α = 2.0)Keep rate r=0.7, Backbone=DeiT-S2026.04 | 85.6 | — | |
| Ours (Shannon)Keep rate r=0.5, Backbone=DeiT-S2026.04 | 85.54 | — | |
| Ours (Rényi, α = 2.0)Keep rate r=0.6, Backbone=DeiT-S2026.04 | 85.5 | — | |
| Ours (Rényi, α = 5.0)Keep rate r=0.5, Backbone=DeiT-S2026.04 | 85.44 | — | |
| DualPrompt-1Memory=02023.05 | 85.4 | — | |
| Ours (Rényi, α = 2.0)Keep rate r=0.5, Backbone=DeiT-S2026.04 | 85.36 | — | |
| EViTKeep rate r=0.5, Backbone=DeiT-S2026.04 | 85.32 | — | |
| Ours (Rényi, α = 5.0)Keep rate r=0.4, Backbone=DeiT-S2026.04 | 85.18 | — | |
| Ours (Rényi, α = 10.0)Keep rate r=0.4, Backbone=DeiT-S2026.04 | 84.82 | — | |
| Ours (Rényi, α = 2.0)Keep rate r=0.4, Backbone=DeiT-S2026.04 | 84.78 | — | |
| Ours (Shannon)Keep rate r=0.4, Backbone=DeiT-S2026.04 | 84.7 | — | |
| EViTKeep rate r=0.4, Backbone=DeiT-S2026.04 | 84.5 | — | |
| Ours (Rényi, α = 2.0)Keep rate r=0.3, Backbone=DeiT-S2026.04 | 84.3 | — | |
| Ours (Rényi, α = 10.0)Keep rate r=0.3, Backbone=DeiT-S2026.04 | 84.2 | — | |
| Ours (Shannon)Keep rate r=0.3, Backbone=DeiT-S2026.04 | 84.16 | — | |
| Ours (Rényi, α = 5.0)Keep rate r=0.3, Backbone=DeiT-S2026.04 | 84 | — | |
| EViTKeep rate r=0.3, Backbone=DeiT-S2026.04 | 83.92 | — | |
| AttriCLIPMemory=02023.05 | 83.3 | — | |
| SwAV + UOTASSL Algorithm=SwAV, UOTA Integration=True, Backbone=ResNet-18, Linear evaluation=True, Pre-train epochs=2002021.12 | 82.2 | 96.1 | |
| DualPrompt-2Memory=02023.05 | 81.9 | — | |
| MoCo-v2* + UOTASSL Algorithm=MoCo-v2*, UOTA Integration=True, Backbone=ResNet-18, Linear evaluation=True, Pre-train epochs=200, Multi-crop augmentation=True2021.12 | 81.4 | 95.7 | |
| DeiT-tiny + ContraNormNumber of Layers (#L)=24, Scale factor (s)=0.22023.03 | 81.28 | — | |
| SwAVSSL Algorithm=SwAV, UOTA Integration=False, Backbone=ResNet-18, Linear evaluation=True, Pre-train epochs=2002021.12 | 81.1 | 95.9 | |
| DeiT-tiny + ContraNormNumber of Layers (#L)=16, Scale factor (s)=0.22023.03 | 80.44 | — | |
| DCL+IP-IRMEvaluation Protocol=Linear2021.10 | 80.38 | 95.32 | |
| MobileNetInitialization=ImageNet-1K pretrained weights, FLOPs (B)=2.22026.03 | 80.36 | 96.5 | |
| HCL+IP-IRMEvaluation Protocol=Linear2021.10 | 80.3 | 95.38 | |
| MoCo-v2*SSL Algorithm=MoCo-v2*, UOTA Integration=False, Backbone=ResNet-18, Linear evaluation=True, Pre-train epochs=200, Multi-crop augmentation=True2021.12 | 80.3 | 95.5 | |
| SimCLR+IP-IRMEvaluation Protocol=Linear2021.10 | 79.52 | 94.76 | |
| DeiT-tiny + ContraNormNumber of Layers (#L)=12, Scale factor (s)=0.22023.03 | 79.34 | — | |
| CoOp-1Memory=10002023.05 | 79.3 | — | |
| CoOp-2Memory=10002023.05 | 79.3 | — | |
| ARI-1Memory=20002023.05 | 79.3 | — | |
| HCLEvaluation Protocol=Linear2021.10 | 79.15 | 94.72 | |
| ARI-2Memory=20002023.05 | 77.9 | — | |
| DCLEvaluation Protocol=Linear2021.10 | 77.32 | 94.74 | |
| DeiT-tinyNumber of Layers (#L)=24, Scale factor (s)=0.22023.03 | 76.76 | — | |
| SimCLREvaluation Protocol=Linear2021.10 | 76.58 | 94.22 | |
| DeiT-tinyNumber of Layers (#L)=12, Scale factor (s)=0.22023.03 | 76.58 | — | |
| BYOL + UOTASSL Algorithm=BYOL, UOTA Integration=True, Backbone=ResNet-18, Linear evaluation=True, Pre-train epochs=2002021.12 | 76.2 | 93.7 | |
| Saccader-MobileInitialization=ImageNet-1K pretrained weights, FLOPs (B)=37.272026.03 | 75.9 | 94.5 | |
| Continual-CLIPMemory=02023.05 | 75.4 | — | |
| DeiT-tinyNumber of Layers (#L)=16, Scale factor (s)=0.22023.03 | 75.34 | — | |
| BYOLSSL Algorithm=BYOL, UOTA Integration=False, Backbone=ResNet-18, Linear evaluation=True, Pre-train epochs=2002021.12 | 75.3 | 93.4 | |
| MoCo-v2 + UOTASSL Algorithm=MoCo-v2, UOTA Integration=True, Backbone=ResNet-18, Linear evaluation=True, Pre-train epochs=2002021.12 | 74 | 92.4 | |
| MoCo-v2SSL Algorithm=MoCo-v2, UOTA Integration=False, Backbone=ResNet-18, Linear evaluation=True, Pre-train epochs=2002021.12 | 73 | 92 | |
| EVA-MobileInitialization=ImageNet-1K pretrained weights, FLOPs (B)=8.872026.03 | 71.92 | 91.92 | |
| HCL+IP-IRMEvaluation Protocol=k-NN2021.10 | 71.78 | 91.94 | |
| DCL+IP-IRMEvaluation Protocol=k-NN2021.10 | 69.62 | 91.62 | |
| HCLEvaluation Protocol=k-NN2021.10 | 69.24 | 90.66 | |
| SimCLR+IP-IRMEvaluation Protocol=k-NN2021.10 | 69.08 | 90.96 | |
| EVAFLOPs (B)=6.62026.03 | 65.86 | 80.24 | |
| DCLEvaluation Protocol=k-NN2021.10 | 65.12 | 89.86 | |
| SimCLREvaluation Protocol=k-NN2021.10 | 64.64 | 89.36 | |
| iCaRL-1Memory=20002023.05 | 59.5 | — | |
| iCaRL-2Memory=20002023.05 | 58.7 | — | |
| MobileNetInitialization=scratch, FLOPs (B)=2.22026.03 | 47.64 | 76.12 | |
| EVA-MobileInitialization=scratch, FLOPs (B)=8.872026.03 | 42.62 | 68.24 | |
| AiT-MediumSize (M)=45.92023.09 | 36.72 | — | |
| ViT-BaseSize (M)=85.72023.09 | 34.62 | — | |
| AiT-SmallSize (M)=15.82023.09 | 33.84 | — | |
| ViT-MediumSize (M)=42.72023.09 | 31.72 | — | |
| ViT-SmallSize (M)=14.92023.09 | 28.16 | — | |
| DRAMFLOPs (B)=1.062026.03 | 21.44 | 47.3 | |
| DRAM-MobileInitialization=ImageNet-1K pretrained weights, FLOPs (B)=8.832026.03 | 19 | 43.12 | |
| MRAMFLOPs (B)=0.352026.03 | 12.88 | 34.76 | |
| RAMFLOPs (B)=0.352026.03 | 11.26 | 32.78 |