Image Classification on ImageNet-1K (Top-1 Accuracy and Pruning Ratio)
75.9Top-1 AccuracyFull Attention
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Full AttentionBackbone=ViT-B2024.06 | 75.9 | 0 | |
| FibottentionBackbone=ViT-B2024.06 | 75.5 | 98.01 | |
| Top-k AttentionBackbone=ViT-B2024.06 | 73.4 | 98.48 | |
| LongformerBackbone=ViT-B2024.06 | 71.6 | 98.47 | |
| BigBirdBackbone=ViT-B2024.06 | 71.5 | 97.96 | |
| Efficient AttentionBackbone=ViT-B2024.06 | 70.1 | 97.98 | |
| Random AttentionBackbone=ViT-B2024.06 | 68.7 | 98.52 | |
| Sparse TransformerBackbone=ViT-B2024.06 | 68.7 | 98.47 | |
| LinformerBackbone=ViT-B2024.06 | 60.1 | 97.96 |