Image Classification on Tiny-ImageNet (Top-1 Accuracy, Pruning Ratio)
79.1Top-1 AccuracyFibottention
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FibottentionBackbone=ViT-B2024.06 | 79.1 | 98.01 | |
| Full AttentionBackbone=ViT-B2024.06 | 75.2 | 0 | |
| LongformerBackbone=ViT-B2024.06 | 74.3 | 98.47 | |
| Efficient AttentionBackbone=ViT-B2024.06 | 73.7 | 97.98 | |
| BigBirdBackbone=ViT-B2024.06 | 73.4 | 97.96 | |
| Top-k AttentionBackbone=ViT-B2024.06 | 72.9 | 98.48 | |
| Sparse TransformerBackbone=ViT-B2024.06 | 70.3 | 98.47 | |
| Random AttentionBackbone=ViT-B2024.06 | 69.4 | 98.52 | |
| LinformerBackbone=ViT-B2024.06 | 62.8 | 97.96 |