Image Classification on CIFAR-100 (Accuracy and Pruning Efficiency)
70.7Top-1 AccuracyFibottention
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FibottentionBackbone=ViT-B2024.06 | 70.7 | 98.01 | |
| LongformerBackbone=ViT-B2024.06 | 64.7 | 98.47 | |
| BigBirdBackbone=ViT-B2024.06 | 63.4 | 97.96 | |
| Efficient AttentionBackbone=ViT-B2024.06 | 62.6 | 97.98 | |
| Full AttentionBackbone=ViT-B2024.06 | 59.4 | 0 | |
| Sparse TransformerBackbone=ViT-B2024.06 | 58.2 | 98.47 | |
| Top-k AttentionBackbone=ViT-B2024.06 | 57.1 | 98.48 | |
| Random AttentionBackbone=ViT-B2024.06 | 56.5 | 98.52 | |
| LinformerBackbone=ViT-B2024.06 | 48.7 | 97.96 |