Image Classification on ImageNet (val) (Top-1 Accuracy and Error)
81.7Top-1 AccuracyFull Attention
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Full AttentionBackbone=T2T-ViT, Attention Replacement=Full Attention2021.10 | 81.7 | — | |
| ScatterbrainBackbone=T2T-ViT, Attention Replacement=Scatterbrain2021.10 | 80.7 | 5.3 | |
| PerformerBackbone=T2T-ViT, Attention Replacement=Performer2021.10 | 80.1 | 7.5 | |
| SMYRFBackbone=T2T-ViT, Attention Replacement=SMYRF2021.10 | 79.8 | 11.4 | |
| Baseline SMYRF + PerformerBackbone=T2T-ViT, Attention Replacement=SMYRF + Performer2021.10 | 79.7 | 12.6 |