Text Classification on AMAZON
76.3AMAZON ScorecosFormer
Evaluation results
| Method | Links | ||
|---|---|---|---|
| cosFormerMode=Fine-tuning, Pre-training=WikiText-103, Iterations=50K, Sequence length=512, Backbone=RoBERTa2022.02 | 76.3 | 85.25 | |
| Vanilla TransformerMode=Fine-tuning, Pre-training=WikiText-103, Iterations=50K, Sequence length=512, Backbone=RoBERTa2022.02 | 75.79 | 85.7 | |
| LongformerMode=Fine-tuning, Pre-training=WikiText-103, Iterations=50K, Sequence length=512, Backbone=RoBERTa2022.02 | 73.34 | 83.17 | |
| Linear Trans.Mode=Fine-tuning, Pre-training=WikiText-103, Iterations=50K, Sequence length=512, Backbone=RoBERTa2022.02 | 72.5 | 78 | |
| RFAMode=Fine-tuning, Pre-training=WikiText-103, Iterations=50K, Sequence length=512, Backbone=RoBERTa2022.02 | 68.15 | 71.3 | |
| PerformerMode=Fine-tuning, Pre-training=WikiText-103, Iterations=50K, Sequence length=512, Backbone=RoBERTa2022.02 | 64.84 | 56.28 | |
| ReformerMode=Fine-tuning, Pre-training=WikiText-103, Iterations=50K, Sequence length=512, Backbone=RoBERTa2022.02 | 64.28 | 52.77 |