Language Modeling on Copy (test)
1Perplexity (PPL)Full Attention
Evaluation results
| Method | Links | |
|---|---|---|
| Full AttentionParameter Budget=1/8 of full2021.10 | 1 | |
| ScatterbrainParameter Budget=1/8 of full2021.10 | 2.58 | |
| ReformerParameter Budget=1/8 of full2021.10 | 6.8 | |
| PerformerParameter Budget=1/8 of full2021.10 | 49 |