Sequence Prediction on WikiText-2 (average)
25.07Average PPLCTAT (Softmax-GL)
Evaluation Results
| Method | Links | |
|---|---|---|
| CTAT (Softmax-GL)Layer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L^2)2026.06 | 25.07 | |
| Transformer-GLLayer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L^2)2026.06 | 25.16 | |
| CTAT (Linear-GL)Layer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L)2026.06 | 25.47 | |
| CTAT (Linear-FFT)Layer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L log L)2026.06 | 25.47 | |
| TransformerLayer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L^2)2026.06 | 27.52 |