List your company
ResearchFormerly WizWand

Sequence Prediction on WikiText-2 (average)

25.07Average PPLCTAT (Softmax-GL)
24.97225.633526.29526.9565Jun 7, 2026
Updated 4mo ago

Evaluation results

MethodLinks
CTAT (Softmax-GL)Layer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L^2)2026.0625.07
Transformer-GLLayer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L^2)2026.0625.16
CTAT (Linear-GL)Layer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L)2026.0625.47
CTAT (Linear-FFT)Layer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L log L)2026.0625.47
TransformerLayer count=8, Sequence length=512, d=128, Number of parameters=28.10M, Theoretical complexity=O(L^2)2026.0627.52