Autoregressive Language Modeling on WikiText-103 (first 10M tokens)
90.5Perplexity (PPL)TF-GPT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TF-GPTInference Cache=O(N)2026.04 | 90.5 | — | |
| MIPT+CacheK=64, Inference Cache=O(64)2026.04 | 92.1 | 1.8 | |
| MIPT+CacheK=16, Inference Cache=O(16)2026.04 | 96.3 | 6.4 | |
| MIPT+CacheK=8, Inference Cache=O(8)2026.04 | 98.1 | 8.4 | |
| MIPT-LMInference Cache=O(1)2026.04 | 102.2 | 12.9 |