Document Perplexity on FinePDFs
1.96PPLTitans
Evaluation Results
| Method | Links | |
|---|---|---|
| TitansDistance=65,536, Multiplier=32x2026.06 | 1.96 | |
| TitansDistance=32,768, Multiplier=16x2026.06 | 2.01 | |
| TitansDistance=16,384, Multiplier=8x2026.06 | 2.09 | |
| TitansDistance=8,192, Multiplier=4x2026.06 | 2.22 | |
| Softmax + Titans memoryDistance=16,384, Multiplier=8x2026.06 | 2.29 | |
| Softmax + Titans memoryDistance=32,768, Multiplier=16x2026.06 | 2.29 | |
| Softmax + Titans memoryDistance=65,536, Multiplier=32x2026.06 | 2.34 | |
| Softmax + Titans memoryDistance=8,192, Multiplier=4x2026.06 | 2.39 | |
| TitansDistance=4,096, Multiplier=2x2026.06 | 2.45 | |
| Softmax + Titans memoryDistance=4,096, Multiplier=2x2026.06 | 2.52 | |
| Softmax + Titans memoryDistance=2,048, Multiplier=1x2026.06 | 2.66 | |
| TitansDistance=2,048, Multiplier=1x2026.06 | 2.7 |