Retrieval on Needle-in-a-Haystack Induction
98AccuracySoftmax + Titans memory
Evaluation results
| Method | Links | |
|---|---|---|
| Softmax + Titans memoryDistance=2,048, Multiplier=1x2026.06 | 98 | |
| Softmax + Titans memoryDistance=4,096, Multiplier=2x2026.06 | 98 | |
| TitansDistance=16,384, Multiplier=8x2026.06 | 98 | |
| Polar AttentionDistance=2,048, Multiplier=1x2026.06 | 96 | |
| TitansDistance=32,768, Multiplier=16x2026.06 | 96 | |
| TitansDistance=4,096, Multiplier=2x2026.06 | 95 | |
| Softmax + Titans memoryDistance=8,192, Multiplier=4x2026.06 | 94 | |
| TitansDistance=8,192, Multiplier=4x2026.06 | 93 | |
| TitansDistance=65,536, Multiplier=32x2026.06 | 93 | |
| TitansDistance=2,048, Multiplier=1x2026.06 | 91 | |
| Softmax + Titans memoryDistance=16,384, Multiplier=8x2026.06 | 85 | |
| Softmax + Titans memoryDistance=32,768, Multiplier=16x2026.06 | 48 | |
| Polar AttentionDistance=4,096, Multiplier=2x2026.06 | 40 | |
| Softmax + Titans memoryDistance=65,536, Multiplier=32x2026.06 | 16 | |
| Polar AttentionDistance=8,192, Multiplier=4x2026.06 | 15 | |
| Polar AttentionDistance=16,384, Multiplier=8x2026.06 | 8 | |
| Polar AttentionDistance=32,768, Multiplier=16x2026.06 | 3 | |
| Polar AttentionDistance=65,536, Multiplier=32x2026.06 | 0 |