Needle-in-a-haystack retrieval on RULER S-NIAH-2 (ID)
100Retrieval Success Rate (1K)Softmax
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SoftmaxPositional encoding=NAPE, Training context length=20482025.06 | 100 | 100 | |
| EntmaxPositional encoding=NAPE, Training context length=20482025.06 | 99.6 | 99.4 | |
| SSMaxPositional encoding=NAPE, Training context length=20482025.06 | 99.4 | 99.2 | |
| ASEntmaxPositional encoding=NAPE, Training context length=20482025.06 | 99.4 | 99.4 |