Needle-in-a-haystack retrieval on RULER S-NIAH-2 OOD
83.2Success Rate (4K Context)ASEntmax
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ASEntmaxPositional encoding=NAPE, Training context length=20482025.06 | 83.2 | 25.4 | |
| EntmaxPositional encoding=NAPE, Training context length=20482025.06 | 64.8 | 7.2 | |
| SSMaxPositional encoding=NAPE, Training context length=20482025.06 | 64.4 | 14.8 | |
| SoftmaxPositional encoding=NAPE, Training context length=20482025.06 | 4.8 | 0 |