Needle-in-a-haystack retrieval on RULER S-NIAH-1 ID
100Retrieval Success Rate (1K Context)Softmax
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SoftmaxPositional encoding=NAPE, Training context length=20482025.06 | 100 | 99.4 | |
| SSMaxPositional encoding=NAPE, Training context length=20482025.06 | 100 | 99.8 | |
| EntmaxPositional encoding=NAPE, Training context length=20482025.06 | 99.8 | 99.8 | |
| ASEntmaxPositional encoding=NAPE, Training context length=20482025.06 | 99.6 | 100 |