Needle-In-A-Haystack Retrieval on RULER (test)
2,800Multi-Query Success RateDroPE transformer
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DroPE transformermode=Zero-shot, context_length=2x training context2025.12 | 2,800 | 4,160 | 2,330 | |
| RoPE transformer + RoPE-NTKmode=Zero-shot, context_length=2x training context, extension_strategy=NTK-RoPE2025.12 | 2,110 | 1,940 | 1,650 | |
| RoPE transformer + YaRNmode=Zero-shot, context_length=2x training context, extension_strategy=YaRN2025.12 | 1,780 | 50 | 1,460 | |
| NoPE transformermode=Zero-shot, context_length=2x training context2025.12 | 920 | 3,620 | 2,140 | |
| ALiBi transformermode=Zero-shot, context_length=2x training context2025.12 | 520 | 0 | 110 | |
| RNoPE-SWA transformermode=Zero-shot, context_length=2x training context2025.12 | 520 | 2,560 | 2,060 | |
| RoPE transformermode=Zero-shot, context_length=2x training context2025.12 | 0 | 0 | 0 | |
| RoPE transformer + PImode=Zero-shot, context_length=2x training context, extension_strategy=Position Interpolation (PI)2025.12 | 0 | 0 | 0 |