Long-Context Retrieval on RULER 128K
94.67ScoreHiLS-Attn HoPE-Q-Cal
Evaluation results
| Method | Links | |
|---|---|---|
| HiLS-Attn HoPE-Q-CalTuning Protocol=Full param. CPT, Positional Encoding=HoPE-Q-Cal2026.07 | 94.67 | |
| Nemotron-3-Ultra 550B-A55B-Baseshots=0-shot2026.06 | 92.49 | |
| DeepSeek-V3.2 Exp-Baseshots=0-shot2026.06 | 91.88 | |
| Kimi-K2 Baseshots=0-shot2026.06 | 88.61 | |
| Hybrid-KSASetting=CPT setting2026.04 | 71.67 | |
| Hybrid-LinearSetting=CPT setting2026.04 | 67.98 | |
| KSASetting=CPT setting2026.04 | 66.81 | |
| Hybrid-SWASetting=CPT setting2026.04 | 66.27 | |
| Full AttentionSetting=CPT setting2026.04 | 65.86 | |
| Hybrid-KSAsetting=Train-from-scratch2026.04 | 65.35 | |
| Hybrid-SCASetting=CPT setting2026.04 | 60.94 | |
| Hybrid-GDNsetting=Train-from-scratch2026.04 | 59.87 | |
| Hybrid-SCAsetting=Train-from-scratch2026.04 | 58.01 | |
| Hybrid-SWAsetting=Train-from-scratch2026.04 | 56.64 | |
| Mistral-Large-3 675B-Base-2512shots=0-shot2026.06 | 55.77 | |
| Fullsetting=Train-from-scratch2026.04 | 48.75 | |
| KSAsetting=Train-from-scratch2026.04 | 39.29 | |
| HiLS-Attn RoPE-Q-CalTuning Protocol=Full param. CPT, Positional Encoding=RoPE-Q-Cal2026.07 | 1 | |
| LMK token tuningTuning Protocol=Freezing param.2026.07 | 0.67 | |
| GLM-4.5 Baseshots=0-shot2026.06 | 0 | |
| Olmo3 BaseTuning Protocol=Freezing param.2026.07 | 0 | |
| Olmo3 512swaTuning Protocol=Full param. CPT2026.07 | 0 | |
| HiLS-Attn NoPE-Q-CalTuning Protocol=Full param. CPT, Positional Encoding=NoPE-Q-Cal2026.07 | 0 |