List your company
ResearchFormerly WizWand

Long-Context Retrieval on RULER 64K

97.33ScoreHiLS-Attn HoPE-Q-Cal
-3.893222.385948.66574.9441Apr 27, 2026May 8, 2026May 19, 2026May 30, 2026Jun 10, 2026Jun 21, 2026Jul 3, 2026
Updated 3mo ago

Evaluation results

MethodLinks
HiLS-Attn HoPE-Q-CalTuning Protocol=Full param. CPT, Positional Encoding=HoPE-Q-Cal2026.0797.33
Nemotron-3-Ultra 550B-A55B-Baseshots=0-shot2026.0695.3
Kimi-K2 Baseshots=0-shot2026.0693.79
DeepSeek-V3.2 Exp-Baseshots=0-shot2026.0693.3
Mistral-Large-3 675B-Base-2512shots=0-shot2026.0690.11
Hybrid-KSAsetting=Train-from-scratch2026.0469.95
Hybrid-GDNsetting=Train-from-scratch2026.0469.39
Fullsetting=Train-from-scratch2026.0465.32
Hybrid-SCAsetting=Train-from-scratch2026.0463.54
Hybrid-SWAsetting=Train-from-scratch2026.0463.03
KSAsetting=Train-from-scratch2026.0457.13
GLM-4.5 Baseshots=0-shot2026.0616.12
HiLS-Attn RoPE-Q-CalTuning Protocol=Full param. CPT, Positional Encoding=RoPE-Q-Cal2026.074.33
Olmo3 512swaTuning Protocol=Full param. CPT2026.071.33
LMK token tuningTuning Protocol=Freezing param.2026.071
HiLS-Attn NoPE-Q-CalTuning Protocol=Full param. CPT, Positional Encoding=NoPE-Q-Cal2026.070.33
Olmo3 BaseTuning Protocol=Freezing param.2026.070