Long-context Retrieval and Reasoning on RULER 32K context
46.98RULER ScoreSWA-128-NoPE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SWA-128-NoPEScale=S5 (0.66B), Training Tokens=≈ 100B2026.06 | 46.98 | 70.42 | |
| FullScale=S5 (0.66B), Training Tokens=≈ 100B2026.06 | 43.9 | 62.61 | |
| SWA-128Scale=S5 (0.66B), Training Tokens=≈ 100B2026.06 | 41.86 | 60.17 |