In-context retrieval on RULER 512K
99S-N ScoreHiLS-Attn-HoPE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HiLS-Attn-HoPEParameters=345M2026.07 | 99 | 91 | 66 | |
| HSA-UltralongParameters=345M2026.07 | 65 | 42 | 8 | |
| SWA-RoPEParameters=345M2026.07 | 0 | 1 | 8 |
| Method | Links | |||
|---|---|---|---|---|
| HiLS-Attn-HoPEParameters=345M2026.07 | 99 | 91 | 66 | |
| HSA-UltralongParameters=345M2026.07 | 65 | 42 | 8 | |
| SWA-RoPEParameters=345M2026.07 | 0 | 1 | 8 |