In-context retrieval on RULER 128K
99S-N ScoreHiLS-Attn-HoPE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HiLS-Attn-HoPEParameters=345M2026.07 | 99 | 95 | 61 | |
| HSA-UltralongParameters=345M2026.07 | 80 | 60 | 21 | |
| Naive-BSA-HoPEParameters=345M2026.07 | 46 | 0 | 0 |
| Method | Links | |||
|---|---|---|---|---|
| HiLS-Attn-HoPEParameters=345M2026.07 | 99 | 95 | 61 | |
| HSA-UltralongParameters=345M2026.07 | 80 | 60 | 21 | |
| Naive-BSA-HoPEParameters=345M2026.07 | 46 | 0 | 0 |