Long-context retrieval and tracking on RULER 128K
88.94S-N ScoreSWA+HSA
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SWA+HSAEnc=2, CLS=yes, Bypass=yes2025.10 | 88.94 | 76.21 | 91.81 | 85.65 | |
| SWA+HSAEnc=1, CLS=yes, Bypass=yes2025.10 | 88.2 | 74 | 87.8 | 83.33 | |
| SWA+HSAEnc=2, CLS=no, Bypass=yes2025.10 | 86.97 | 71.17 | 92.99 | 83.71 | |
| SWA+HSAEnc=2, CLS=yes, Bypass=no2025.10 | 85.69 | 67.32 | 93.48 | 82.16 | |
| SWA+HSAEnc=1, CLS=no, Bypass=yes2025.10 | 84.4 | 73.4 | 90.2 | 82.67 | |
| SWA+HSAEnc=0, CLS=no, Bypass=yes2025.10 | 84.21 | 69 | 65.75 | 72.99 | |
| SWA+HSAEnc=0, CLS=no, Bypass=no2025.10 | 77.49 | 35.14 | 7.8 | 40.14 | |
| SWA+HSAEnc=1, CLS=no, Bypass=no2025.10 | 77.2 | 62.6 | 86.6 | 75.47 | |
| SWA+HSAEnc=1, CLS=yes, Bypass=no2025.10 | 75.6 | 62.2 | 84.8 | 74.2 | |
| SWA+HSAEnc=2, CLS=no, Bypass=no2025.10 | 74.83 | 64.76 | 87.27 | 75.62 | |
| Stick-BreakEnc=0, CLS=—, Bypass=—2025.10 | 33.33 | 18.18 | 30.3 | 27.27 | |
| Mamba2Enc=0, CLS=—, Bypass=—2025.10 | 0 | 0 | 0 | 0 | |
| NSAEnc=0, CLS=—, Bypass=—2025.10 | 0 | 0 | 0 | 0 | |
| Llama-YarnEnc=0, CLS=—, Bypass=—2025.10 | 0 | 0 | 0 | 0 | |
| LandmarkEnc=0, CLS=—, Bypass=—2025.10 | 0 | 0 | 0 | 0 |