Long-context retrieval and tracking on RULER 32K
89.4S-N ScoreSWA+HSA
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SWA+HSAEnc=1, CLS=no, Bypass=yes2025.10 | 89.4 | 75.6 | 90.8 | 85.27 | |
| SWA+HSAEnc=1, CLS=yes, Bypass=yes2025.10 | 89.2 | 72.8 | 89.8 | 83.93 | |
| SWA+HSAEnc=2, CLS=yes, Bypass=yes2025.10 | 89.14 | 73.84 | 91.61 | 84.86 | |
| SWA+HSAEnc=2, CLS=no, Bypass=yes2025.10 | 88.15 | 72.36 | 92.2 | 84.24 | |
| SWA+HSAEnc=0, CLS=no, Bypass=yes2025.10 | 88.06 | 71.08 | 69.2 | 76.11 | |
| SWA+HSAEnc=2, CLS=yes, Bypass=no2025.10 | 87.46 | 68.11 | 92.89 | 82.82 | |
| LandmarkEnc=0, CLS=—, Bypass=—2025.10 | 83 | 15 | 20 | 39.33 | |
| SWA+HSAEnc=1, CLS=no, Bypass=no2025.10 | 81 | 66.2 | 87.4 | 78.2 | |
| SWA+HSAEnc=1, CLS=yes, Bypass=no2025.10 | 81 | 65.4 | 86.4 | 77.6 | |
| SWA+HSAEnc=0, CLS=no, Bypass=no2025.10 | 78.97 | 38.8 | 12.14 | 43.3 | |
| SWA+HSAEnc=2, CLS=no, Bypass=no2025.10 | 78.58 | 65.75 | 87.66 | 77.33 | |
| Stick-BreakEnc=0, CLS=—, Bypass=—2025.10 | 73.88 | 24.63 | 31.34 | 43.28 | |
| Mamba2Enc=0, CLS=—, Bypass=—2025.10 | 1.48 | 1.49 | 0.4 | 1.12 | |
| NSAEnc=0, CLS=—, Bypass=—2025.10 | 0 | 0 | 4.48 | 1.49 | |
| Llama-YarnEnc=0, CLS=—, Bypass=—2025.10 | 0 | 0 | 0 | 0 |