Long-context retrieval and tracking on RULER 4K
99.54S-N ScoreMamba2
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Mamba2Enc=0, CLS=—, Bypass=—2025.10 | 99.54 | 15.96 | 80.8 | 65.43 | |
| Llama-YarnEnc=0, CLS=—, Bypass=—2025.10 | 92.39 | 69.11 | 93.51 | 85 | |
| SWA+HSAEnc=2, CLS=no, Bypass=yes2025.10 | 91.71 | 72.56 | 93.68 | 85.98 | |
| SWA+HSAEnc=2, CLS=yes, Bypass=yes2025.10 | 91.51 | 78.78 | 93.48 | 87.92 | |
| SWA+HSAEnc=1, CLS=yes, Bypass=yes2025.10 | 91.2 | 75.2 | 88.8 | 85.07 | |
| SWA+HSAEnc=0, CLS=no, Bypass=yes2025.10 | 90.33 | 71.77 | 72.26 | 78.12 | |
| SWA+HSAEnc=1, CLS=no, Bypass=yes2025.10 | 89.2 | 77 | 93.8 | 86.67 | |
| SWA+HSAEnc=2, CLS=yes, Bypass=no2025.10 | 86.28 | 71.96 | 93.98 | 84.07 | |
| SWA+HSAEnc=0, CLS=no, Bypass=no2025.10 | 84.4 | 61.3 | 37.02 | 60.91 | |
| LandmarkEnc=0, CLS=—, Bypass=—2025.10 | 83 | 37 | 40 | 53.33 | |
| SWA+HSAEnc=1, CLS=no, Bypass=no2025.10 | 81.2 | 68.4 | 87.8 | 79.13 | |
| SWA+HSAEnc=1, CLS=yes, Bypass=no2025.10 | 79.8 | 70.4 | 90 | 80.07 | |
| SWA+HSAEnc=2, CLS=no, Bypass=no2025.10 | 77.39 | 67.52 | 89.24 | 78.05 | |
| Stick-BreakEnc=0, CLS=—, Bypass=—2025.10 | 76.72 | 22.17 | 37.2 | 45.36 | |
| NSAEnc=0, CLS=—, Bypass=—2025.10 | 22.82 | 3.15 | 12.43 | 12.8 |