Long-context retrieval and reasoning on RULER 4k context
100S1 ScoreTransformer
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Transformer2026.03 | 100 | 100 | 90 | 90 | 28 | 49 | 44 | 6.8 | 36 | 27 | 42 | |
| GDN2026.03 | 100 | 100 | 52 | 37 | 2 | 27 | 22 | 23 | 13 | 22 | 19 | |
| GDN-GSAKV cache ratio=50%2026.03 | 100 | 100 | 93 | 83 | 15 | 72 | 59 | 12 | 28 | 37 | 36 | |
| HAM Fixed τKV cache ratio=50%, Threshold type=Fixed2026.03 | 100 | 99 | 2 | 39 | 0.8 | 35 | 20 | 56 | 28 | 27 | 26 | |
| HAM Learned τKV cache ratio=50%, Threshold type=Learned2026.03 | 100 | 95 | 27 | 42 | 0.6 | 24 | 28 | 4.3 | 32 | 26 | 22 | |
| HAM Learned routerKV cache ratio=50%, Routing mechanism=Learned router2026.03 | 100 | 78 | 87 | 95 | 67 | 70 | 82 | 58 | 1 | 31 | 42 | |
| HAM Learned router + EDAKV cache ratio=50%, Routing mechanism=Learned router, Exponential decay averaging (EDA)=true2026.03 | 100 | 78 | 43 | 59 | 27 | 36 | 30 | 31 | 3.4 | 28 | 36 |