Long-context Retrieval and Reasoning on RULER 32K context length
100S1TRM
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TRMContext Length=32K2026.06 | 100 | 98.8 | 83.4 | 57 | 38.8 | 0.8 | 45.6 | 42.6 | 21.2 | 20.2 | 1.8 | 43.8 | 46.2 | |
| YOCO (Dense)Context Length=32K, Attention=Dense2026.06 | 100 | 90.2 | 74.8 | 53.2 | 84 | 43.6 | 27 | 29 | 30.6 | 30.6 | 4.6 | 60.3 | 52.3 | |
| YOCO (CLSA)Context Length=32K, Attention=Cross-Layer Sparse Attention2026.06 | 100 | 93.6 | 83.2 | 58.4 | 88.8 | 38 | 31.6 | 29.8 | 29.2 | 29.2 | 5.1 | 50.2 | 53.1 |