Long-context Modeling on LongBench 16K length
50.5LCCMDN
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MDNNumber of Parameters=1.3B, Sequence Length=16K, Evaluation Framework=lm-evaluation-harness2026.05 | 50.5 | 39.13 | 10.24 | 17.2 | 18.85 | 2.63 | 6.27 | 12.21 | 5.81 | 7.26 | 3.63 | 51 | 42.13 | 15.65 | 20.18 | |
| CombaNumber of Parameters=1.3B, Sequence Length=16K, Evaluation Framework=lm-evaluation-harness2026.05 | 44.74 | 36.6 | 9.6 | 16.52 | 16.74 | 2.39 | 6.2 | 13.13 | 6.87 | 7.82 | 3.12 | 48 | 35.95 | 5.79 | 18.11 | |
| GDNNumber of Parameters=1.3B, Sequence Length=16K, Evaluation Framework=lm-evaluation-harness2026.05 | 42.94 | 30.7 | 7.81 | 15.98 | 16.82 | 2.46 | 5.68 | 13.29 | 5.89 | 9.2 | 3.28 | 55 | 34.59 | 26.27 | 19.28 | |
| TransformerNumber of Parameters=1.3B, Sequence Length=16K, Evaluation Framework=lm-evaluation-harness2026.05 | 38.8 | 11.94 | 5.49 | 7.38 | 17.04 | 0.39 | 3.84 | 7.55 | 0.87 | 3.03 | 0.13 | 9 | 8.44 | 3.04 | 8.35 | |
| Mamba2Number of Parameters=1.3B, Sequence Length=16K, Evaluation Framework=lm-evaluation-harness2026.05 | 38.51 | 30.69 | 5.44 | 16.6 | 14.73 | 2.18 | 5.23 | 13.28 | 5.68 | 7.03 | 2.69 | 32.5 | 31.02 | 4.46 | 15 | |
| KDANumber of Parameters=1.3B, Sequence Length=16K, Evaluation Framework=lm-evaluation-harness2026.05 | 37.99 | 33.39 | 10.02 | 15.59 | 16.95 | 2.4 | 5.14 | 12.13 | 6.58 | 8.32 | 3.01 | 37 | 46.87 | 25.3 | 18.62 |