Long-context language modeling on HELMET
247Summarization ScoreQwen2.5-14B + RLVR
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen2.5-14B + RLVRBackbone=Qwen2.5-14B, Maximum input length=16K, Maximum output length=4K, Alignment=RLVR2025.09 | 247 | — | 420 | — | 46.7 | 25.5 | |
| Qwen2.5-14B + SPELLBackbone=Qwen2.5-14B, Maximum input length=16K, Maximum output length=4K, Alignment=SPELL2025.09 | 230 | — | 392 | — | 52.3 | 31.6 | |
| Qwen2.5-32B + SPELLBackbone=Qwen2.5-32B, Maximum input length=16K, Maximum output length=4K, Alignment=SPELL2025.09 | 212 | — | 564 | — | 61.4 | 35 | |
| Qwen2.5-32B + RLVRBackbone=Qwen2.5-32B, Maximum input length=16K, Maximum output length=4K, Alignment=RLVR2025.09 | 210 | — | 167 | — | 52.7 | 25.8 | |
| Qwen2.5-32BBackbone=Qwen2.5-32B, Maximum input length=16K, Maximum output length=4K2025.09 | 162 | — | 428 | — | 59 | 28.1 | |
| Qwen2.5-7B + RLVRBackbone=Qwen2.5-7B, Maximum input length=16K, Maximum output length=4K, Alignment=RLVR2025.09 | 143 | — | 26 | — | 49.3 | 18.5 | |
| Qwen2.5-7B + SPELLBackbone=Qwen2.5-7B, Maximum input length=16K, Maximum output length=4K, Alignment=SPELL2025.09 | 137 | — | 104 | — | 54.2 | 24.2 | |
| Qwen2.5-7BBackbone=Qwen2.5-7B, Maximum input length=16K, Maximum output length=4K2025.09 | 41 | — | 35 | — | 50 | 11.5 | |
| Qwen2.5-14BBackbone=Qwen2.5-14B, Maximum input length=16K, Maximum output length=4K2025.09 | 37 | — | 15 | — | 42.4 | 15.2 | |
| CoPEContext Length=32k, Backbone=Llama-3-8B2026.02 | 32.78 | 21.02 | 85.5 | 99 | 67.83 | 61.23 | |
| CoPEContext Length=256k, Backbone=Llama-3-8B, Scaling Technique=YaRN2026.02 | 32.37 | 19.06 | — | 34 | — | 28.48 | |
| CoPEContext Length=64k, Backbone=Llama-3-8B2026.02 | 30.88 | 15.07 | 86.4 | 97.88 | 63.17 | 58.68 | |
| CoPEContext Length=16k, Backbone=Llama-3-8B2026.02 | 30.81 | 16.89 | 83.7 | 98.88 | 67.72 | 59.6 | |
| CoPEContext Length=8k, Backbone=Llama-3-8B2026.02 | 29.76 | 13.1 | 79.4 | 99.63 | 68.67 | 58.11 | |
| RoPEContext Length=8k, Backbone=Llama-3-8B2026.02 | 29.18 | 6.46 | 74.6 | 99.75 | 68.38 | 55.74 | |
| RoPEContext Length=16k, Backbone=Llama-3-8B2026.02 | 28.46 | 8.39 | 80.2 | 99.13 | 67.44 | 56.86 | |
| CoPEContext Length=128k, Backbone=Llama-3-8B, Scaling Technique=YaRN2026.02 | 27.89 | 18.23 | 84.7 | 76 | 56.78 | 52.72 | |
| HardClipContext Length=16k, Backbone=Llama-3-8B2026.02 | 25.7 | 9.28 | 77 | 98.5 | 67.5 | 55.6 | |
| HardClipContext Length=8k, Backbone=Llama-3-8B2026.02 | 25.68 | 7.44 | 73.1 | 99.75 | 68.06 | 54.81 | |
| RoPEContext Length=32k, Backbone=Llama-3-8B2026.02 | 21.76 | 8.52 | 83.4 | 98.13 | 66.67 | 55.7 | |
| HardClipContext Length=32k, Backbone=Llama-3-8B2026.02 | 18.55 | 10.16 | 79.8 | 98.5 | 66.11 | 54.62 | |
| RoPEContext Length=64k, Backbone=Llama-3-8B2026.02 | 11.1 | 7.67 | 85.5 | 97.63 | 62.78 | 52.94 | |
| HardClipContext Length=128k, Backbone=Llama-3-8B, Scaling Technique=YaRN2026.02 | 9.33 | 9.31 | 77.3 | 82.13 | 62.05 | 48.02 | |
| RoPEContext Length=256k, Backbone=Llama-3-8B, Scaling Technique=YaRN2026.02 | 9.06 | 7.93 | — | 26.13 | — | 14.37 | |
| HardClipContext Length=256k, Backbone=Llama-3-8B, Scaling Technique=YaRN2026.02 | 8.6 | 9.24 | — | 36.86 | — | 18.23 | |
| HardClipContext Length=64k, Backbone=Llama-3-8B2026.02 | 6.93 | 10.31 | 82.2 | 94.38 | 59.72 | 50.71 | |
| RoPEContext Length=128k, Backbone=Llama-3-8B, Scaling Technique=YaRN2026.02 | 6.31 | 8.21 | 82.1 | 71.38 | 53.44 | 44.29 |