Long-context understanding on LongBench English
19.63AccuracyYaRN
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| YaRNModel Size=Llama-3B, Sequence Length=16k2026.02 | 19.63 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HalfRoPEModel Size=Llama-3B, Sequence Length=4k2026.02 | 19.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| YaRNModel Size=Llama-3B, Sequence Length=8k2026.02 | 19.29 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPEModel Size=Llama-3B, Sequence Length=4k2026.02 | 18.62 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPE-ID (scaling)Model Size=Llama-3B, Sequence Length=16k2026.02 | 17.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPE-ID (scaling)Model Size=Llama-3B, Sequence Length=8k2026.02 | 17.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPE-ID (scaling)Model Size=Llama-3B, Sequence Length=4k2026.02 | 15.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| YaRNModel Size=Llama-3B, Sequence Length=4k2026.02 | 15.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPE-ID (scaling)Model Size=Llama-1B, Sequence Length=4k2026.02 | 15.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPE-ID (scaling)Model Size=Llama-1B, Sequence Length=8k2026.02 | 15.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPE-ID (scaling)Model Size=Llama-1B, Sequence Length=16k2026.02 | 15.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HalfRoPEModel Size=Llama-1B, Sequence Length=4k2026.02 | 15.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| YaRNModel Size=Llama-1B, Sequence Length=4k2026.02 | 14.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPEModel Size=Llama-1B, Sequence Length=4k2026.02 | 14.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| YaRNModel Size=Llama-1B, Sequence Length=8k2026.02 | 14.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| High frequencyModel Size=Llama-3B, Sequence Length=4k2026.02 | 14.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| YaRNModel Size=Llama-1B, Sequence Length=16k2026.02 | 14.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| High frequencyModel Size=Llama-3B, Sequence Length=8k2026.02 | 13.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| High frequencyModel Size=Llama-3B, Sequence Length=16k2026.02 | 13.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| High frequencyModel Size=Llama-1B, Sequence Length=4k2026.02 | 11.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| High frequencyModel Size=Llama-1B, Sequence Length=8k2026.02 | 11.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPEModel Size=Llama-3B, Sequence Length=8k2026.02 | 11.36 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| High frequencyModel Size=Llama-1B, Sequence Length=16k2026.02 | 11.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HalfRoPEModel Size=Llama-3B, Sequence Length=8k2026.02 | 10.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HalfRoPEModel Size=Llama-3B, Sequence Length=16k2026.02 | 10.62 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPEModel Size=Llama-3B, Sequence Length=16k2026.02 | 10.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HalfRoPEModel Size=Llama-1B, Sequence Length=16k2026.02 | 8.86 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPEModel Size=Llama-1B, Sequence Length=16k2026.02 | 8.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HalfRoPEModel Size=Llama-1B, Sequence Length=8k2026.02 | 8.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RoPEModel Size=Llama-1B, Sequence Length=8k2026.02 | 8.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CISRetrieval complexity (rho_hat)=0.1772026.02 | — | 26.47 | 8.47 | 27.54 | 21.24 | 32.51 | 50.99 | 79.57 | 62.5 | 20.53 | 17.9 | 26.63 | 55.35 | 2.69 | 40.96 | 8.5 | 36.64 | 32.4 | |
| CISRetrieval complexity (rho_hat)=0.0832026.02 | — | 25.98 | 6.98 | 28.64 | 21.18 | 30.78 | 50.03 | 81.46 | 63.5 | 20.93 | 17.05 | 26.39 | 54.43 | 3.12 | 40.52 | 7 | 36.54 | 32.15 | |
| CIS*Retrieval complexity (rho_hat)=0.1642026.02 | — | 26.18 | 8.74 | 26.72 | 20.87 | 30.05 | 50.65 | 81.04 | 64.5 | 20.98 | 17.65 | 26.07 | 54.89 | 2.38 | 39.04 | 6 | 37.29 | 32.06 | |
| CIS*Retrieval complexity (rho_hat)=0.0852026.02 | — | 26.4 | 8.68 | 27.6 | 20.79 | 30.7 | 50.88 | 79.76 | 62.5 | 20.92 | 16.3 | 26.39 | 55.1 | 3.34 | 39.68 | 9.5 | 37.06 | 32.22 | |
| CPERetrieval complexity (rho_hat)=0.1742026.02 | — | 26.11 | 8.69 | 27.39 | 20.94 | 29.06 | 51.64 | 79.78 | 65 | 20.78 | 16.55 | 25.63 | 55.62 | 3.27 | 40.11 | 6 | 37.47 | 32.11 | |
| CPERetrieval complexity (rho_hat)=0.0792026.02 | — | 26.02 | 7.5 | 27.1 | 21.63 | 31.15 | 51.12 | 81.35 | 62 | 20.66 | 17.96 | 25.78 | 54.22 | 1.93 | 40.19 | 6 | 37.96 | 32.02 | |
| DS2026.02 | — | 26.3 | 8.72 | 27.44 | 20.98 | 32.01 | 48.44 | 83.23 | 61.05 | 20.75 | 17.54 | 27.04 | 54.87 | 3.15 | 41.3 | 6 | 35.83 | 32.11 | |
| H2O2026.02 | — | 24.88 | 6.24 | 27.58 | 17.32 | 31.38 | 51.81 | 81.9 | 62.5 | 20.74 | 17.01 | 23.45 | 56.55 | 1.87 | 40.18 | 5 | 33.64 | 31.38 | |
| HShareRetrieval complexity (rho_hat)=0.2812026.02 | — | 26.05 | 8.42 | 27.32 | 20.12 | 31.46 | 50.79 | 83.91 | 60.5 | 20.59 | 17.31 | 25.85 | 56.23 | 2.3 | 39.77 | 6 | 34.08 | 31.97 | |
| HShareRetrieval complexity (rho_hat)=0.1252026.02 | — | 25.22 | 8.09 | 29.13 | 19.6 | 31.43 | 51.63 | 82.16 | 58.5 | 20.74 | 17.31 | 25.85 | 56.11 | 2.3 | 40.02 | 6 | 32.37 | 31.49 | |
| Original2026.02 | — | 26.22 | 8.65 | 27.72 | 21.88 | 31.18 | 52.14 | 83.09 | 64.5 | 20.91 | 18.83 | 26.55 | 58.26 | 2.77 | 41.01 | 6.5 | 36.15 | 32.9 | |
| Quest2026.02 | — | 26.49 | 4.62 | 21.94 | 16.96 | 29.42 | 50.24 | 82.42 | 64 | 21.36 | 18.21 | 25.46 | 56.74 | 2.33 | 40.91 | 6.5 | 31.99 | 31.35 |