Question Answering on RULER
94.29Score (8K Context)Llama-3.1-8B-Instruct
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Llama-3.1-8B-InstructSparse Configuration=FULL2026.05 | 94.29 | 90.65 | 85.93 | 84.68 | 73.52 | 85.81 | — | — | — | |
| FluxionModel=Llama-3.1-8B-Instruct2026.05 | 93.4 | 90.64 | 85.03 | 83.97 | 72.09 | 85.03 | -0.78 | — | — | |
| Qwen2.5-7B-InstructSparse Configuration=FULL2026.05 | 92.52 | 92.34 | 88.42 | 71.34 | 25.74 | 74.07 | — | — | — | |
| Llama-3.1-8B-Instructblk=16, bgt=0.052026.05 | 91.52 | 89.78 | 85.66 | 84.09 | 71.72 | 84.55 | — | — | — | |
| FluxionModel=Qwen2.5-7B-Instruct2026.05 | 90.46 | 90.39 | 87.28 | 69.57 | 24.94 | 72.53 | -1.54 | — | — | |
| Qwen2.5-7B-Instructblk=16, bgt=0.052026.05 | 89.72 | 90.24 | 88.21 | 69.23 | 24.63 | 72.41 | — | — | — | |
| Llama-3.1-8B-Instructblk=32, bgt=0.052026.05 | 88.35 | 86.58 | 83.26 | 81.93 | 70.63 | 82.15 | — | — | — | |
| Llama-3.1-8B-Instructblk=16, bgt=0.022026.05 | 86.79 | 85.75 | 83.04 | 81.54 | 70.18 | 81.46 | — | — | — | |
| Qwen2.5-7B-Instructblk=32, bgt=0.052026.05 | 86.23 | 87.35 | 84.99 | 69.4 | 25.26 | 70.65 | — | — | — | |
| Qwen2.5-7B-Instructblk=16, bgt=0.022026.05 | 83.1 | 85.65 | 84.14 | 67.75 | 23.98 | 68.92 | — | — | — | |
| Llama-3.1-8B-Instructblk=32, bgt=0.022026.05 | 80.55 | 81.53 | 78.5 | 76.71 | 67.82 | 77.02 | — | — | — | |
| Qwen2.5-7B-Instructblk=32, bgt=0.022026.05 | 80.38 | 81.76 | 80.35 | 66.62 | 24.49 | 66.72 | — | — | — | |
| DPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=SoLoPO2025.05 | 70.53 | 68.52 | 62.03 | — | — | 68.51 | — | 72.95 | 172 | |
| DPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=long2025.05 | 69.49 | 67.69 | 59.83 | — | — | 67.6 | — | 73.37 | 249 | |
| Llama3.1-70B-InstructBackbone=Llama3.1-70B2025.05 | 68.8 | 52.4 | 23.2 | — | — | 54.1 | — | 72.1 | — | |
| SoLo-ORPOBackbone=Qwen2.5-7B, Training Context=SoLo2025.05 | 68.3 | 64 | 57.3 | — | — | 65.1 | — | 70.8 | — | |
| SimPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=SoLoPO2025.05 | 67.85 | 66.52 | 60.47 | — | — | 66.66 | — | 71.79 | 169 | |
| ORPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=SoLoPO2025.05 | 67.85 | 65.1 | 60.78 | — | — | 66.54 | — | 72.45 | 170 | |
| SimPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=long2025.05 | 66.24 | 63.27 | 56.91 | — | — | 64.19 | — | 70.33 | 248 | |
| SoLo-SimPOBackbone=Qwen2.5-7B, Training Context=SoLo2025.05 | 66 | 62.7 | 57.8 | — | — | 63.9 | — | 69.2 | — | |
| SimPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=short2025.05 | 65.87 | 63.21 | 55.69 | — | — | 64.05 | — | 71.43 | — | |
| DPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=short2025.05 | 65.79 | 63.12 | 55.97 | — | — | 64.05 | — | 71.32 | — | |
| SoLo-DPOBackbone=Qwen2.5-7B, Training Context=SoLo2025.05 | 64.5 | 62.7 | 57.7 | — | — | 62.8 | — | 66.4 | — | |
| 72B-InstructModel=Qwen2.5-Instruct-72B2025.05 | 64.4 | 61.2 | 55 | — | — | 61.6 | — | 65.7 | — | |
| Qwen2.5-72B-InstructBackbone=Qwen2.5-72B2025.05 | 64.4 | 61.2 | 55 | — | — | 61.6 | — | 65.7 | — | |
| DPOBackbone=Qwen2.5-7B, Training Context=long2025.05 | 64.2 | 60.6 | 53.2 | — | — | 62.2 | — | 71 | — | |
| ORPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=long2025.05 | 64.19 | 62.16 | 52.83 | — | — | 61.97 | — | 68.69 | 248 | |
| SimPOBackbone=Qwen2.5-7B, Training Context=long2025.05 | 64.1 | 49.1 | 47.2 | — | — | 57.5 | — | 69.8 | — | |
| ORPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=short2025.05 | 63.85 | 61.47 | 52.96 | — | — | 61.73 | — | 68.63 | — | |
| DPOBackbone=Qwen2.5-7B, Training Context=short2025.05 | 63.3 | 45.3 | 46.9 | — | — | 56.6 | — | 70.9 | — | |
| SimPOBackbone=Qwen2.5-7B, Training Context=short2025.05 | 62.5 | 42.1 | 48.8 | — | — | 56 | — | 70.8 | — | |
| SoLo-ORPOBackbone=Llama3.1-8B, Training Context=SoLo2025.05 | 62.5 | 60.1 | 58.2 | — | — | 61.3 | — | 64.4 | — | |
| ORPOBackbone=Qwen2.5-7B, Training Context=short2025.05 | 62.1 | 50.8 | 46.6 | — | — | 57.1 | — | 69.1 | — | |
| SFTBase Model=Qwen2.5-Instruct-14B, Training Strategy=long2025.05 | 61.44 | 52.56 | 43.83 | — | — | 55.89 | — | 65.72 | — | |
| SFTBackbone=Qwen2.5-7B, Training Context=long2025.05 | 61.4 | 58.4 | 52.4 | — | — | 59.5 | — | 65.9 | — | |
| SFTBackbone=Llama3.1-8B, Training Context=short2025.05 | 61 | 58.5 | 52.2 | — | — | 59.2 | — | 65 | — | |
| ORPOBackbone=Llama3.1-8B, Training Context=long2025.05 | 60.1 | 58.9 | 53.2 | — | — | 58.9 | — | 63.2 | — | |
| ORPOBackbone=Qwen2.5-7B, Training Context=long2025.05 | 59.9 | 50 | 45.6 | — | — | 55.1 | — | 64.9 | — | |
| SFTBase Model=Qwen2.5-Instruct-14B, Training Strategy=short2025.05 | 59.46 | 55.67 | 46.31 | — | — | 56.48 | — | 64.47 | — | |
| ORPOBackbone=Llama3.1-8B, Training Context=short2025.05 | 59.3 | 59.7 | 50.4 | — | — | 58.4 | — | 64 | — | |
| SFTBackbone=Llama3.1-8B, Training Context=long2025.05 | 59 | 57.5 | 53.5 | — | — | 58.4 | — | 63.7 | — | |
| SFTBackbone=Qwen2.5-7B, Training Context=short2025.05 | 56.7 | 42.3 | 31.8 | — | — | 48.7 | — | 63.8 | — | |
| LongPO-Qwen2.5-7BImplementation=reimp, Backbone=Qwen2.5-7B2025.05 | 54.4 | 48.9 | 43.1 | — | — | 52.2 | — | 62.4 | — | |
| 14B-InstructModel=Qwen2.5-Instruct-14B2025.05 | 52.27 | 49.69 | 43.86 | — | — | 50.6 | — | 56.6 | — | |
| Qwen2.5-32B-InstructBackbone=Qwen2.5-32B2025.05 | 52.1 | 46 | 43.9 | — | — | 50.1 | — | 58.4 | — | |
| LongPO-Qwen2.5-7BImplementation=pub, Backbone=Qwen2.5-7B2025.05 | 51.9 | 40.6 | 36.3 | — | — | 45.9 | — | 54.7 | — | |
| InstructBackbone=Qwen2.5-7B2025.05 | 50.1 | 37.6 | 34.6 | — | — | 44 | — | 53.9 | — | |
| InstructBackbone=Llama3.1-8B2025.05 | 49.2 | 42.9 | 35.6 | — | — | 46.5 | — | 58.3 | — |