Question Answering on LongBench V1
39.8S-Doc QA AccuracyORPO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ORPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=SoLoPO2025.05 | 39.8 | 65.7 | 52.8 | 170 | |
| ORPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=short2025.05 | 39.37 | 63.8 | 51.6 | — | |
| ORPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=long2025.05 | 38.8 | 62.73 | 50.8 | 248 | |
| SimPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=SoLoPO2025.05 | 38.4 | 65.67 | 52 | 169 | |
| SoLo-SimPOBackbone=Qwen2.5-7B, Training Context=SoLo2025.05 | 38.1 | 58.6 | 48.4 | — | |
| SoLo-DPOBackbone=Qwen2.5-7B, Training Context=SoLo2025.05 | 38 | 57.6 | 47.8 | — | |
| DPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=short2025.05 | 37.83 | 65.33 | 51.6 | — | |
| 72B-InstructModel=Qwen2.5-Instruct-72B2025.05 | 37.8 | 61.1 | 49.4 | — | |
| DPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=SoLoPO2025.05 | 37.8 | 67.2 | 53.4 | 172 | |
| Qwen2.5-72B-InstructBackbone=Qwen2.5-72B2025.05 | 37.8 | 61.1 | 49.4 | — | |
| SoLo-ORPOBackbone=Qwen2.5-7B, Training Context=SoLo2025.05 | 37.6 | 61.4 | 49.5 | — | |
| SimPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=short2025.05 | 37.47 | 64.3 | 50.9 | — | |
| SimPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=long2025.05 | 36.77 | 66.1 | 51.4 | 248 | |
| DPOBase Model=Qwen2.5-Instruct-14B, Training Strategy=long2025.05 | 36.63 | 66.63 | 51.6 | 249 | |
| SFTBase Model=Qwen2.5-Instruct-14B, Training Strategy=short2025.05 | 36.2 | 59.13 | 47.7 | — | |
| DPOBackbone=Qwen2.5-7B, Training Context=long2025.05 | 35.7 | 58.2 | 46.9 | — | |
| ORPOBackbone=Llama3.1-8B, Training Context=long2025.05 | 35.4 | 55.4 | 45.4 | — | |
| SoLo-ORPOBackbone=Llama3.1-8B, Training Context=SoLo2025.05 | 35.2 | 57.3 | 46.3 | — | |
| SFTBackbone=Llama3.1-8B, Training Context=long2025.05 | 35 | 57.3 | 46.1 | — | |
| LongPO-Qwen2.5-7BImplementation=reimp, Backbone=Qwen2.5-7B2025.05 | 34.8 | 52.6 | 43.7 | — | |
| SFTBackbone=Qwen2.5-7B, Training Context=long2025.05 | 34.8 | 55.8 | 45.3 | — | |
| ORPOBackbone=Qwen2.5-7B, Training Context=long2025.05 | 34.8 | 55.8 | 45.3 | — | |
| SimPOBackbone=Qwen2.5-7B, Training Context=short2025.05 | 34.7 | 53.6 | 44.1 | — | |
| DPOBackbone=Qwen2.5-7B, Training Context=short2025.05 | 34.6 | 51.8 | 43.2 | — | |
| SFTBase Model=Qwen2.5-Instruct-14B, Training Strategy=long2025.05 | 34.53 | 61.07 | 47.8 | — | |
| 14B-InstructModel=Qwen2.5-Instruct-14B2025.05 | 34.4 | 52.07 | 43.3 | — | |
| SimPOBackbone=Qwen2.5-7B, Training Context=long2025.05 | 34.2 | 54.9 | 44.6 | — | |
| Qwen2.5-32B-InstructBackbone=Qwen2.5-32B2025.05 | 34.1 | 49.8 | 41.9 | — | |
| ORPOBackbone=Llama3.1-8B, Training Context=short2025.05 | 33.1 | 55.1 | 44.1 | — | |
| SFTBackbone=Llama3.1-8B, Training Context=short2025.05 | 33 | 56.2 | 44.6 | — | |
| InstructBackbone=Llama3.1-8B2025.05 | 30.3 | 49.3 | 39.8 | — | |
| InstructBackbone=Qwen2.5-7B2025.05 | 29.4 | 39.4 | 34.4 | — | |
| SFTBackbone=Qwen2.5-7B, Training Context=short2025.05 | 28.9 | 48.4 | 38.6 | — | |
| ORPOBackbone=Qwen2.5-7B, Training Context=short2025.05 | 28.9 | 48.4 | 38.6 | — | |
| Llama3.1-70B-InstructBackbone=Llama3.1-70B2025.05 | 28.5 | 64.1 | 46.3 | — | |
| LongPO-Qwen2.5-7BImplementation=pub, Backbone=Qwen2.5-7B2025.05 | 27.5 | 38.3 | 32.9 | — |