Open-ended Evaluation on BQD
8.93p+ ScoreDPO
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DPOBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 8.93 | 8.3 | |
| PFTBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 8.69 | 8.29 | |
| CPOBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 8.65 | 8.2 | |
| SFTBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 8.63 | 8.18 | |
| SFTBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 8.5 | 8.26 | |
| DPOBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 8.45 | 7.96 | |
| CPOBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 8.4 | 8.05 | |
| CAABackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 8.38 | 8.15 | |
| SFTBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 8.37 | 8.16 | |
| CPOBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 8.35 | 8.22 | |
| DPOBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 8.16 | 8.28 | |
| CAABackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 8.03 | 8.18 | |
| BaseBackbone=LLAMA-3.1-8B-INSTRUCT, Training Protocol=Base, Preference Setting=-2026.04 | 7.77 | 8.23 | |
| PFTBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 6 | 7.97 | |
| SFTBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 5.53 | 8.18 | |
| PFTBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 5.49 | 8.39 | |
| CPOBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 5.48 | 8.12 | |
| CAABackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 5.3 | 7.86 | |
| SFTBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 5.29 | 8.27 | |
| CPOBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 5.15 | 8.22 | |
| BaseBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Base, Preference Setting=-2026.04 | 5.12 | 8.15 | |
| CAABackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 5.1 | 7.92 | |
| CPOBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 5.08 | 8.3 | |
| SFTBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 5.04 | 8.36 | |
| DPOBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 4.98 | 8.28 | |
| DPOBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 4.93 | 8.45 | |
| DPOBackbone=QWEN2.5-3B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 4.92 | 7.93 | |
| SFTBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 4.9 | 7.57 | |
| CAABackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 4.82 | 7.55 | |
| DPOBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 4.8 | 7.93 | |
| CPOBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 4.78 | 7.62 | |
| SFTBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 4.75 | 7.28 | |
| CPOBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 4.7 | 7.35 | |
| BaseBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Base, Preference Setting=-2026.04 | 4.63 | 7.61 | |
| DPOBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p+2026.04 | 4.62 | 7.84 | |
| CPOBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 4.6 | 7.55 | |
| CAABackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 4.56 | 7.68 | |
| SFTBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Single Preference, Preference Setting=p-2026.04 | 4.52 | 7.47 | |
| DPOBackbone=QWEN2.5-7B-INSTRUCT, Training Protocol=Pair Preference, Preference Setting=-2026.04 | 4.49 | 7.94 |