Multi-turn Instruction Following on MT-Bench High-Variance (Top 20%)
7.54Reward ScorecDPO (Best-of-K)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| cDPO (Best-of-K)Generator=Llama-3.1-8B-Instruct, Type=Training2026.03 | 7.54 | 8.39 | -9.24 | 312 | |
| cDPO + DARC-ϵGenerator=Llama-3.1-8B-Instruct, Type=Training2026.03 | 7.51 | 8.11 | -8.71 | 323 | |
| rDPO (Best-of-K)Generator=Llama-3.1-8B-Instruct, Type=Training2026.03 | 6.58 | 7.98 | -9.38 | 337 | |
| rDPO + DARC-ϵGenerator=Llama-3.1-8B-Instruct, Type=Training2026.03 | 6.41 | 7.16 | -7.91 | 316 | |
| DeALGenerator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.96 | 9.71 | -13.46 | 374 | |
| Base (Best-of-K)Generator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.79 | 9.91 | -14.03 | 382 | |
| rDPO (Best-of-K)Generator=Qwen2.5-7B-Instruct, Type=Training2026.03 | 5.79 | 5.34 | -4.89 | 304 | |
| DeALGenerator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 5.75 | 8.43 | -11.11 | 316 | |
| rDPO + DARC-ϵGenerator=Qwen2.5-7B-Instruct, Type=Training2026.03 | 5.56 | 5.03 | -4.5 | 301 | |
| cDPO (Best-of-K)Generator=Qwen2.5-7B-Instruct, Type=Training2026.03 | 5.52 | 5.67 | -5.82 | 317 | |
| DARC-ϵ(ours)Generator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.49 | 7 | -8.51 | 382 | |
| BoP(HedgeTune)Generator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.45 | 7.9 | -10.35 | 382 | |
| DARC-τ(ours)Generator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.43 | 7.9 | -10.37 | 382 | |
| cDPO + DARC-ϵGenerator=Qwen2.5-7B-Instruct, Type=Training2026.03 | 5.42 | 5.17 | -4.92 | 316 | |
| CautionGenerator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.41 | 7.42 | -9.43 | 382 | |
| DARC(ours)Generator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.32 | 7.33 | -9.34 | 382 | |
| RBoNGenerator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.28 | 7.3 | -9.32 | 372 | |
| MC-DropoutGenerator=Llama-3.1-8B-Instruct, Type=Inference2026.03 | 5.2 | 7.48 | -9.76 | 384 | |
| Base (Best-of-K)Generator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 5.12 | 6 | -7.12 | 308 | |
| DARC-ϵ(ours)Generator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 5.03 | 4.77 | -4.51 | 306 | |
| DARC(ours)Generator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 4.99 | 5.05 | -5.13 | 307 | |
| RBoNGenerator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 4.9 | 5.45 | -6 | 300 | |
| CautionGenerator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 4.8 | 5.39 | -5.98 | 307 | |
| DARC-τ(ours)Generator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 4.78 | 4.85 | -4.92 | 294 | |
| MC-DropoutGenerator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 4.75 | 5.25 | -5.25 | 306 | |
| BoP(HedgeTune)Generator=Qwen2.5-7B-Instruct, Type=Inference2026.03 | 4.67 | 5.12 | -5.57 | 307 |