Multi-turn Chat Evaluation on MT-Bench
8.58MT-Bench ScoreVRM-PPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VRM-PPOBackbone=Qwen 3-8B2026.03 | 8.58 | — | — | |
| PPOBackbone=Qwen 3-8B2026.03 | 8.44 | — | — | |
| GSPO + GraphAEModel=Qwen2.5-7B-Instruct2026.06 | 8.01 | 8.6 | 7.43 | |
| VRM-PPOBackbone=Qwen2.5-7B2026.03 | 7.98 | — | — | |
| RLOO + GraphAEModel=Qwen2.5-7B-Instruct2026.06 | 7.98 | 8.6 | 7.2 | |
| KTOBackbone=Qwen 3-8B2026.03 | 7.97 | — | — | |
| DPOBackbone=Qwen 3-8B2026.03 | 7.86 | — | — | |
| GSPOModel=Qwen2.5-7B-Instruct2026.06 | 7.86 | 8.5 | 7.23 | |
| GRPO + GraphAEModel=Qwen2.5-7B-Instruct2026.06 | 7.85 | 8.6 | 7.1 | |
| GSPO + SmoothModel=Qwen2.5-7B-Instruct2026.06 | 7.85 | 8.48 | 7.21 | |
| GRPOModel=Qwen2.5-7B-Instruct2026.06 | 7.84 | 8.66 | 7.01 | |
| DPOBackbone=Qwen2.5-7B2026.03 | 7.83 | — | — | |
| GRPO + SmoothModel=Qwen2.5-7B-Instruct2026.06 | 7.82 | 8.61 | 7.02 | |
| WPOBackbone=Qwen2.5-7B2026.03 | 7.81 | — | — | |
| PPOBackbone=Qwen2.5-7B2026.03 | 7.81 | — | — | |
| RLOOModel=Qwen2.5-7B-Instruct2026.06 | 7.8 | 8.55 | 7.05 | |
| RLOO + SmoothModel=Qwen2.5-7B-Instruct2026.06 | 7.77 | 8.5 | 7.04 | |
| SELECTIVE DPOBackbone=Qwen2.5-7B2026.03 | 7.74 | — | — | |
| PPOModel=Qwen2.5-7B-Instruct2026.06 | 7.73 | 8.3 | 7.15 | |
| IPOBackbone=Qwen2.5-7B2026.03 | 7.64 | — | — | |
| KTOBackbone=Qwen2.5-7B2026.03 | 7.63 | — | — | |
| w/o RLModel=Qwen2.5-7B-Instruct2026.06 | 7.59 | 8.23 | 6.96 | |
| SIMPOBackbone=Qwen2.5-7B2026.03 | 7.58 | — | — | |
| GSPO + GraphAEModel=Llama-3-8B-Instruct2026.06 | 7.23 | 7.64 | 6.83 | |
| GRPO + GraphAEModel=Llama-3-8B-Instruct2026.06 | 7.21 | 7.68 | 6.74 | |
| RLOO + GraphAEModel=Llama-3-8B-Instruct2026.06 | 7.16 | 7.62 | 6.75 | |
| PPOModel=Llama-3-8B-Instruct2026.06 | 7.15 | 7.35 | 6.95 | |
| GRPOModel=Llama-3-8B-Instruct2026.06 | 7.13 | 7.64 | 6.63 | |
| GRPO + SmoothModel=Llama-3-8B-Instruct2026.06 | 7.11 | 7.6 | 6.62 | |
| RLOOModel=Llama-3-8B-Instruct2026.06 | 7.05 | 7.55 | 6.6 | |
| RLOO + SmoothModel=Llama-3-8B-Instruct2026.06 | 7.05 | 7.51 | 6.58 | |
| GSPOModel=Llama-3-8B-Instruct2026.06 | 7.01 | 7.49 | 6.53 | |
| GSPO + SmoothModel=Llama-3-8B-Instruct2026.06 | 6.99 | 7.46 | 6.52 | |
| w/o RLModel=Llama-3-8B-Instruct2026.06 | 6.62 | 7.19 | 6.05 | |
| IPOBackbone=Qwen 3-8B2026.03 | 6.55 | — | — | |
| WPOBackbone=Qwen 3-8B2026.03 | 6.38 | — | — | |
| SIMPOBackbone=Qwen 3-8B2026.03 | 6.24 | — | — | |
| SELECTIVE DPOBackbone=Qwen 3-8B2026.03 | 6.13 | — | — | |
| wICIBackbone=Llama3.1-8B, Instruction Tuning Dataset=WizardLM, Training Budget=10% slice2026.04 | 5.28 | — | — | |
| wICIBackbone=Llama3.1-8B, Instruction Tuning Dataset=Alpaca-GPT4, Training Budget=10% slice2026.04 | 4.88 | — | — | |
| wICIBackbone=Mistral-7B-v0.3, Instruction Tuning Dataset=WizardLM, Training Budget=10% slice2026.04 | 4.4 | — | — | |
| wICIBackbone=Mistral-7B-v0.3, Instruction Tuning Dataset=Alpaca-GPT4, Training Budget=10% slice2026.04 | 4.18 | — | — |