Multi-turn Conversation Rating (1-10) on MT-Bench
8.7Conversation Rating (1-10)GRPO (RM)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRPO (RM)#Data=10K2026.01 | 8.7 | — | |
| GRPO (RLVRR)#Data=10K2026.01 | 8.7 | — | |
| DPO#Data=10K2026.01 | 8.6 | — | |
| SFT#Data=10K2026.01 | 8.5 | — | |
| SFT#Data=100K2026.01 | 8.5 | — | |
| Instruct#Data=n/a2026.01 | 8.4 | — | |
| GRPO (GRM)#Data=10K2026.01 | 8.4 | — | |
| GRPO (RLPR)#Data=10K2026.01 | 8.3 | — | |
| GRPO (BLEU)#Data=10K2026.01 | 8.2 | — | |
| vs. Qwen3-8BTraining Steps=3002026.02 | 8.15 | — | |
| Elo-EvolveTraining Steps=3002026.02 | 8.04 | — | |
| Elo-EvolveTraining Steps=1002026.02 | 8.03 | — | |
| vs. Qwen2.5-14BTraining Steps=3002026.02 | 7.99 | — | |
| vs. Qwen2.5-14BTraining Steps=5002026.02 | 7.99 | — | |
| vs. Qwen2.5-14BTraining Steps=1002026.02 | 7.98 | — | |
| DNO (replicated)Training Steps=5002026.02 | 7.97 | — | |
| vs. Qwen2.5-32BTraining Steps=3002026.02 | 7.96 | — | |
| DNO (replicated)Training Steps=1002026.02 | 7.95 | — | |
| DNO (replicated)Training Steps=3002026.02 | 7.92 | — | |
| Point GRPOTraining Steps=3002026.02 | 7.91 | — | |
| vs. Qwen2.5-32BTraining Steps=5002026.02 | 7.89 | — | |
| vs. Qwen3-8BTraining Steps=5002026.02 | 7.86 | — | |
| Qwen2.5-7B (base model)Training Steps=0 (Base)2026.02 | 7.84 | — | |
| Elo-EvolveTraining Steps=5002026.02 | 7.82 | — | |
| Point GRPOTraining Steps=1002026.02 | 7.81 | — | |
| vs. Qwen3-8BTraining Steps=1002026.02 | 7.81 | — | |
| Point GRPOTraining Steps=5002026.02 | 7.79 | — | |
| vs. Qwen2.5-32BTraining Steps=1002026.02 | 7.79 | — | |
| RevFFNModel Backbone=Qwen1.5-MoE-A2.7B, Fine-tuning type=Full-Parameter2025.12 | 7.65 | — | |
| GRPO (Random)#Data=10K2026.01 | 7.6 | — | |
| SFT + CheckpointingModel Backbone=Qwen1.5-MoE-A2.7B, Fine-tuning type=Full-Parameter2025.12 | 7.52 | — | |
| LOMOModel Backbone=Qwen1.5-MoE-A2.7B, Fine-tuning type=Full-Parameter2025.12 | 7.5 | — | |
| GaLoreModel Backbone=Qwen1.5-MoE-A2.7B, Fine-tuning type=Full-Parameter2025.12 | 7.46 | — | |
| DoRAModel Backbone=Qwen1.5-MoE-A2.7B, Fine-tuning type=PEFT2025.12 | 7.25 | — | |
| LoRAModel Backbone=Qwen1.5-MoE-A2.7B, Fine-tuning type=PEFT2025.12 | 7.18 | — | |
| (IA)³Model Backbone=Qwen1.5-MoE-A2.7B, Fine-tuning type=PEFT2025.12 | 7.15 | — | |
| ADPOBase Model=Llama3-8B2026.02 | 7.1 | — | |
| DPOBase Model=Llama3-8B2026.02 | 7 | — | |
| SimPOBase Model=Llama3-8B2026.02 | 7 | — | |
| SFTBase Model=Llama3-8B2026.02 | 6.9 | — | |
| Base ModelModel Backbone=Qwen1.5-MoE-A2.7B2025.12 | 6.25 | — | |
| CPOBackbone=Mistral-Instruct (7B)2025.06 | — | 6.3 | |
| CPOBackbone=Llama3-Instruct (8B)2025.06 | — | 7 | |
| CPOBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 7.2 | |
| DPOBackbone=Mistral-Instruct (7B)2025.06 | — | 6.3 | |
| DPOBackbone=Llama3-Instruct (8B)2025.06 | — | 7 | |
| DPOBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 7 | |
| DPO (50%)Backbone=Mistral-Instruct (7B)2025.06 | — | 6.3 | |
| DPO (50%)Backbone=Llama3-Instruct (8B)2025.06 | — | 6.9 | |
| DPO (50%)Backbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 6.9 | |
| DPO+SamSBackbone=Mistral-Instruct (7B)2025.06 | — | 6.7 | |
| DPO+SamSBackbone=Llama3-Instruct (8B)2025.06 | — | 7.1 | |
| DPO+SamSBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 7.3 | |
| IPOBackbone=Mistral-Instruct (7B)2025.06 | — | 6.4 | |
| IPOBackbone=Llama3-Instruct (8B)2025.06 | — | 7 | |
| IPOBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 7.2 | |
| KTOBackbone=Mistral-Instruct (7B)2025.06 | — | 6.4 | |
| KTOBackbone=Llama3-Instruct (8B)2025.06 | — | 6.9 | |
| KTOBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 7.2 | |
| ORPOBackbone=Mistral-Instruct (7B)2025.06 | — | 6.4 | |
| ORPOBackbone=Llama3-Instruct (8B)2025.06 | — | 6.8 | |
| ORPOBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 7.2 | |
| R-DPOBackbone=Mistral-Instruct (7B)2025.06 | — | 6.2 | |
| R-DPOBackbone=Llama3-Instruct (8B)2025.06 | — | 7 | |
| R-DPOBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 7 | |
| RRHFBackbone=Mistral-Instruct (7B)2025.06 | — | 6.5 | |
| RRHFBackbone=Llama3-Instruct (8B)2025.06 | — | 6.7 | |
| RRHFBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 7.1 | |
| SFTBackbone=Mistral-Instruct (7B)2025.06 | — | 6.2 | |
| SFTBackbone=Llama3-Instruct (8B)2025.06 | — | 6.9 | |
| SFTBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 6.9 | |
| SLiC-HFBackbone=Mistral-Instruct (7B)2025.06 | — | 6.5 | |
| SLiC-HFBackbone=Llama3-Instruct (8B)2025.06 | — | 6.8 | |
| SLiC-HFBackbone=Llama3-Instruct v0.2 (8B)2025.06 | — | 6.9 |