Human Preference Alignment on MT-Bench
6.83SAGRAG-Pref
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RAG-PrefOffline Alignment=RTO2026.05 | 6.83 | — | — | |
| RAGOffline Alignment=RTO2026.05 | 6.75 | — | — | |
| RTO (Offline Baseline)Online Alignment=None2026.05 | 6.54 | — | — | |
| RAG-PrefOffline Alignment=PPO2026.05 | 6.49 | — | — | |
| RAGOffline Alignment=PPO2026.05 | 6.45 | — | — | |
| PPO (Offline Baseline)Online Alignment=None2026.05 | 6.22 | — | — | |
| RAG-PrefOffline Alignment=DPO2026.05 | 6.19 | — | — | |
| RAGOffline Alignment=DPO2026.05 | 6.12 | — | — | |
| RAG-PrefOffline Alignment=SFT2026.05 | 6.01 | — | — | |
| DPO (Offline Baseline)Online Alignment=None2026.05 | 6 | — | — | |
| RAGOffline Alignment=SFT2026.05 | 5.97 | — | — | |
| RAG-PrefOffline Alignment=SimPO2026.05 | 5.84 | — | — | |
| SimPO (Offline Baseline)Online Alignment=None2026.05 | 5.84 | — | — | |
| RAGOffline Alignment=SimPO2026.05 | 5.74 | — | — | |
| SFT (Offline Baseline)Online Alignment=None2026.05 | 5.74 | — | — | |
| OPADOffline Alignment=PPO2026.05 | 5.14 | — | — | |
| OPADOffline Alignment=RTO2026.05 | 4.58 | — | — | |
| OPADOffline Alignment=DPO2026.05 | 4.05 | — | — | |
| OPADOffline Alignment=SFT2026.05 | 3.58 | — | — | |
| OPADOffline Alignment=SimPO2026.05 | 3.16 | — | — | |
| BaseBackbone=Qwen3-1.7B-Base2026.06 | — | 5.09 | 4.41 | |
| BaseBackbone=Qwen3-4B-Base2026.06 | — | 5.56 | 4.83 | |
| DPOBackbone=Qwen3-1.7B-Base2026.06 | — | 6 | 4.81 | |
| DPOBackbone=Qwen3-4B-Base2026.06 | — | 6.79 | 5.74 | |
| IPOBackbone=Qwen3-1.7B-Base2026.06 | — | 6.56 | 5.1 | |
| IPOBackbone=Qwen3-4B-Base2026.06 | — | 7.43 | 6.14 | |
| KTOBackbone=Qwen3-1.7B-Base2026.06 | — | 6.92 | 5.32 | |
| KTOBackbone=Qwen3-4B-Base2026.06 | — | 8.22 | 6.93 | |
| RePOBackbone=Qwen3-1.7B-Base2026.06 | — | 6.88 | 5.43 | |
| RePOBackbone=Qwen3-4B-Base2026.06 | — | 8.09 | 6.78 | |
| RePO_detBackbone=Qwen3-1.7B-Base2026.06 | — | 6.89 | 5.16 | |
| RePO_detBackbone=Qwen3-4B-Base2026.06 | — | 8.18 | 6.97 | |
| RPOBackbone=Qwen3-1.7B-Base2026.06 | — | 5.98 | 5.13 | |
| RPOBackbone=Qwen3-4B-Base2026.06 | — | 7.05 | 6.13 | |
| TDPOBackbone=Qwen3-1.7B-Base2026.06 | — | 5.56 | 4.73 | |
| TDPOBackbone=Qwen3-4B-Base2026.06 | — | 6.24 | 5.38 |