Helpful Assistant Alignment on Helpful Assistant normalized rewards (test)
53Helpfulness Reward (r1)MOPO-Lag
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MOPO-LagModel=Zephyr-7b-beta2025.12 | 53 | 51 | |
| MOPO-LagModel=Zephyr-7b-beta2025.12 | 52 | 53 | |
| MOPO-LBModel=Zephyr-7b-beta2025.12 | 51 | 52 | |
| MOPO-LBModel=Mistral-7b-vo.2 (Instruct)2025.12 | 49 | 48 | |
| MOPO-LagModel=Mistral-7b-vo.2 (Instruct)2025.12 | 49 | 47 | |
| MOPO-LBModel=Zephyr-7b-beta2025.12 | 49 | 48 | |
| MOPO-LagModel=Llama-3.1-8B2025.12 | 48 | 46 | |
| PARMModel=Mistral-7b-vo.2 (Instruct)2025.12 | 48 | 47 | |
| PARMModel=Zephyr-7b-beta2025.12 | 48 | 50 | |
| MOPO-LBModel=Llama-3.1-8B2025.12 | 45 | 43 | |
| MOPO-LagModel=Mistral-7b-vo.2 (Instruct)2025.12 | 45 | 47 | |
| PARMModel=Zephyr-7b-beta2025.12 | 45 | 48 | |
| MOPO-LagModel=Llama-3.1-8B2025.12 | 45 | 53 | |
| RiCModel=Zephyr-7b-beta2025.12 | 45 | 47 | |
| RiCModel=Mistral-7b-vo.2 (Instruct)2025.12 | 44 | 41 | |
| MOPO-LBModel=Mistral-7b-vo.2 (Instruct)2025.12 | 44 | 45 | |
| MOPO-LagModel=OpenChat-v3.52025.12 | 43 | 41 | |
| RiCModel=Zephyr-7b-beta2025.12 | 43 | 46 | |
| RiCModel=Llama-3.1-8B2025.12 | 43 | 41 | |
| RiCModel=Mistral-7b-vo.2 (Instruct)2025.12 | 43 | 44 | |
| MOPO-LagModel=OpenChat-v3.52025.12 | 42 | 40 | |
| MOPO-LBModel=Llama-3.1-8B2025.12 | 42 | 50 | |
| MOPO-LBModel=OpenChat-v3.52025.12 | 41 | 39 | |
| RiCModel=Llama-3.1-8B2025.12 | 41 | 47 | |
| MODPOModel=Mistral-7b-vo.2 (Instruct)2025.12 | 41 | 39 | |
| PARMModel=Mistral-7b-vo.2 (Instruct)2025.12 | 41 | 46 | |
| RiCModel=OpenChat-v3.52025.12 | 40 | 33 | |
| PARMModel=Llama-3.1-8B2025.12 | 40 | 42 | |
| MOPO-LBModel=OpenChat-v3.52025.12 | 40 | 39 | |
| MODPOModel=Mistral-7b-vo.2 (Instruct)2025.12 | 40 | 42 | |
| MODPOModel=Zephyr-7b-beta2025.12 | 40 | 40 | |
| MODPOModel=OpenChat-v3.52025.12 | 39 | 37 | |
| PARMModel=OpenChat-v3.52025.12 | 38 | 37 | |
| DPO on D_JModel=Zephyr-7b-beta2025.12 | 38 | 41 | |
| MOPO-LagModel=phi-1.52025.12 | 37 | 36 | |
| RiCModel=OpenChat-v3.52025.12 | 37 | 40 | |
| MODPOModel=Llama-3.1-8B2025.12 | 37 | 45 | |
| PARMModel=OpenChat-v3.52025.12 | 36 | 39 | |
| DPO on D_JModel=OpenChat-v3.52025.12 | 36 | 35 | |
| DPO on D_JModel=Mistral-7b-vo.2 (Instruct)2025.12 | 36 | 39 | |
| MODPOModel=OpenChat-v3.52025.12 | 35 | 33 | |
| MODPOModel=Zephyr-7b-beta2025.12 | 35 | 33 | |
| PARMModel=Llama-3.1-8B2025.12 | 35 | 44 | |
| DPO on D_JModel=Llama-3.1-8B2025.12 | 35 | 40 | |
| DPO on D_JModel=Zephyr-7b-beta2025.12 | 34 | 38 | |
| MOPO-LBModel=phi-1.52025.12 | 34 | 35 | |
| PARMModel=phi-1.52025.12 | 33 | 32 | |
| MODPOModel=Llama-3.1-8B2025.12 | 32 | 33 | |
| DPO on D_JModel=Mistral-7b-vo.2 (Instruct)2025.12 | 32 | 30 | |
| DPO on D_JModel=Llama-3.1-8B2025.12 | 31 | 30 | |
| RiCModel=phi-1.52025.12 | 31 | 27 | |
| MOPO-LagModel=phi-1.52025.12 | 29 | 26 | |
| PARMModel=phi-1.52025.12 | 28 | 24 | |
| DPO on D_JModel=OpenChat-v3.52025.12 | 28 | 26 | |
| MOPO-LBModel=phi-1.52025.12 | 27 | 24 | |
| DPO on D_JModel=phi-1.52025.12 | 27 | 25 | |
| MODPOModel=phi-1.52025.12 | 25 | 24 | |
| RiCModel=phi-1.52025.12 | 23 | 21 | |
| DPO on D_JModel=phi-1.52025.12 | 22 | 19 | |
| MODPOModel=phi-1.52025.12 | 21 | 20 |