Sequential Preference Optimization on HelpSteer2
99.86Harmless RateMORA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MORABackbone=LLAMA-32026.05 | 99.86 | 72.54 | 71.01 | 81.13 | |
| OrthAlignBackbone=LLAMA-32026.05 | 91.34 | 68.83 | 67.69 | 75.95 | |
| MORABackbone=MISTRAL2026.05 | 90 | 82.11 | 66.01 | 79.37 | |
| OrthAlignBackbone=MISTRAL2026.05 | 88.12 | 67.08 | 65.34 | 73.51 | |
| KnotsBackbone=LLAMA-32026.05 | 87.11 | 59.62 | 63.66 | 70.13 | |
| SPOBackbone=MISTRAL2026.05 | 84.03 | 65.21 | 56.25 | 68.49 | |
| TSV-MBackbone=LLAMA-32026.05 | 78.07 | 62.12 | 62.36 | 67.51 | |
| RSDPOBackbone=LLAMA-32026.05 | 75.57 | 70.8 | 63.4 | 69.92 | |
| KnotsBackbone=MISTRAL2026.05 | 75.28 | 61.66 | 64.4 | 67.11 | |
| TSV-MBackbone=MISTRAL2026.05 | 73.3 | 64.17 | 65.28 | 67.58 | |
| MODPOBackbone=MISTRAL2026.05 | 71.36 | 61.55 | 59.32 | 64.07 | |
| MODPOBackbone=LLAMA-32026.05 | 70.96 | 69.44 | 62.08 | 67.49 | |
| RSDPOBackbone=MISTRAL2026.05 | 66.32 | 68.95 | 64.35 | 66.54 | |
| SPOBackbone=LLAMA-32026.05 | 66.15 | 68.24 | 62.01 | 65.46 | |
| SoupsBackbone=LLAMA-32026.05 | 56.92 | 61.18 | 58.93 | 59.01 | |
| SoupsBackbone=MISTRAL2026.05 | 54.03 | 59.75 | 59.92 | 57.9 | |
| DPO BaselineBackbone=LLAMA-32026.05 | 51.92 | 72.91 | 66.5 | 63.78 | |
| SFTBackbone=LLAMA-32026.05 | 46.73 | 50 | 53.41 | 50.04 | |
| DPO BaselineBackbone=MISTRAL2026.05 | 39.23 | 73.16 | 66.01 | 59.46 | |
| SFTBackbone=MISTRAL2026.05 | 20.19 | 26.83 | 43.03 | 30.01 |