Sequential Preference Optimization on UltraFeedback
99.71Harmless RateMORA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MORABackbone=LLAMA-32026.05 | 99.71 | 76.64 | 72.53 | 82.96 | |
| MORABackbone=MISTRAL2026.05 | 99.23 | 81.98 | 69.36 | 83.52 | |
| OrthAlignBackbone=LLAMA-32026.05 | 87.3 | 71.57 | 66.58 | 75.15 | |
| KnotsBackbone=LLAMA-32026.05 | 82.3 | 63.73 | 61.78 | 69.27 | |
| RSDPOBackbone=LLAMA-32026.05 | 80.57 | 71.92 | 63.87 | 72.12 | |
| OrthAlignBackbone=MISTRAL2026.05 | 78 | 75.51 | 65.28 | 72.93 | |
| RSDPOBackbone=MISTRAL2026.05 | 71.87 | 73.24 | 63.25 | 69.45 | |
| TSV-MBackbone=MISTRAL2026.05 | 71.26 | 70.63 | 65.13 | 69 | |
| TSV-MBackbone=LLAMA-32026.05 | 68.65 | 66.75 | 63.51 | 66.3 | |
| SPOBackbone=MISTRAL2026.05 | 68.07 | 75.51 | 61.57 | 68.38 | |
| KnotsBackbone=MISTRAL2026.05 | 64.5 | 72.8 | 59.23 | 65.51 | |
| SPOBackbone=LLAMA-32026.05 | 58.42 | 71.08 | 64.22 | 64.57 | |
| MODPOBackbone=MISTRAL2026.05 | 58.07 | 73.41 | 59.95 | 63.81 | |
| MODPOBackbone=LLAMA-32026.05 | 56.46 | 71.42 | 65.79 | 64.55 | |
| SoupsBackbone=LLAMA-32026.05 | 56.15 | 60 | 64.59 | 60.24 | |
| DPO BaselineBackbone=LLAMA-32026.05 | 52.69 | 70.93 | 67.03 | 63.55 | |
| SFTBackbone=LLAMA-32026.05 | 46.73 | 50 | 53.41 | 50.04 | |
| SoupsBackbone=MISTRAL2026.05 | 34.23 | 71.92 | 61.05 | 55.73 | |
| DPO BaselineBackbone=MISTRAL2026.05 | 27.11 | 72.91 | 66.55 | 55.52 | |
| SFTBackbone=MISTRAL2026.05 | 20.19 | 26.83 | 43.03 | 30.01 |