LLM Alignment on Ultrafeedback
86Win RateGreedy SFT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Greedy SFTBase Model=Llama 3B2025.09 | 86 | 0.58 | |
| Greedy DPOBase Model=Llama 3B2025.09 | 86 | 0.68 | |
| Bo2Base Model=Llama 3B2025.09 | 85 | 0.85 | |
| Greedy SFTBase Model=Llama 8B2025.09 | 85 | 0.87 | |
| Bo2Base Model=Llama 8B2025.09 | 85 | 1.06 | |
| Greedy DPOBase Model=Llama 8B2025.09 | 84 | 0.98 | |
| EFTBase Model=Llama 3B2025.09 | 83 | 1.04 | |
| EFTBase Model=Llama 8B2025.09 | 81 | 1.27 | |
| Greedy SFTBase Model=Qwen 0.6B2025.09 | 80 | -0.88 | |
| Greedy SFTBase Model=Qwen 4B2025.09 | 80 | 0.22 | |
| Greedy DPOBase Model=Qwen 4B2025.09 | 79 | 0.29 | |
| Greedy DPOBase Model=Qwen 0.6B2025.09 | 78 | -0.69 | |
| Bo2Base Model=Qwen 0.6B2025.09 | 78 | -0.62 | |
| Bo2Base Model=Qwen 4B2025.09 | 77 | 0.47 | |
| EFTBase Model=Qwen 4B2025.09 | 73 | 0.58 | |
| EFTBase Model=Qwen 0.6B2025.09 | 67 | -0.19 | |
| Clean Data2026.06 | 27.15 | — | |
| DeMixrepair=true2026.06 | 26.04 | — | |
| DeMixrepair=false2026.06 | 26.02 | — | |
| DeMix-unifrepair=false2026.06 | 25.97 | — | |
| DeMix-unifrepair=true2026.06 | 25.84 | — | |
| DDA-repair2026.06 | 25.82 | — | |
| DDA-select2026.06 | 25.37 | — | |
| Error. Data2026.06 | 24.38 | — | |
| AADBase Model=Llama 3B2025.09 | — | 2.21 | |
| AADBase Model=Llama 8B2025.09 | — | 2.22 | |
| AADBase Model=Qwen 0.6B2025.09 | — | 0.34 | |
| AADBase Model=Qwen 4B2025.09 | — | 1.19 |