Safety Alignment on Salad Bench
0.68MDShaPO-T
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ShaPO-TBackbone=Qwen2.5-7B2026.02 | 0.68 | 0.07 | |
| ShaPO-RBackbone=Qwen2.5-7B2026.02 | 0.98 | 0.06 | |
| wDPOBackbone=Qwen-2.5-7B2026.03 | 1.04 | 0.1 | |
| ShaPO-RBackbone=LLaMA-3-8B2026.02 | 1.13 | 0.08 | |
| ShaPO-TBackbone=LLaMA-3.2-3B2026.02 | 1.21 | 0.25 | |
| Llama-3.2-3B + wDPOBackbone=Llama-3.2-3B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 1.42 | 0.27 | |
| Dr.DPOBackbone=Qwen2.5-7B2026.02 | 1.53 | 0.46 | |
| Dr.DPOBackbone=Qwen-2.5-7B2026.03 | 1.53 | 0.46 | |
| ShaPO-RBackbone=LLaMA-3.2-3B2026.02 | 1.59 | 0.3 | |
| Dr.DPOBackbone=LLaMA-3.2-3B2026.02 | 1.78 | 0.33 | |
| Llama-3.2-3B + Dr.DPOBackbone=Llama-3.2-3B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 1.78 | 0.33 | |
| ShaPO-TBackbone=LLaMA-3-8B2026.02 | 2.48 | 0.23 | |
| wDPOBackbone=Llama-3-8B2026.03 | 3.33 | 0.45 | |
| Dr.DPOBackbone=LLaMA-3-8B2026.02 | 3.52 | 0.51 | |
| Dr.DPOBackbone=Llama-3-8B2026.03 | 3.52 | 0.51 | |
| rDPOBackbone=LLaMA-3.2-3B2026.02 | 4.19 | 1.97 | |
| Llama-3.2-3B + rDPOBackbone=Llama-3.2-3B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 4.19 | 1.97 | |
| rDPOBackbone=Qwen2.5-7B2026.02 | 4.23 | 1.76 | |
| rDPOBackbone=Qwen-2.5-7B2026.03 | 4.23 | 1.76 | |
| rDPOBackbone=LLaMA-3-8B2026.02 | 5.38 | 2.38 | |
| rDPOBackbone=Llama-3-8B2026.03 | 5.38 | 2.38 | |
| IPOBackbone=LLaMA-3.2-3B2026.02 | 11.12 | 6.07 | |
| Llama-3.2-3B + IPOBackbone=Llama-3.2-3B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 11.12 | 6.07 | |
| DPOBackbone=LLaMA-3.2-3B2026.02 | 11.31 | 6.13 | |
| Llama-3.2-3B + DPOBackbone=Llama-3.2-3B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 11.31 | 6.13 | |
| ShaPO-RBackbone=Pythia-2.8B2026.02 | 11.7 | 2 | |
| Pythia-2.8B + wDPOBackbone=Pythia-2.8B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 13.21 | 2.52 | |
| ShaPO-TBackbone=Pythia-2.8B2026.02 | 14.02 | 1.65 | |
| Pythia-2.8B + Dr.DPOBackbone=Pythia-2.8B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 18.82 | 4.9 | |
| IPOBackbone=Qwen2.5-7B2026.02 | 22.16 | 8.24 | |
| IPOBackbone=Qwen-2.5-7B2026.03 | 22.16 | 8.24 | |
| DPOBackbone=Qwen2.5-7B2026.02 | 22.97 | 8.63 | |
| DPOBackbone=Qwen-2.5-7B2026.03 | 22.97 | 8.63 | |
| cDPOBackbone=LLaMA-3.2-3B2026.02 | 26.65 | 13.29 | |
| Llama-3.2-3B + cDPOBackbone=Llama-3.2-3B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 26.65 | 13.29 | |
| IPOBackbone=LLaMA-3-8B2026.02 | 30.12 | 15.69 | |
| IPOBackbone=Llama-3-8B2026.03 | 30.12 | 15.69 | |
| Dr.DPOBackbone=Pythia-2.8B2026.02 | 30.27 | 18.24 | |
| DPOBackbone=LLaMA-3-8B2026.02 | 31.55 | 19.75 | |
| DPOBackbone=Llama-3-8B2026.03 | 31.55 | 19.75 | |
| cDPOBackbone=Qwen2.5-7B2026.02 | 32.56 | 13.66 | |
| cDPOBackbone=Qwen-2.5-7B2026.03 | 32.56 | 13.66 | |
| VanillaBackbone=Qwen2.5-7B2026.02 | 33.85 | 13.77 | |
| Qwen-2.5-7BBackbone=Qwen-2.5-7B2026.03 | 33.85 | 13.77 | |
| cDPOBackbone=LLaMA-3-8B2026.02 | 42.53 | 26.44 | |
| cDPOBackbone=Llama-3-8B2026.03 | 42.53 | 26.44 | |
| rDPOBackbone=Pythia-2.8B2026.02 | 60.45 | 20.94 | |
| Pythia-2.8B + rDPOBackbone=Pythia-2.8B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 60.45 | 20.94 | |
| DPOBackbone=Pythia-2.8B2026.02 | 63.6 | 20.79 | |
| Pythia-2.8B + DPOBackbone=Pythia-2.8B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 63.6 | 20.79 | |
| IPOBackbone=Pythia-2.8B2026.02 | 64.55 | 22.74 | |
| Pythia-2.8B + IPOBackbone=Pythia-2.8B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 64.55 | 22.74 | |
| cDPOBackbone=Pythia-2.8B2026.02 | 67.61 | 19.24 | |
| Pythia-2.8B + cDPOBackbone=Pythia-2.8B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 67.61 | 19.24 | |
| VanillaBackbone=LLaMA-3-8B2026.02 | 73.14 | 23.26 | |
| Llama-3-8BBackbone=Llama-3-8B2026.03 | 73.14 | 23.26 | |
| SFTBackbone=Qwen2.5-7B2026.02 | 73.45 | 55.05 | |
| SFTBackbone=Qwen-2.5-7B2026.03 | 73.45 | 55.05 | |
| VallinaBackbone=LLaMA-3.2-3B2026.02 | 74.17 | 22.32 | |
| Llama-3.2-3BBackbone=Llama-3.2-3B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 74.17 | 22.32 | |
| SFTBackbone=LLaMA-3.2-3B2026.02 | 74.28 | 50.58 | |
| Llama-3.2-3B + SFTBackbone=Llama-3.2-3B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 74.28 | 50.58 | |
| SFTBackbone=LLaMA-3-8B2026.02 | 75.1 | 58.68 | |
| SFTBackbone=Llama-3-8B2026.03 | 75.1 | 58.68 | |
| VallinaBackbone=Pythia-2.8B2026.02 | 78.95 | 29.69 | |
| Pythia-2.8BBackbone=Pythia-2.8B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 78.95 | 29.69 | |
| SFTBackbone=Pythia-2.8B2026.02 | 79.05 | 30.23 | |
| Pythia-2.8B + SFTBackbone=Pythia-2.8B, Training Dataset=PKU-SafeRLHF-30K2026.03 | 79.05 | 30.23 |