Safety Alignment on AdvBench
-0.38RewardSEA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SEABackbone=LLaMA-3.2-Instruct (1B)2025.05 | -0.38 | — | — | — | 0.19 | 75.32 | |
| RSBackbone=LLaMA-3.2-Instruct (1B)2025.05 | -1.51 | — | — | — | 0.96 | -24.68 | |
| BoN-64Backbone=LLaMA-3.2-Instruct (1B)2025.05 | -1.55 | — | — | — | 0.77 | 0 | |
| BoN-32Backbone=LLaMA-3.2-Instruct (1B)2025.05 | -1.75 | — | — | — | 0.96 | -24.68 | |
| SEABackbone=LLaMA-3-Base (8B)2025.05 | -1.83 | — | — | — | 3.85 | 73.3 | |
| CBSBackbone=LLaMA-3.2-Instruct (1B)2025.05 | -2.11 | — | — | — | 0.96 | -24.68 | |
| SFTBackbone=LLaMA-3.2-Instruct (1B)2025.05 | -2.36 | — | — | — | 0.77 | — | |
| BoN-8Backbone=LLaMA-3.2-Instruct (1B)2025.05 | -2.45 | — | — | — | 0.38 | 50.65 | |
| CBSBackbone=LLaMA-3-Base (8B)2025.05 | -3.84 | — | — | — | 6.35 | 55.96 | |
| SEABackbone=LLaMA-3.2-Base (3B)2025.05 | -4.03 | — | — | — | 6.92 | 86.37 | |
| BoN-64Backbone=LLaMA-3-Base (8B)2025.05 | -4.29 | — | — | — | 8.85 | 38.63 | |
| ARGSBackbone=LLaMA-3.2-Instruct (1B)2025.05 | -4.96 | — | — | — | 0.19 | 75.32 | |
| BoN-32Backbone=LLaMA-3-Base (8B)2025.05 | -5 | — | — | — | 8.65 | 40.01 | |
| ARGSBackbone=LLaMA-3-Base (8B)2025.05 | -5.41 | — | — | — | 8.27 | 42.65 | |
| SEABackbone=LLaMA-3.2-Base (1B)2025.05 | -5.61 | — | — | — | 5.58 | 91.54 | |
| BoN-64Backbone=LLaMA-3.2-Base (3B)2025.05 | -6.13 | — | — | — | 28.27 | 44.32 | |
| BoN-8Backbone=LLaMA-3-Base (8B)2025.05 | -6.32 | — | — | — | 11.73 | 18.65 | |
| BoN-32Backbone=LLaMA-3.2-Base (3B)2025.05 | -6.86 | — | — | — | 28.27 | 44.32 | |
| BoN-64Backbone=LLaMA-3.2-Base (1B)2025.05 | -7.16 | — | — | — | 43.85 | 33.52 | |
| RSBackbone=LLaMA-3-Base (8B)2025.05 | -7.41 | — | — | — | 6 | 58.39 | |
| CBSBackbone=LLaMA-3.2-Base (3B)2025.05 | -7.62 | — | — | — | 23.65 | 53.42 | |
| ARGSBackbone=LLaMA-3.2-Base (3B)2025.05 | -7.97 | — | — | — | 22.5 | 55.68 | |
| BoN-32Backbone=LLaMA-3.2-Base (1B)2025.05 | -8.07 | — | — | — | 43.65 | 33.82 | |
| SFTBackbone=LLaMA-3-Base (8B)2025.05 | -8.1 | — | — | — | 14.42 | — | |
| CBSBackbone=LLaMA-3.2-Base (1B)2025.05 | -8.24 | — | — | — | 24.81 | 62.38 | |
| BoN-8Backbone=LLaMA-3.2-Base (3B)2025.05 | -8.48 | — | — | — | 32.12 | 36.73 | |
| ARGSBackbone=LLaMA-3.2-Base (1B)2025.05 | -8.76 | — | — | — | 25.96 | 60.64 | |
| BoN-8Backbone=LLaMA-3.2-Base (1B)2025.05 | -9.59 | — | — | — | 49.23 | 25.36 | |
| RSBackbone=LLaMA-3.2-Base (3B)2025.05 | -9.98 | — | — | — | 40 | 21.21 | |
| RSBackbone=LLaMA-3.2-Base (1B)2025.05 | -10.73 | — | — | — | 50 | 24.2 | |
| SFTBackbone=LLaMA-3.2-Base (3B)2025.05 | -11.95 | — | — | — | 50.77 | — | |
| SFTBackbone=LLaMA-3.2-Base (1B)2025.05 | -12.42 | — | — | — | 65.96 | — | |
| Best-of-NMain Model=GPT-4o-mini, Projector=GPT-4o-mini2025.06 | — | 99 | 1 | 0 | — | — | |
| Chain-of-ThoughtMain Model=GPT-4o-mini, Projector=GPT-4o-mini2025.06 | — | 99 | 0 | 1 | — | — | |
| Multi-Agent DebateMain Model=GPT-4o-mini, Projector=GPT-4o-mini2025.06 | — | 99 | 1 | 0 | — | — | |
| RLAIFtraining_context=after re-alignment training2025.06 | — | 33 | 64 | 3 | — | — | |
| Self-Refine (Debate)training_context=after re-alignment training2025.06 | — | 33 | 61 | 6 | — | — |