Safety Alignment on StrongReject
58Safe@1R1 - 7B + UnsafeChain full
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| R1 - 7B + UnsafeChain fullSize=7B, Alignment=UnsafeChain (full)2025.07 | 58 | — | — | — | — | — | — | — | |
| R1 - 8B + UnsafeChain fullSize=8B, Alignment=UnsafeChain (full)2025.07 | 49 | — | — | — | — | — | — | — | |
| R1 - 8B + STAR-1Size=8B, Alignment=STAR-12025.07 | 48 | — | — | — | — | — | — | — | |
| R1 - 1.5B + UnsafeChain fullSize=1.5B, Alignment=UnsafeChain (full)2025.07 | 48 | — | — | — | — | — | — | — | |
| R1 - 8B + UnsafeChain selectedSize=8B, Alignment=UnsafeChain (selected subset)2025.07 | 44 | — | — | — | — | — | — | — | |
| R1 - 8B + SafeChainSize=8B, Alignment=SafeChain2025.07 | 42.49 | — | — | — | — | — | — | — | |
| R1 - 8B + UnsafeChain randomSize=8B, Alignment=UnsafeChain (random subset)2025.07 | 35 | — | — | — | — | — | — | — | |
| R1 - 7BSize=7B2025.07 | 31.95 | — | — | — | — | — | — | — | |
| R1 - 8BSize=8B2025.07 | 31.63 | — | — | — | — | — | — | — | |
| R1 - 7B + STAR-1Size=7B, Alignment=STAR-12025.07 | 31 | — | — | — | — | — | — | — | |
| R1 - 7B + UnsafeChain selectedSize=7B, Alignment=UnsafeChain (selected subset)2025.07 | 30 | — | — | — | — | — | — | — | |
| R1 - 7B + SafeChainSize=7B, Alignment=SafeChain2025.07 | 26.52 | — | — | — | — | — | — | — | |
| R1 - 7B + UnsafeChain randomSize=7B, Alignment=UnsafeChain (random subset)2025.07 | 26 | — | — | — | — | — | — | — | |
| R1 - 1.5B + SafeChainSize=1.5B, Alignment=SafeChain2025.07 | 13 | — | — | — | — | — | — | — | |
| R1 - 1.5BSize=1.5B2025.07 | 9 | — | — | — | — | — | — | — | |
| R1 - 1.5B + UnsafeChain selectedSize=1.5B, Alignment=UnsafeChain (selected subset)2025.07 | 9 | — | — | — | — | — | — | — | |
| R1 - 1.5B + UnsafeChain randomSize=1.5B, Alignment=UnsafeChain (random subset)2025.07 | 7 | — | — | — | — | — | — | — | |
| R1 - 1.5B + STAR-1Size=1.5B, Alignment=STAR-12025.07 | 4 | — | — | — | — | — | — | — | |
| AdamWRuntime (min)=110.32026.04 | — | — | — | — | — | — | — | 33.2 | |
| Base modelRuntime (min)=N/A2026.04 | — | — | — | — | — | — | — | 19.5 | |
| BLURRuntime (min)=114.72026.04 | — | — | — | — | — | — | — | 35.8 | |
| DSA:LLD:TAlambda value=02025.05 | — | — | — | — | — | — | — | 11 | |
| DSA:LST+:TAlambda value=02025.05 | — | — | — | — | — | — | — | 4 | |
| ImplicitRMPolicy model=Qwen2.5-Instruct-7B2026.03 | — | — | — | — | — | 97.1 | 7.8 | — | |
| ImplicitRMPolicy model=Qwen3-Instruct-4B2026.03 | — | — | — | — | — | 98.68 | 7 | — | |
| LAGAMPolicy model=Qwen2.5-Instruct-7B2026.03 | — | — | — | — | — | 94.12 | 4.5 | — | |
| LAGAMPolicy model=Qwen3-Instruct-4B2026.03 | — | — | — | — | — | 95.88 | 4 | — | |
| MuonRuntime (min)=191.22026.04 | — | — | — | — | — | — | — | 36.4 | |
| NaivePolicy model=Qwen2.5-Instruct-7B2026.03 | — | — | — | — | — | 90.07 | — | — | |
| NaivePolicy model=Qwen3-Instruct-4B2026.03 | — | — | — | — | — | 92.23 | — | — | |
| POMERuntime (min)=115.42026.04 | — | — | — | — | — | — | — | 36.7 | |
| SDR2Policy model=Qwen2.5-Instruct-7B2026.03 | — | — | — | — | — | 91.38 | 1.5 | — | |
| SDR2Policy model=Qwen3-Instruct-4B2026.03 | — | — | — | — | — | 94.17 | 2.1 | — | |
| SelectMixPolicy model=Qwen2.5-Instruct-7B2026.03 | — | — | — | — | — | 94.37 | 4.8 | — | |
| SelectMixPolicy model=Qwen3-Instruct-4B2026.03 | — | — | — | — | — | 95.34 | 3.4 | — | |
| SIFTRuntime (min)=215.62026.04 | — | — | — | — | — | — | — | 42.8 | |
| Zephyr Base Modellambda value=12025.05 | — | — | — | — | — | — | — | 58 | |
| Zephyr LoRA-alignedlambda value=02025.05 | — | — | — | — | — | — | — | 4 |