Adversarial Safety Evaluation on AdvBench (test)
73.86GCG Success RateSFT
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SFTBase Model=LLaMA-3.1-8B-Instruct, Safety Alignment=SFT2026.03 | 73.86 | 72.88 | 80.48 | 96.44 | 28.57 | 86.6 | 51.24 | 49.62 | |
| NemoGuardBase Model=LLaMA-3.1-8B-Instruct, Safety Alignment=NemoGuard2026.03 | 59.57 | 65 | 81.53 | 94.62 | 57.14 | 64.52 | 26.42 | 31.35 | |
| RLHFBase Model=LLaMA-2-7B-Chat, Safety Alignment=RLHF2026.03 | 47.84 | 51.02 | 31.35 | 80.16 | 24.17 | 8.42 | 27.1 | 0 | |
| SFTBase Model=Qwen2.5-7B-Instruct, Safety Alignment=SFT2026.03 | 43.48 | 27 | 47.21 | 93.27 | 36.73 | 88.65 | 25.62 | 8.72 | |
| Egida-DPOBase Model=Qwen2.5-7B-Instruct, Safety Alignment=Egida-DPO2026.03 | 39.36 | 29.5 | 50.78 | 89.81 | 28.57 | 83.65 | 14.68 | 6.92 | |
| PSR (N=8)Base Model=LLaMA-3.1-8B-Instruct, Safety Alignment=PSR (N=8)2026.03 | 25.02 | 0 | 10.05 | 35.58 | 22.45 | 32.69 | 0.26 | 0 | |
| SFTBase Model=Qwen2.5-14B-Instruct, Safety Alignment=SFT2026.03 | 23.85 | 18.5 | 24.71 | 74.62 | 29.25 | 72.6 | 11.35 | 1.73 | |
| PSA DetectorBase Model=Qwen2.5-14B-Instruct, Safety Alignment=PSA Detector2026.03 | 23.65 | 0 | 13.04 | 0 | 26.53 | 64.04 | 2.32 | 0 | |
| ShieldGemmaBase Model=Qwen2.5-14B-Instruct, Safety Alignment=ShieldGemma2026.03 | 22.12 | 14 | 24.51 | 74.81 | 28.57 | 70.77 | 4.51 | 1.73 | |
| TSC-GRPOBase Model=LLaMA-2-7B-Chat, Safety Alignment=TSC-GRPO2026.03 | 18.62 | 15.56 | 18.17 | 25.51 | 12.46 | 6.47 | 0 | 0 | |
| TSC-GRPOBase Model=LLaMA-3.1-8B-Instruct, Safety Alignment=TSC-GRPO2026.03 | 17.94 | 0 | 4.63 | 24.48 | 17.28 | 24.49 | 0 | 0 | |
| PSR (N=8)Base Model=Qwen2.5-7B-Instruct, Safety Alignment=PSR (N=8)2026.03 | 3.42 | 1 | 10.05 | 35.58 | 22.45 | 0.71 | 1.86 | 0 | |
| TSC-GRPOBase Model=Qwen2.5-7B-Instruct, Safety Alignment=TSC-GRPO2026.03 | 1.35 | 0 | 2.58 | 26.32 | 16.45 | 0 | 0 | 0 | |
| PSR (N=8)Base Model=Qwen2.5-14B-Instruct, Safety Alignment=PSR (N=8)2026.03 | 1.35 | 0 | 14.59 | 37.44 | 19.39 | 6.54 | 0 | 0 | |
| TSC-GRPOBase Model=Qwen2.5-14B-Instruct, Safety Alignment=TSC-GRPO2026.03 | 0 | 0 | 4.47 | 25.37 | 14.48 | 0.71 | 0 | 0 |