Harmful score evaluation on BeaverTails (test)
28.4Harmful ScoreSFT
Evaluation Results
| Method | Links | |
|---|---|---|
| SFTn=2000, Harmful Ratio (p)=0.12026.02 | 28.4 | |
| SFTHarmful Ratio (p)=0.2, n=10002026.02 | 28 | |
| SFT2026.02 | 27.6 | |
| SFTHarmful Ratio (p)=0.1, n=10002026.02 | 27.6 | |
| SFTn=Average, Harmful Ratio (p)=0.12026.02 | 25.84 | |
| SFTHarmful Ratio (p)=0.15, n=10002026.02 | 25.8 | |
| SFTn=1500, Harmful Ratio (p)=0.12026.02 | 24.7 | |
| SFTn=500, Harmful Ratio (p)=0.12026.02 | 24.4 | |
| SFTn=2500, Harmful Ratio (p)=0.12026.02 | 24.1 | |
| SFTHarmful Ratio (p)=Average, n=10002026.02 | 23.1 | |
| SFTHarmful Ratio (p)=0.05, n=10002026.02 | 21 | |
| SPARD2026.02 | 20.6 | |
| SPARDHarmful Ratio (p)=Average, n=10002026.02 | 19.84 | |
| SafeGrad2026.02 | 18.5 | |
| ConstrainedSFTHarmful Ratio (p)=Average, n=10002026.02 | 17.8 | |
| ConstrainedSFT2026.02 | 17.6 | |
| DSSHarmful Ratio (p)=Average, n=10002026.02 | 16.86 | |
| DSS2026.02 | 16.8 | |
| SafeGradHarmful Ratio (p)=Average, n=10002026.02 | 16.3 | |
| AsFT2026.02 | 16.1 | |
| AsFTHarmful Ratio (p)=Average, n=10002026.02 | 15.96 | |
| SFTHarmful Ratio (p)=clean, n=10002026.02 | 15.1 | |
| Lisa2026.02 | 14.8 | |
| LisaHarmful Ratio (p)=Average, n=10002026.02 | 13.88 | |
| Surgeryn=500, Harmful Ratio (p)=0.12026.02 | 9.7 | |
| SurgeryHarmful Ratio (p)=0.15, n=10002026.02 | 9.6 | |
| Surgeryn=1500, Harmful Ratio (p)=0.12026.02 | 9.4 | |
| SurgeryHarmful Ratio (p)=0.2, n=10002026.02 | 9.3 | |
| Surgeryn=Average, Harmful Ratio (p)=0.12026.02 | 8.94 | |
| Surgery2026.02 | 8.9 | |
| SurgeryHarmful Ratio (p)=0.1, n=10002026.02 | 8.9 | |
| SurgeryHarmful Ratio (p)=0.05, n=10002026.02 | 8.8 | |
| SurgeryHarmful Ratio (p)=Average, n=10002026.02 | 8.42 | |
| Surgeryn=2500, Harmful Ratio (p)=0.12026.02 | 8.4 | |
| Surgeryn=2000, Harmful Ratio (p)=0.12026.02 | 8.3 | |
| SurgeryHarmful Ratio (p)=clean, n=10002026.02 | 5.5 | |
| Unlearn-SmoothModel=OPTML-Group/NPO-SAM-WMDP, Attack=Adaptive attack2026.05 | 0.738 | |
| VAAModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.707 | |
| VaccineModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.707 | |
| VAAModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.705 | |
| SDDModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.705 | |
| VaccineModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.701 | |
| SDDModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.693 | |
| SDDModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.691 | |
| VaccineModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.5 | |
| BoosterModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.416 | |
| CTRAPModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.375 | |
| CTRAPModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.324 | |
| BoosterModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.307 | |
| VAAModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.239 | |
| CTRAPModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.229 | |
| BoosterModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.2 |