Attack Success Rate on DirectHarm4
85.86Attack Success RateBaseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineJudge=Heuristic2026.06 | 85.86 | |
| SafeLMJudge=Heuristic2026.06 | 79.45 | |
| SEALModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 71.75 | |
| GuardModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 66.75 | |
| GradSafeModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 65 | |
| Bi-AnchorModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 55.5 | |
| GuardModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 55.25 | |
| GradSafeModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 54 | |
| GuardModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 51 | |
| SEALModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 49.75 | |
| Bi-AnchorModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 46.5 | |
| GradSafeModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 43 | |
| GradSafeModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 37.25 | |
| RandomModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 37 | |
| RandomModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 36.75 | |
| GradSafeModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 36.5 | |
| GradSafeModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 33.5 | |
| Bi-AnchorModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 31.5 | |
| Bi-AnchorModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 30 | |
| SEALModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 28.25 | |
| SEALModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 27.75 | |
| GuardModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 27.5 | |
| SEALModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 26.75 | |
| RandomModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 26.75 | |
| RandomModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 26.25 | |
| RandomModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 26.25 | |
| SafeLMJudge=LLM2026.06 | 24.89 | |
| RandomModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 21.5 | |
| SEALModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 20 | |
| LARFModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 19.25 | |
| BaselineJudge=LLM2026.06 | 19.11 | |
| GuardModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 18.75 | |
| GuardModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 18.75 | |
| InstructModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 12.75 | |
| InstructModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 12.75 | |
| Bi-AnchorModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 12.25 | |
| InstructModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 11 | |
| InstructModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 11 | |
| DataShieldModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 10.2 | |
| LARFModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 9.5 | |
| InstructModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 9.5 | |
| InstructModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 9.5 | |
| SRPJudge=Heuristic2026.06 | 7.91 | |
| LARFModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 5.5 | |
| Bi-AnchorModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 4 | |
| DataShieldModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 3.25 | |
| DataShieldModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 2 | |
| DataShieldModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 1.5 | |
| SRPJudge=LLM2026.06 | 1.33 | |
| LARFModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 1.25 | |
| DataShieldModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 0.75 | |
| LARFModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 0 | |
| LARFModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 0 | |
| DataShieldModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 0 |