Adversarial Attack on HarmBench standard behaviors
91ASRAdvGRPO
Evaluation results
| Method | Links | |
|---|---|---|
| AdvGRPOAttacker=Qwen2.5-14B, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 91 | |
| AdvGRPOAttacker=Qwen2.5-7B, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 84.5 | |
| SEMAAttacker=Qwen2.5-14B, Defender=GPT-4.12026.06 | 84.5 | |
| SEMAAttacker=Qwen2.5-3B, Defender=GPT-4.12026.06 | 80.5 | |
| AdvGRPOAttacker=Qwen2.5-14B, Defender=GPT-4.1, Evaluation Setting=ST2026.06 | 79.5 | |
| AdvGRPOAttacker=Qwen3.5-9B, Defender=GPT-4.1, Evaluation Setting=ST-Think2026.06 | 71 | |
| AdvGRPOAttacker=Qwen2.5-7B, Defender=GPT-4.1, Evaluation Setting=ST2026.06 | 69.5 | |
| GRP-OblitAttacker=Qwen-2.5-14B, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 55 | |
| Unsafe-SFTAttacker=GPT-4o, Defender=GPT-4.1, Evaluation Setting=Crescendo2026.06 | 35.5 | |
| GRP-OblitAttacker=Qwen-2.5-14B, Defender=GPT-4.1, Evaluation Setting=ST2026.06 | 25.5 | |
| AbliterationAttacker=Llama-3.1-8B, Defender=GPT-4.1, Evaluation Setting=ST2026.06 | 14 | |
| AbliterationAttacker=Llama-3.1-8B, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 13.5 | |
| Unsafe-SFTAttacker=GPT-4o, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 11.5 |