Adversarial Attack on AdvBench (held-out split)
92.5ASRSEMA
Evaluation results
| Method | Links | |
|---|---|---|
| SEMAAttacker=Qwen2.5-14B, Defender=GPT-4.12026.06 | 92.5 | |
| SEMAAttacker=Qwen2.5-3B, Defender=GPT-4.12026.06 | 92.3 | |
| AdvGRPOAttacker=Qwen2.5-7B, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 90 | |
| AdvGRPOAttacker=Qwen2.5-14B, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 90 | |
| AdvGRPOAttacker=Qwen3.5-9B, Defender=GPT-4.1, Evaluation Setting=ST-Think2026.06 | 79.1 | |
| AdvGRPOAttacker=Qwen2.5-14B, Defender=GPT-4.1, Evaluation Setting=ST2026.06 | 79.1 | |
| AdvGRPOAttacker=Qwen2.5-7B, Defender=GPT-4.1, Evaluation Setting=ST2026.06 | 72.5 | |
| GRP-OblitAttacker=Qwen-2.5-14B, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 52.5 | |
| Unsafe-SFTAttacker=GPT-4o, Defender=GPT-4.1, Evaluation Setting=Crescendo2026.06 | 45.8 | |
| GRP-OblitAttacker=Qwen-2.5-14B, Defender=GPT-4.1, Evaluation Setting=ST2026.06 | 22.5 | |
| AbliterationAttacker=Llama-3.1-8B, Defender=GPT-4.1, Evaluation Setting=ST2026.06 | 14.1 | |
| Unsafe-SFTAttacker=GPT-4o, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 11.6 | |
| AbliterationAttacker=Llama-3.1-8B, Defender=GPT-4.1, Evaluation Setting=MT2026.06 | 10.8 |