Jailbreak Attack on AdvBench (held-out)
69.1ASRAdvGRPO
Evaluation results
| Method | Links | |
|---|---|---|
| AdvGRPOTransfer Defender=Gemma-2-9B, Attacker=Qwen2.5-14B, Turn Type=Single-turn (ST)2026.06 | 69.1 | |
| AdvGRPOTransfer Defender=Llama-3.1-8B, Attacker=Qwen2.5-14B, Turn Type=Single-turn (ST)2026.06 | 71.6 | |
| AdvGRPOTransfer Defender=Phi-4-mini, Attacker=Qwen2.5-14B, Turn Type=Single-turn (ST)2026.06 | 77.5 | |
| AdvGRPOTransfer Defender=Gemma-2-9B, Attacker=Qwen2.5-14B, Turn Type=Multi-turn (MT)2026.06 | 88.3 | |
| AdvGRPOTransfer Defender=Phi-4-mini, Attacker=Qwen2.5-14B, Turn Type=Multi-turn (MT)2026.06 | 90.8 | |
| SEMATransfer Defender=Llama-3.1-8B, Attacker=Qwen2.5-14B2026.06 | 90.8 | |
| AdvGRPOTransfer Defender=Llama-3.1-8B, Attacker=Qwen2.5-14B, Turn Type=Multi-turn (MT)2026.06 | 92.5 | |
| SEMATransfer Defender=Phi-4-mini, Attacker=Qwen2.5-14B2026.06 | 93.3 | |
| SEMATransfer Defender=Gemma-2-9B, Attacker=Qwen2.5-14B2026.06 | 96.6 |