Adversarial Attack Success Rate on AdvBench
0ASRShips
Evaluation results
| Method | Links | |
|---|---|---|
| ShipsModel=Qwen2.5-7B2026.01 | 0 | |
| Base2025.10 | 0.58 | |
| RLBF2026.02 | 4.7 | |
| BSAFE+2026.02 | 6.6 | |
| Circuit Breakers2026.02 | 10.7 | |
| PatcherBase Model=Qwen2.5-1.5B2026.06 | 18.2 | |
| AutoDANTarget Model=Llama2-7B2026.01 | 24.04 | |
| PatcherBase Model=Qwen3-4B2026.06 | 33.1 | |
| CodeAttackTarget Model=GPT-4-1106, Defense=Rand-insert2024.03 | 34 | |
| GCGTarget Model=Qwen2.5-7B2026.01 | 37.38 | |
| Llama2-7BModel=Llama2-7B, Condition=Vanilla2026.01 | 38 | |
| ShipsModel=Mistral-7B2026.01 | 38 | |
| CodeAttackTarget Model=GPT-4-1106, Defense=Rand-patch2024.03 | 40 | |
| GCGTarget Model=Llama2-7B2026.01 | 46.92 | |
| GCGTarget Model=Llama3.1-8B2026.01 | 48.08 | |
| CodeAttackTarget Model=GPT-4-1106, Defense=Rand-swap2024.03 | 52 | |
| BoosterBase Model=Qwen3-4B2026.06 | 52 | |
| SFTBase Model=Qwen2.5-1.5B2026.06 | 56 | |
| SFTBase Model=Qwen3-4B2026.06 | 56.2 | |
| PatcherBase Model=Llama3-8B2026.06 | 56.4 | |
| BoosterBase Model=Qwen2.5-1.5B2026.06 | 57.4 | |
| DSNTarget Model=Qwen2.5-7B2026.01 | 58.27 | |
| AdvPrefixTarget Model=Llama2-7B2026.01 | 60.58 | |
| DSNTarget Model=Llama2-7B2026.01 | 61.92 | |
| AutoDANTarget Model=Llama3.1-8B2026.01 | 62.5 | |
| RL2026.02 | 62.6 | |
| DSNTarget Model=Llama3.1-8B2026.01 | 63.08 | |
| IT2026.02 | 65.6 | |
| VaccineBase Model=Qwen3-4B2026.06 | 70.9 | |
| VaccineBase Model=Qwen2.5-1.5B2026.06 | 72.6 | |
| CodeAttackTarget Model=CodeLlama-70b, Defense=Paraphrase2024.03 | 74 | |
| AutoDANTarget Model=Mistral-7B2026.01 | 74.23 | |
| AutoDANTarget Model=Qwen2.5-7B2026.01 | 76.73 | |
| AdvPrefixTarget Model=Llama3.1-8B2026.01 | 77.31 | |
| CodeAttackTarget Model=CodeLlama-70b, Defense=OpenAI2024.03 | 78 | |
| CodeAttackTarget Model=Claude-2, Defense=Paraphrase2024.03 | 80 | |
| AdvPrefixTarget Model=Qwen2.5-7B2026.01 | 80.19 | |
| GCGTarget Model=Mistral-7B2026.01 | 80.96 | |
| CodeAttackTarget Model=GPT-3.5, Defense=OpenAI2024.03 | 82 | |
| CodeAttackTarget Model=Claude-1, Defense=OpenAI2024.03 | 82 | |
| CodeAttackTarget Model=GPT-3.5, Defense=Rand-insert2024.03 | 82 | |
| CodeAttackTarget Model=CodeLlama-70b, Defense=Rand-insert2024.03 | 82 | |
| CodeAttackTarget Model=GPT-3.5, Defense=Rand-swap2024.03 | 82 | |
| CodeAttackTarget Model=GPT-3.5, Defense=Rand-patch2024.03 | 82 | |
| ORTHOTarget Model=Mistral-7B2026.01 | 82.12 | |
| CodeAttackTarget Model=GPT-3.5, Defense=No defenses2024.03 | 84 | |
| CodeAttackTarget Model=Claude-1, Defense=No defenses2024.03 | 84 | |
| CodeAttackTarget Model=Claude-2, Defense=OpenAI2024.03 | 84 | |
| CodeAttackTarget Model=GPT-3.5, Defense=Paraphrase2024.03 | 84 | |
| CodeAttackTarget Model=GPT-4-1106, Defense=Paraphrase2024.03 | 84 | |
| CodeAttackTarget Model=Claude-1, Defense=Paraphrase2024.03 | 84 | |
| CodeAttackTarget Model=Claude-1, Defense=Rand-insert2024.03 | 84 | |
| CodeAttackTarget Model=Claude-1, Defense=Rand-swap2024.03 | 84 | |
| CodeAttackTarget Model=Claude-1, Defense=Rand-patch2024.03 | 84 | |
| CodeAttackTarget Model=Claude-2, Defense=Rand-patch2024.03 | 84 | |
| CodeAttackTarget Model=CodeLlama-70b, Defense=Rand-patch2024.03 | 84 | |
| ORTHOTarget Model=Qwen2.5-7B2026.01 | 84.81 | |
| CodeAttackTarget Model=GPT-4-1106, Defense=No defenses2024.03 | 86 | |
| CodeAttackTarget Model=Claude-2, Defense=No defenses2024.03 | 86 | |
| CodeAttackTarget Model=CodeLlama-70b, Defense=No defenses2024.03 | 86 | |
| CodeAttackTarget Model=GPT-4-1106, Defense=OpenAI2024.03 | 86 | |
| CodeAttackTarget Model=Claude-2, Defense=Rand-insert2024.03 | 86 | |
| CodeAttackTarget Model=Claude-2, Defense=Rand-swap2024.03 | 86 | |
| CodeAttackTarget Model=CodeLlama-70b, Defense=Rand-swap2024.03 | 86 | |
| GOSVTarget Model=Qwen2.5-7B2026.01 | 86.35 | |
| DSNTarget Model=Mistral-7B2026.01 | 86.92 | |
| BoosterBase Model=Llama3-8B2026.06 | 88.4 | |
| ORTHOTarget Model=Llama2-7B2026.01 | 88.65 | |
| SFT2025.10 | 89.23 | |
| SFTBase Model=Llama3-8B2026.06 | 90.7 | |
| GOSVTarget Model=Llama2-7B2026.01 | 92.5 | |
| VaccineBase Model=Llama3-8B2026.06 | 92.8 | |
| ORTHOTarget Model=Llama3.1-8B2026.01 | 94.42 | |
| AdvPrefixTarget Model=Mistral-7B2026.01 | 95.77 | |
| Llama3.1-8BModel=Llama3.1-8B, Condition=Vanilla2026.01 | 96 | |
| GOSVTarget Model=Llama3.1-8B2026.01 | 96.35 | |
| RLVR2025.10 | 99.04 | |
| GOSVTarget Model=Mistral-7B2026.01 | 100 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Condition=Vanilla2026.01 | 173 | |
| ShipsModel=Llama3.1-8B2026.01 | 1,519 | |
| ShipsModel=Llama2-7B2026.01 | 3,308 | |
| Mistral-7BModel=Mistral-7B, Condition=Vanilla2026.01 | 3,558 | |
| GOSVModel=Qwen2.5-7B2026.01 | 8,635 | |
| Fine-TuningModel=Qwen2.5-7B2026.01 | 9,058 | |
| Fine-TuningModel=Llama2-7B2026.01 | 9,192 | |
| GOSVModel=Llama2-7B2026.01 | 9,250 | |
| Fine-TuningModel=Llama3.1-8B2026.01 | 9,481 | |
| Fine-TuningModel=Mistral-7B2026.01 | 9,500 | |
| GOSVModel=Llama3.1-8B2026.01 | 9,635 | |
| GOSVModel=Mistral-7B2026.01 | 10,000 |