Jailbreak Success Rate on AdvBench
94.4GPT-4o Jailbreak Success RateCodeChameleon
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| CodeChameleonAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 94.4 | 57.7 | 43.8 | 94 | 30.4 | 84 | 98.7 | 71.9 | |
| ReNeLLMAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 88.3 | 45.4 | 7.3 | 26.5 | 10 | 64.9 | 94.4 | 48.9 | |
| PosteriorAttackAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 78.7 | 99 | 33.7 | 78.3 | 93.7 | 98.5 | 99.2 | 83 | |
| FlipAttackAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 73.3 | 13.8 | 0 | 60.8 | 0 | 74 | 95.4 | 45.3 | |
| AutoDANAttack Type=Transferred White-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 17.7 | 2.7 | 0.2 | 0.6 | 0.4 | 5.4 | 54.2 | 11.6 | |
| DeepInceptionAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 9.6 | 1 | 0 | 42.5 | 15.6 | 46 | 97.7 | 30.3 | |
| PromptAttackAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 7.3 | 1.3 | 0.2 | 9 | 77.1 | 8.5 | 26.3 | 18.5 | |
| ArtPromptAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 1.7 | 6 | 5.6 | 19 | 0.6 | 2.5 | 50.8 | 12.3 | |
| GCGAttack Type=Transferred White-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 0 | 0.2 | 0 | 0 | 0 | 0 | 0.2 | 0.1 | |
| ManyShotAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 0 | 0.2 | 0 | 0 | 0 | 25.4 | 2.3 | 4 | |
| SelfCipherAttack Type=Black-box, Judge=GPT-4o-mini, Reasoning Effort=Low (for GPT-5 and Claude models)2026.06 | 0 | 6.7 | 0.2 | 0 | 0 | 43.3 | 77.7 | 18.3 |