Jailbreaking on Harmbench Standard
0ASR (Claude-Sonnet-4.5)AutoDAN
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| AutoDANsetting=black-box2025.11 | 0 | 33.5 | 27 | 22.5 | 25.5 | 8.5 | 28.5 | 20.8 | |
| AutoDAN-Turbosetting=black-box2025.11 | 0 | 74.5 | 75.5 | 20.5 | 63 | 30.5 | 84 | 49.7 | |
| CoAsetting=black-box2025.11 | 0 | 3 | 12 | 2 | 18.5 | 22 | 4.5 | 8.9 | |
| Crescendosetting=black-box2025.11 | 0 | 32.5 | 46 | 6 | 62.5 | 58 | 42.5 | 35.4 | |
| RedQueensetting=black-box2025.11 | 0 | 46 | 12 | 0 | 35 | 15 | 17 | 17.9 | |
| RainbowTeamingsetting=black-box2025.11 | 4 | 79.5 | 54 | 10.5 | 45 | 28 | 26.5 | 35.4 | |
| TreeAttacksetting=black-box2025.11 | 6.5 | 80 | 74 | 37.5 | 84 | 38.5 | 87 | 58.2 | |
| PAIRsetting=black-box2025.11 | 10 | 86.5 | 78 | 58 | 79 | 54.5 | 89 | 65 | |
| AutoRedTeamersetting=black-box2025.11 | 10.5 | 35 | 70.5 | 23.5 | 85 | 50.5 | 84.5 | 51.4 | |
| CodeAttacksetting=black-box2025.11 | 12.5 | 78.5 | 79.5 | 32.5 | 79.5 | 69.5 | 80 | 61.7 | |
| RACEsetting=black-box2025.11 | 16.5 | 65.5 | 78.5 | 24 | 32 | 24 | 41.5 | 40.3 | |
| ActorAttacksetting=black-box2025.11 | 27.5 | 87.5 | 83.5 | 22.5 | 79 | 74.5 | 89 | 66.2 | |
| X-Teamingsetting=black-box2025.11 | 52.5 | 97.5 | 96 | 88.5 | 83.5 | 82 | 99 | 85.7 | |
| EvoSynthsetting=black-box, query_budget=180 queries2025.11 | 85.5 | 98 | 97.5 | 94.5 | 98.5 | 98 | 99.5 | 95.9 |