Audio Jailbreak on JBB, AdvBench, and HarmBench Combined
98.16ASR (Qwen 2-Audio)SACRED-Bench
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SACRED-BenchMethod Category=Instance-Specific, Query Time=22026.05 | 98.16 | 81.5 | 92.83 | — | 70.05 | 70.05 | — | 66.75 | — | — | 85.12 | |
| AdvWave (black-box)Method Category=Instance-Specific, Query Time=312026.05 | 96.7 | — | — | 100 | — | 91.1 | — | — | — | 95.1 | — | |
| AIA (Ours)Method Category=Universal, Query Time=11.72026.05 | 96.3 | 69.05 | 100 | 32.26 | 72.73 | 75.61 | 93.1 | 100 | — | — | — | |
| BoNMethod Category=Instance-Specific, Query Time=6002026.05 | 85.4 | — | — | 99.6 | — | 59 | — | — | — | 97.6 | 74.21 | |
| Multi-AudioJailMethod Category=Instance-Specific, Query Time=12026.05 | 57.79 | — | — | — | — | — | — | — | — | — | — | |
| Speech-Specific JailbreakMethod Category=Instance-Specific, Query Time=62026.05 | 56.67 | — | — | 41.9 | — | 34.6 | — | — | — | 93.9 | 70.67 | |
| AJailBenchMethod Category=Instance-Specific, Query Time=10+B2026.05 | 52.6 | — | — | 64.8 | — | 31.4 | — | — | — | 73.7 | — | |
| Jailbreak-AudioBenchMethod Category=Instance-Specific, Query Time=322026.05 | 48.8 | — | — | — | — | 8.4 | — | — | 49.4 | — | — | |
| AMSEMethod Category=Universal, Query Time=182026.05 | 41.9 | — | — | 21.1 | — | 5.7 | — | — | — | 34.6 | — | |
| AudioJailbreakMethod Category=Universal, Query Time=12026.05 | 1.71 | — | — | 20.41 | — | — | — | — | — | — | — |