Audio LLM Jailbreak Prompts
94.23JSR (%)GRM-S
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GRM-STarget Model=Qwen2-Audio2026.04 | 94.23 | 3.98 | 6.59 | |
| GRM-STarget Model=MERaLiON2026.04 | 94.23 | 7.07 | 6.29 | |
| GRM-STarget Model=Qwen2.5-Omni2026.04 | 93.27 | 17.06 | 5.88 | |
| SSJTarget Model=MERaLiON2026.04 | 93.27 | — | — | |
| GRMTarget Model=Qwen2-Audio2026.04 | 92.31 | 3.59 | 6.66 | |
| AutoDANTarget Model=Qwen2.5-Omni2026.04 | 92.31 | 41.37 | 7.1 | |
| SSJTarget Model=LLaMA-Omni22026.04 | 92.31 | — | — | |
| GRMTarget Model=MERaLiON2026.04 | 92.31 | 6.71 | 6.62 | |
| SSJTarget Model=Qwen2-Audio2026.04 | 91.35 | — | — | |
| GRM-STarget Model=Average2026.04 | 91.11 | 11.29 | 6.14 | |
| SSJTarget Model=Average2026.04 | 90.87 | — | — | |
| AutoDANTarget Model=MERaLiON2026.04 | 88.46 | 6.89 | 5.76 | |
| GRMTarget Model=Average2026.04 | 88.46 | 10.45 | 6.33 | |
| GRMTarget Model=Qwen2.5-Omni2026.04 | 87.5 | 16 | 6.21 | |
| SSJTarget Model=Qwen2.5-Omni2026.04 | 84.62 | — | — | |
| GRM-STarget Model=LLaMA-Omni22026.04 | 82.69 | 17.04 | 5.79 | |
| GRMTarget Model=LLaMA-Omni22026.04 | 81.73 | 15.51 | 5.84 | |
| AutoDANTarget Model=Average2026.04 | 75.72 | 35.81 | 6.27 | |
| AudioBenchTarget Model=Qwen2.5-Omni2026.04 | 71.15 | 12.18 | 6.11 | |
| AutoDANTarget Model=Qwen2-Audio2026.04 | 69.23 | 44.12 | 6.58 | |
| BoNTarget Model=LLaMA-Omni22026.04 | 68.27 | 74.6 | 4.26 | |
| GCGTarget Model=Qwen2.5-Omni2026.04 | 62.5 | 38.69 | 7.05 | |
| BoNTarget Model=Qwen2.5-Omni2026.04 | 60.58 | 98.06 | 4.53 | |
| BoNTarget Model=Average2026.04 | 56.97 | 68.8 | 4.09 | |
| AudioBenchTarget Model=Average2026.04 | 54.09 | 16.77 | 6.3 | |
| AutoDANTarget Model=LLaMA-Omni22026.04 | 52.88 | 50.86 | 5.63 | |
| AudioBenchTarget Model=LLaMA-Omni22026.04 | 51.92 | 30.81 | 6.1 | |
| BoNTarget Model=MERaLiON2026.04 | 50.96 | 54.35 | 3.51 | |
| AudioBenchTarget Model=Qwen2-Audio2026.04 | 49.04 | 11.48 | 5.99 | |
| BoNTarget Model=Qwen2-Audio2026.04 | 48.08 | 48.2 | 4.07 | |
| AudioBenchTarget Model=MERaLiON2026.04 | 44.23 | 12.61 | 6.99 | |
| GCGTarget Model=Average2026.04 | 36.3 | 41.48 | 6.29 | |
| GCGTarget Model=Qwen2-Audio2026.04 | 30.77 | 37.69 | 6.42 | |
| GCGTarget Model=LLaMA-Omni22026.04 | 27.88 | 46.59 | 5.81 | |
| VanillaTarget Model=LLaMA-Omni22026.04 | 25.96 | 11.04 | 5.9 | |
| GCGTarget Model=MERaLiON2026.04 | 24.04 | 42.93 | 5.88 | |
| VanillaTarget Model=Qwen2-Audio2026.04 | 19.23 | 3.03 | 6.99 | |
| VanillaTarget Model=Qwen2.5-Omni2026.04 | 19.23 | 6.32 | 7.21 | |
| VanillaTarget Model=Average2026.04 | 17.79 | 6.33 | 6.69 | |
| VanillaTarget Model=MERaLiON2026.04 | 6.73 | 4.93 | 6.66 |