Jailbreak Attack on AdvBench (ASR, Harmful Score)
0Attack Success Rate (ASR)DIRECT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DIRECTVictim=LLaDA-Instruct2026.06 | 0 | 1 | |
| DIRECTVictim=LLaDA 1.52026.06 | 0 | 1 | |
| DIRECTVictim=LLaDA2.0-mini2026.06 | 0 | 1 | |
| DIRECTVictim=Dream-Instruct2026.06 | 0 | 1 | |
| PADVictim=LLaDA2.0-mini2026.06 | 9 | 2.96 | |
| DIJAVictim=LLaDA2.0-mini2026.06 | 15 | 3.25 | |
| CFATarget Model=Gemini 2.5 Pro2026.04 | 26.7 | 4.22 | |
| CFATarget Model=GPT-4o2026.04 | 30.6 | 3.82 | |
| PAIRTarget Model=GPT-4o2026.04 | 39.8 | 4.39 | |
| PAIRTarget Model=Gemini 2.5 Pro2026.04 | 40.8 | 4.79 | |
| CFATarget Model=GLM-4.52026.04 | 44.6 | 4.75 | |
| PAIRTarget Model=Deepseek V32026.04 | 45 | 5.01 | |
| PAIRTarget Model=GLM-4.52026.04 | 45.6 | 5 | |
| MASKFORGE-GoalVictim=LLaDA2.0-mini, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Goal sampling2026.06 | 47 | 4.57 | |
| PAIRTarget Model=Qwen 32026.04 | 49.8 | 5.1 | |
| CIATarget Model=GPT-4o2026.04 | 51.7 | 4.98 | |
| DIJAVictim=LLaDA 1.52026.06 | 55 | 4.17 | |
| PADVictim=LLaDA-Instruct2026.06 | 56 | 4.22 | |
| DIJAVictim=LLaDA-Instruct2026.06 | 58 | 4.18 | |
| CrescendoTarget Model=GPT-4o2026.04 | 58.3 | 5.76 | |
| CFATarget Model=Qwen 32026.04 | 58.3 | 6 | |
| PADVictim=LLaDA 1.52026.06 | 59 | 4.2 | |
| CFATarget Model=Deepseek V32026.04 | 60 | 6.39 | |
| CIATarget Model=GLM-4.52026.04 | 60.2 | 5.77 | |
| CIATarget Model=Qwen 32026.04 | 60.2 | 5.31 | |
| CIATarget Model=Gemini 2.5 Pro2026.04 | 60.8 | 6.22 | |
| MASKFORGE-GoalVictim=LLaDA 1.5, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Goal sampling2026.06 | 61 | 4.69 | |
| DIJAVictim=Dream-Instruct2026.06 | 62 | 4.28 | |
| DIRECTVictim=MMaDA-CoT2026.06 | 62 | 4.23 | |
| DIJAVictim=MMaDA-CoT2026.06 | 63 | 4.37 | |
| PADVictim=Dream-Instruct2026.06 | 64 | 4.34 | |
| GOATTarget Model=GPT-4o2026.04 | 65.3 | 6.01 | |
| CrescendoTarget Model=GLM-4.52026.04 | 69.4 | 6.43 | |
| MASKFORGE-GoalVictim=Dream-Instruct, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Goal sampling2026.06 | 70 | 4.37 | |
| GOATTarget Model=Gemini 2.5 Pro2026.04 | 70.5 | 6.97 | |
| CIATarget Model=Deepseek V32026.04 | 71.5 | 6.96 | |
| CrescendoTarget Model=Gemini 2.5 Pro2026.04 | 73.5 | 7.05 | |
| MASKFORGE-GoalVictim=LLaDA-Instruct, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Goal sampling2026.06 | 74 | 4.78 | |
| MASKFORGE-EmbedVictim=LLaDA2.0-mini, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Embedding similarity2026.06 | 74 | 4.89 | |
| PADVictim=MMaDA-CoT2026.06 | 76 | 4.66 | |
| CrescendoTarget Model=Deepseek V32026.04 | 76.2 | 7.2 | |
| GOATTarget Model=GLM-4.52026.04 | 79.4 | 7.49 | |
| CrescendoTarget Model=Qwen 32026.04 | 80 | 7.57 | |
| GOATTarget Model=Deepseek V32026.04 | 81 | 7.86 | |
| GOATTarget Model=Qwen 32026.04 | 83.9 | 8.17 | |
| Salami AttackTarget Model=GPT-4o, Shots=5-shot2026.04 | 86.9 | 8.47 | |
| MASKFORGE-EmbedVictim=LLaDA-Instruct, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Embedding similarity2026.06 | 88 | 4.87 | |
| MASKFORGE-EmbedVictim=Dream-Instruct, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Embedding similarity2026.06 | 90 | 4.81 | |
| Salami AttackTarget Model=GPT-4o, Shots=10-shot2026.04 | 90.4 | 9.19 | |
| MASKFORGE-EmbedVictim=LLaDA 1.5, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Embedding similarity2026.06 | 91 | 4.84 | |
| Salami AttackTarget Model=Gemini 2.5 Pro, Shots=5-shot2026.04 | 91.2 | 9.06 | |
| Salami AttackTarget Model=GLM-4.5, Shots=5-shot2026.04 | 91.3 | 8.98 | |
| Salami AttackTarget Model=Gemini 2.5 Pro, Shots=10-shot2026.04 | 91.5 | 9.11 | |
| Salami AttackTarget Model=GLM-4.5, Shots=10-shot2026.04 | 93.3 | 9.16 | |
| A-SalamiTarget Model=Gemini 2.5 Pro2026.04 | 94.6 | 9.28 | |
| MASKFORGE-GoalVictim=MMaDA-CoT, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Goal sampling2026.06 | 95 | 4.96 | |
| Salami AttackTarget Model=Qwen 3, Shots=5-shot2026.04 | 95.2 | 9.36 | |
| A-SalamiTarget Model=GPT-4o2026.04 | 95.8 | 9.37 | |
| Salami AttackTarget Model=Deepseek V3, Shots=5-shot2026.04 | 96 | 9.24 | |
| Salami AttackTarget Model=Deepseek V3, Shots=10-shot2026.04 | 96.7 | 9.44 | |
| Salami AttackTarget Model=Qwen 3, Shots=10-shot2026.04 | 97.7 | 9.64 | |
| MASKFORGE-EmbedVictim=MMaDA-CoT, k=5, Pattern source=JailbreakBench/HarmBench/StrongReject, Retrieval strategy=Embedding similarity2026.06 | 98 | 5 | |
| A-SalamiTarget Model=Deepseek V32026.04 | 98.1 | 9.7 | |
| A-SalamiTarget Model=GLM-4.52026.04 | 98.8 | 9.69 | |
| A-SalamiTarget Model=Qwen 32026.04 | 99.2 | 9.71 |