Multimodal Jailbreak Attack on HarmBench
0ASRFigStep
Evaluation Results
| Method | Links | |
|---|---|---|
| FigStepEvaluation Model=HarmBench, Target Model=GPT-4o2025.10 | 0 | |
| FigStepEvaluation Model=GPT-4o-mini, Target Model=GPT-4o2025.10 | 0 | |
| FigStepEvaluation Model=Average, Target Model=GPT-4o2025.10 | 0 | |
| HADESEvaluation Model=HarmBench, Target Model=GPT-4o-mini2025.10 | 3.5 | |
| HADESEvaluation Model=HarmBench, Target Model=GPT-4o2025.10 | 3.5 | |
| HADESEvaluation Model=Average, Target Model=GPT-4o-mini2025.10 | 3.75 | |
| HADESEvaluation Model=GPT-4o-mini, Target Model=GPT-4o-mini2025.10 | 4 | |
| HADESEvaluation Model=Average, Target Model=GPT-4o2025.10 | 4 | |
| HADESEvaluation Model=GPT-4o-mini, Target Model=GPT-4o2025.10 | 4.5 | |
| FigStepEvaluation Model=GPT-4o-mini, Target Model=GPT-4o-mini2025.10 | 8 | |
| FigStepEvaluation Model=Average, Target Model=GPT-4o-mini2025.10 | 9 | |
| CS-DJEvaluation Model=HarmBench, Target Model=GPT-4o2025.10 | 9.5 | |
| Direct RequestType=Single-turn, Target Model=Sonnet 4.52026.02 | 10 | |
| FigStepEvaluation Model=HarmBench, Target Model=GPT-4o-mini2025.10 | 10 | |
| FigStepEvaluation Model=HarmBench, Target Model=Qwen2.5-VL-7B2025.10 | 13 | |
| Direct PlanType=Multi-turn, Target Model=Sonnet 4.52026.02 | 13.3 | |
| CS-DJEvaluation Model=Average, Target Model=GPT-4o2025.10 | 14 | |
| Direct RequestType=Single-turn, Target Model=Qwen3-VL-8B2026.02 | 16.7 | |
| FigStepType=Single-turn, Target Model=Sonnet 4.52026.02 | 16.7 | |
| SI-AttackType=Single-turn, Target Model=Sonnet 4.52026.02 | 16.7 | |
| CS-DJEvaluation Model=GPT-4o-mini, Target Model=GPT-4o2025.10 | 18.5 | |
| SSAType=Multi-turn, Target Model=Sonnet 4.52026.02 | 20 | |
| CS-DJEvaluation Model=HarmBench, Target Model=GPT-4o-mini2025.10 | 20.5 | |
| FigStepEvaluation Model=Average, Target Model=Qwen2.5-VL-7B2025.10 | 21.5 | |
| Direct PlanType=Multi-turn, Target Model=Qwen3-VL-8B2026.02 | 23.3 | |
| CrescendoType=Multi-turn, Target Model=Sonnet 4.52026.02 | 23.3 | |
| SI-AttackType=Single-turn, Target Model=Qwen3-VL-8B2026.02 | 30 | |
| FigStepEvaluation Model=GPT-4o-mini, Target Model=Qwen2.5-VL-7B2025.10 | 30 | |
| CS-DJEvaluation Model=Average, Target Model=GPT-4o-mini2025.10 | 30.75 | |
| FigStepType=Single-turn, Target Model=Qwen3-VL-8B2026.02 | 33.3 | |
| SSAType=Multi-turn, Target Model=Qwen3-VL-8B2026.02 | 33.3 | |
| MM-PlanType=Multi-turn, Target Model=Sonnet 4.52026.02 | 36.7 | |
| CrescendoType=Multi-turn, Target Model=Qwen3-VL-8B2026.02 | 40 | |
| CS-DJEvaluation Model=GPT-4o-mini, Target Model=GPT-4o-mini2025.10 | 41 | |
| HADESEvaluation Model=HarmBench, Target Model=Qwen2.5-VL-7B2025.10 | 45.5 | |
| HADESEvaluation Model=Average, Target Model=Qwen2.5-VL-7B2025.10 | 46.75 | |
| HADESEvaluation Model=GPT-4o-mini, Target Model=Qwen2.5-VL-7B2025.10 | 48 | |
| HADESEvaluation Model=HarmBench, Target Model=InternVL3-8B2025.10 | 50.5 | |
| CS-DJEvaluation Model=HarmBench, Target Model=Qwen2.5-VL-7B2025.10 | 50.5 | |
| HADESEvaluation Model=Average, Target Model=InternVL3-8B2025.10 | 51.5 | |
| HADESEvaluation Model=GPT-4o-mini, Target Model=InternVL3-8B2025.10 | 52.5 | |
| CS-DJEvaluation Model=Average, Target Model=Qwen2.5-VL-7B2025.10 | 53 | |
| CS-DJEvaluation Model=HarmBench, Target Model=InternVL3-8B2025.10 | 54 | |
| CS-DJEvaluation Model=GPT-4o-mini, Target Model=Qwen2.5-VL-7B2025.10 | 55.5 | |
| FigStepEvaluation Model=HarmBench, Target Model=InternVL3-8B2025.10 | 58.5 | |
| CS-DJEvaluation Model=Average, Target Model=InternVL3-8B2025.10 | 59.5 | |
| FigStepEvaluation Model=Average, Target Model=InternVL3-8B2025.10 | 59.75 | |
| VERA-VEvaluation Model=HarmBench, Target Model=GPT-4o-mini2025.10 | 60 | |
| VERA-VEvaluation Model=Average, Target Model=GPT-4o-mini2025.10 | 60.5 | |
| FigStepEvaluation Model=GPT-4o-mini, Target Model=InternVL3-8B2025.10 | 61 | |
| VERA-VEvaluation Model=GPT-4o-mini, Target Model=GPT-4o-mini2025.10 | 61 | |
| MM-PlanType=Multi-turn, Target Model=Qwen3-VL-8B2026.02 | 63.3 | |
| CS-DJEvaluation Model=GPT-4o-mini, Target Model=InternVL3-8B2025.10 | 65 | |
| VERA-VEvaluation Model=HarmBench, Target Model=GPT-4o2025.10 | 65 | |
| VERA-VEvaluation Model=Average, Target Model=GPT-4o2025.10 | 67.75 | |
| VERA-VEvaluation Model=GPT-4o-mini, Target Model=GPT-4o2025.10 | 70.5 | |
| VERA-VEvaluation Model=GPT-4o-mini, Target Model=Qwen2.5-VL-7B2025.10 | 71 | |
| VERA-VEvaluation Model=Average, Target Model=Qwen2.5-VL-7B2025.10 | 72 | |
| VERA-VEvaluation Model=HarmBench, Target Model=Qwen2.5-VL-7B2025.10 | 73 | |
| VERA-VEvaluation Model=HarmBench, Target Model=InternVL3-8B2025.10 | 74.5 | |
| VERA-VEvaluation Model=Average, Target Model=InternVL3-8B2025.10 | 76.5 | |
| VERA-VEvaluation Model=GPT-4o-mini, Target Model=InternVL3-8B2025.10 | 78.5 |