Multimodal Jailbreaking on VE-Safety
65ASRMM-Plan
Evaluation results
| Method | Links | |
|---|---|---|
| MM-PlanTarget Model=InternVL3-8B, Model Access Type=Open-Weight2026.02 | 65 | |
| MM-PlanTarget Model=Llama-3.2-11B, Model Access Type=Open-Weight2026.02 | 64.4 | |
| MM-PlanTarget Model=Qwen3-VL-8B, Model Access Type=Open-Weight2026.02 | 54.4 | |
| MM-PlanTarget Model=Sonnet 4.5, Model Access Type=Proprietary2026.02 | 46.3 | |
| CrescendoTarget Model=InternVL3-8B, Model Access Type=Open-Weight2026.02 | 45 | |
| FigStepTarget Model=InternVL3-8B, Model Access Type=Open-Weight2026.02 | 44.4 | |
| MM-PlanTarget Model=Gemini 2.5 Pro, Model Access Type=Proprietary2026.02 | 43.8 | |
| SSATarget Model=InternVL3-8B, Model Access Type=Open-Weight2026.02 | 39.1 | |
| MM-PlanTarget Model=GPT-4o, Model Access Type=Proprietary2026.02 | 36.9 | |
| Direct PlanTarget Model=InternVL3-8B, Model Access Type=Open-Weight2026.02 | 34.7 | |
| CrescendoTarget Model=Qwen3-VL-8B, Model Access Type=Open-Weight2026.02 | 33.8 | |
| FigStepTarget Model=Qwen3-VL-8B, Model Access Type=Open-Weight2026.02 | 33.1 | |
| SI-AttackTarget Model=InternVL3-8B, Model Access Type=Open-Weight2026.02 | 31.9 | |
| SSATarget Model=Qwen3-VL-8B, Model Access Type=Open-Weight2026.02 | 29.4 | |
| SI-AttackTarget Model=Qwen3-VL-8B, Model Access Type=Open-Weight2026.02 | 29.1 | |
| Direct RequestTarget Model=InternVL3-8B, Model Access Type=Open-Weight2026.02 | 27.2 | |
| MM-PlanTarget Model=Sonnet 3.7, Model Access Type=Proprietary2026.02 | 27.2 | |
| SI-AttackTarget Model=Llama-3.2-11B, Model Access Type=Open-Weight2026.02 | 25.6 | |
| SSATarget Model=Llama-3.2-11B, Model Access Type=Open-Weight2026.02 | 25.3 | |
| FigStepTarget Model=Sonnet 4.5, Model Access Type=Proprietary2026.02 | 24.4 | |
| FigStepTarget Model=Llama-3.2-11B, Model Access Type=Open-Weight2026.02 | 23.8 | |
| Direct PlanTarget Model=Qwen3-VL-8B, Model Access Type=Open-Weight2026.02 | 22.5 | |
| CrescendoTarget Model=Llama-3.2-11B, Model Access Type=Open-Weight2026.02 | 21.9 | |
| Direct PlanTarget Model=Llama-3.2-11B, Model Access Type=Open-Weight2026.02 | 18.1 | |
| CrescendoTarget Model=Sonnet 4.5, Model Access Type=Proprietary2026.02 | 18.1 | |
| SSATarget Model=Sonnet 4.5, Model Access Type=Proprietary2026.02 | 15.9 | |
| CrescendoTarget Model=Gemini 2.5 Pro, Model Access Type=Proprietary2026.02 | 15.9 | |
| SI-AttackTarget Model=Sonnet 4.5, Model Access Type=Proprietary2026.02 | 15.6 | |
| CrescendoTarget Model=Sonnet 3.7, Model Access Type=Proprietary2026.02 | 15 | |
| CrescendoTarget Model=GPT-4o, Model Access Type=Proprietary2026.02 | 14.4 | |
| MM-PlanTarget Model=GPT-5, Model Access Type=Proprietary2026.02 | 13.8 | |
| Direct RequestTarget Model=Llama-3.2-11B, Model Access Type=Open-Weight2026.02 | 13.4 | |
| FigStepTarget Model=Sonnet 3.7, Model Access Type=Proprietary2026.02 | 13.4 | |
| SI-AttackTarget Model=Sonnet 3.7, Model Access Type=Proprietary2026.02 | 12.8 | |
| SI-AttackTarget Model=Gemini 2.5 Pro, Model Access Type=Proprietary2026.02 | 12.5 | |
| SSATarget Model=Gemini 2.5 Pro, Model Access Type=Proprietary2026.02 | 12.2 | |
| Direct RequestTarget Model=Qwen3-VL-8B, Model Access Type=Open-Weight2026.02 | 11.9 | |
| Direct PlanTarget Model=Gemini 2.5 Pro, Model Access Type=Proprietary2026.02 | 11.9 | |
| FigStepTarget Model=Gemini 2.5 Pro, Model Access Type=Proprietary2026.02 | 11.3 | |
| Direct RequestTarget Model=Gemini 2.5 Pro, Model Access Type=Proprietary2026.02 | 9.7 | |
| Direct PlanTarget Model=Sonnet 4.5, Model Access Type=Proprietary2026.02 | 9.7 | |
| SSATarget Model=Sonnet 3.7, Model Access Type=Proprietary2026.02 | 9.7 | |
| Direct PlanTarget Model=GPT-4o, Model Access Type=Proprietary2026.02 | 9.4 | |
| Direct RequestTarget Model=Sonnet 4.5, Model Access Type=Proprietary2026.02 | 8.4 | |
| Direct PlanTarget Model=Sonnet 3.7, Model Access Type=Proprietary2026.02 | 8.1 | |
| SI-AttackTarget Model=GPT-4o, Model Access Type=Proprietary2026.02 | 8.1 | |
| FigStepTarget Model=GPT-4o, Model Access Type=Proprietary2026.02 | 6.6 | |
| SSATarget Model=GPT-4o, Model Access Type=Proprietary2026.02 | 6.3 | |
| Direct RequestTarget Model=GPT-4o, Model Access Type=Proprietary2026.02 | 5 | |
| Direct RequestTarget Model=Sonnet 3.7, Model Access Type=Proprietary2026.02 | 4.7 | |
| CrescendoTarget Model=GPT-5, Model Access Type=Proprietary2026.02 | 3.1 | |
| SI-AttackTarget Model=GPT-5, Model Access Type=Proprietary2026.02 | 1.9 | |
| SSATarget Model=GPT-5, Model Access Type=Proprietary2026.02 | 1.6 | |
| Direct PlanTarget Model=GPT-5, Model Access Type=Proprietary2026.02 | 0.9 | |
| Direct RequestTarget Model=GPT-5, Model Access Type=Proprietary2026.02 | 0.6 | |
| FigStepTarget Model=GPT-5, Model Access Type=Proprietary2026.02 | 0.6 |