Jailbreak Attack on FigStep
78.4Attack Success Rate (ASR)No Defense
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| No DefenseModel=LLaVA-1.5-7B2026.03 | 78.4 | — | — | — | — | — | 8 | |
| LLaVA-v1.6-Vicuna-7BBackbone=Vicuna-7B2026.04 | 50.8 | — | — | — | — | — | — | |
| No DefenseModel=LLaVA-v1.6-7B2026.03 | 42.6 | — | — | — | — | — | 10 | |
| No DefenseModel=Qwen2.5-VL-7B2026.03 | 35.6 | — | — | — | — | — | 14 | |
| No DefenseModel=Qwen3-VL-8B2026.03 | 32.4 | — | — | — | — | — | 12 | |
| Mistral-Medium-32026.04 | 16.2 | — | — | — | — | — | — | |
| VSFAModel=LLaVA-1.5-7B2026.03 | 14.2 | — | — | — | — | — | 45 | |
| AdaShieldModel=LLaVA-1.5-7B, Defense Protocol=Inference-time defense via safety prompting2026.03 | 12.4 | — | — | — | — | — | 2 | |
| Llama-3.2-11B-VisionParameters=11B, Type=Vision2026.04 | 11.6 | — | — | — | — | — | — | |
| VSFAModel=LLaVA-v1.6-7B2026.03 | 8.8 | — | — | — | — | — | 47 | |
| VLGuardModel=LLaVA-1.5-7B, Defense Protocol=Fine-tuned on manually labeled harmful image-text pairs2026.03 | 8.8 | — | — | — | — | — | 25 | |
| Kimi-K2.52026.04 | 7.2 | — | — | — | — | — | — | |
| VSFAModel=Qwen2.5-VL-7B2026.03 | 6.8 | — | — | — | — | — | 53 | |
| VLGuardModel=LLaVA-v1.6-7B, Defense Protocol=Fine-tuned on manually labeled harmful image-text pairs2026.03 | 6.4 | — | — | — | — | — | 28 | |
| VSFAModel=Qwen3-VL-8B2026.03 | 5.6 | — | — | — | — | — | 51 | |
| AdaShieldModel=LLaVA-v1.6-7B, Defense Protocol=Inference-time defense via safety prompting2026.03 | 5.6 | — | — | — | — | — | 3 | |
| DeepSeek-V3.22026.04 | 5 | — | — | — | — | — | — | |
| VLGuardModel=Qwen2.5-VL-7B, Defense Protocol=Fine-tuned on manually labeled harmful image-text pairs2026.03 | 4.8 | — | — | — | — | — | 34 | |
| Gemini-3-Flash2026.04 | 4.6 | — | — | — | — | — | — | |
| VLGuardModel=Qwen3-VL-8B, Defense Protocol=Fine-tuned on manually labeled harmful image-text pairs2026.03 | 3.8 | — | — | — | — | — | 32 | |
| AdaShieldModel=Qwen2.5-VL-7B, Defense Protocol=Inference-time defense via safety prompting2026.03 | 3.2 | — | — | — | — | — | 5 | |
| GPT-5-Mini2026.04 | 2.8 | — | — | — | — | — | — | |
| AdaShieldModel=Qwen3-VL-8B, Defense Protocol=Inference-time defense via safety prompting2026.03 | 2.4 | — | — | — | — | — | 4 | |
| Qwen3-VL-8B-InstructParameters=8B, Alignment=Instruct2026.04 | 2 | — | — | — | — | — | — | |
| Claude-Haiku-4.52026.04 | 1.8 | — | — | — | — | — | — | |
| Qwen3-VL-Plus2026.04 | 0 | — | — | — | — | — | — | |
| Claude-3.5-SonnetModel=Claude-3.5-Sonnet2025.12 | — | — | — | — | — | 83.4 | — | |
| Gemini-2.5-proModel=Gemini-2.5-pro2025.12 | — | — | — | — | — | 77.6 | — | |
| GPT-4oModel=GPT-4o2025.12 | — | — | — | — | — | 77 | — | |
| MousetrapModel=Claude-3-5-Sonnet, Total Samples=5002025.02 | — | 38.6 | 92.6 | 98.8 | 1.2 | — | — | |
| MSR-AlignBackbone=Qwen2.5VL-7B, Alignment Method=MSR-Align2025.12 | — | — | — | — | — | 99.6 | — | |
| MSR-AlignBackbone=Qwen2.5VL-32B, Alignment Method=MSR-Align2025.12 | — | — | — | — | — | 98.8 | — | |
| Qwen2.5VL-32BBackbone=Qwen2.5VL-32B, Alignment Method=None2025.12 | — | — | — | — | — | 72.8 | — | |
| Qwen2.5VL-7BBackbone=Qwen2.5VL-7B, Alignment Method=None2025.12 | — | — | — | — | — | 75 | — | |
| Safework-R1-7BModel=Safework-R1-7B2025.12 | — | — | — | — | — | 93.2 | — | |
| TiSBackbone=Qwen2.5VL-7B, Alignment Method=TiS2025.12 | — | — | — | — | — | 76.6 | — | |
| TiSBackbone=Qwen2.5VL-32B, Alignment Method=TiS2025.12 | — | — | — | — | — | 98.8 | — | |
| TRRBackbone=Qwen2.5VL-7B, Alignment Method=TRR2025.12 | — | — | — | — | — | 99.8 | — | |
| TRRBackbone=Qwen2.5VL-32B, Alignment Method=TRR2025.12 | — | — | — | — | — | 99.4 | — |