Jailbreak Attack Defense on MM-SafetyBench (ASR, CS)
0.2Attack Success Rate (ASR)CASA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CASAModel=Qwen_2.5_Omni, Model Size=7B, Safety Prompt=Yes2026.03 | 0.2 | — | |
| CBModel=Qwen_2.5_Omni, Model Size=3B, Safety Prompt=Yes2026.03 | 1.2 | — | |
| CASAModel=Qwen_2.5_Omni, Model Size=3B, Safety Prompt=Yes2026.03 | 1.2 | — | |
| SSFTModel=Qwen_2.5_Omni, Model Size=7B, Safety Prompt=Yes2026.03 | 2.7 | — | |
| CBModel=Qwen_2.5_Omni, Model Size=7B, Safety Prompt=Yes2026.03 | 3.7 | — | |
| CBModel=Qwen_2.5_Omni, Model Size=3B, Safety Prompt=No2026.03 | 5.1 | — | |
| CBModel=Qwen_2.5_Omni, Model Size=7B, Safety Prompt=No2026.03 | 5.4 | — | |
| AdaShieldModel=Qwen3-VL-8B, Defense Protocol=Inference-time defense via safety prompting2026.03 | 8.57 | 3 | |
| SSFTModel=Qwen_2.5_Omni, Model Size=7B, Safety Prompt=No2026.03 | 8.8 | — | |
| CASAModel=Qwen_2.5_Omni, Model Size=7B, Safety Prompt=No2026.03 | 9 | — | |
| CASAModel=Qwen_2.5_Omni, Model Size=3B, Safety Prompt=No2026.03 | 9.2 | — | |
| AdaShieldModel=Qwen2.5-VL-7B, Defense Protocol=Inference-time defense via safety prompting2026.03 | 10.48 | 4 | |
| VLGuardModel=Qwen3-VL-8B, Defense Protocol=Fine-tuned on manually labeled harmful image-text pairs2026.03 | 10.83 | 29 | |
| CMRMModel=ShareGPT4V-7B2026.03 | 11.2 | — | |
| SSFTModel=Qwen_2.5_Omni, Model Size=3B, Safety Prompt=Yes2026.03 | 12.3 | — | |
| VLGuardModel=Qwen2.5-VL-7B, Defense Protocol=Fine-tuned on manually labeled harmful image-text pairs2026.03 | 12.56 | 31 | |
| JRS-RemModel=InternVL-Chat-19B2026.03 | 12.6 | — | |
| CMRMModel=InternVL-Chat-19B2026.03 | 12.8 | — | |
| JRS-RemModel=ShareGPT4V-7B2026.03 | 12.9 | — | |
| AdaShieldModel=ShareGPT4V-7B2026.03 | 13.7 | — | |
| VSFAModel=Qwen3-VL-8B2026.03 | 14.29 | 48 | |
| AdaShieldModel=LLaVA-v1.6-7B, Defense Protocol=Inference-time defense via safety prompting2026.03 | 14.29 | 2 | |
| SSFTModel=Qwen_2.5_Omni, Model Size=3B, Safety Prompt=No2026.03 | 14.9 | — | |
| VSFAModel=Qwen2.5-VL-7B2026.03 | 16.31 | 49 | |
| VLGuardModel=LLaVA-v1.6-7B, Defense Protocol=Fine-tuned on manually labeled harmful image-text pairs2026.03 | 16.85 | 26 | |
| VSFAModel=LLaVA-v1.6-7B2026.03 | 18.57 | 45 | |
| JRS-RemBackbone=LLaVA-1.5-7B2026.03 | 19.1 | — | |
| JRS-RemModel=LLaVA-1.5-7B2026.03 | 19.1 | — | |
| VLGuardModel=LLaVA-1.5-7B, Defense Protocol=Fine-tuned on manually labeled harmful image-text pairs2026.03 | 20.48 | 23 | |
| AdaShieldModel=LLaVA-1.5-7B, Defense Protocol=Inference-time defense via safety prompting2026.03 | 22.56 | 2 | |
| Pre-TrainedModel=Qwen_2.5_Omni, Model Size=7B, Safety Prompt=Yes2026.03 | 22.6 | — | |
| VSFAModel=LLaVA-1.5-7B2026.03 | 24.64 | 42 | |
| ECSOModel=ShareGPT4V-7B2026.03 | 25.2 | — | |
| Pre-TrainedModel=Qwen_2.5_Omni, Model Size=3B, Safety Prompt=Yes2026.03 | 25.8 | — | |
| ECSOBackbone=LLaVA-1.5-7B2026.03 | 26.8 | — | |
| ECSOModel=LLaVA-1.5-7B2026.03 | 26.8 | — | |
| AdaShieldModel=InternVL-Chat-19B2026.03 | 27.1 | — | |
| ShiftDCModel=ShareGPT4V-7B2026.03 | 28.1 | — | |
| ShiftDCModel=InternVL-Chat-19B2026.03 | 29.6 | — | |
| CMRMBackbone=LLaVA-1.5-7B2026.03 | 30.1 | — | |
| CMRMModel=LLaVA-1.5-7B2026.03 | 30.1 | — | |
| ECSOModel=InternVL-Chat-19B2026.03 | 33.6 | — | |
| Pre-TrainedModel=Qwen_2.5_Omni, Model Size=3B, Safety Prompt=No2026.03 | 37.7 | — | |
| Pre-TrainedModel=Qwen_2.5_Omni, Model Size=7B, Safety Prompt=No2026.03 | 38.1 | — | |
| ShiftDCBackbone=LLaVA-1.5-7B2026.03 | 38.3 | — | |
| ShiftDCModel=LLaVA-1.5-7B2026.03 | 38.3 | — | |
| No DefenseModel=Qwen3-VL-8B2026.03 | 38.63 | 9 | |
| No DefenseModel=Qwen2.5-VL-7B2026.03 | 42.26 | 11 | |
| AdaShieldBackbone=LLaVA-1.5-7B2026.03 | 45.8 | — | |
| AdaShieldModel=LLaVA-1.5-7B2026.03 | 45.8 | — | |
| InternVL-Chat-19BDefense=None2026.03 | 47.7 | — | |
| No DefenseModel=LLaVA-v1.6-7B2026.03 | 48.63 | 8 | |
| No DefenseModel=LLaVA-1.5-7B2026.03 | 62.26 | 6 | |
| ShareGPT4V-7BDefense=None2026.03 | 62.3 | — | |
| LLaVA-1.5-7BDefense=None2026.03 | 67.2 | — | |
| LLaVA-1.5-7BDefense=None2026.03 | 67.2 | — |