Multimodal Large Language Model Safety Evaluation on MM-SafetyBench++
100Illegal Activity Unsafe Refusal RateGPT-5
Evaluation Results
| Method | Links | ||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5Model Category=Proprietary, Judge Model=gpt-5-mini2026.03 | 100 | 5 | 99 | 4.9 | 99.5 | 5 | 97.6 | 4.9 | 100 | 4.9 | 99 | 4.9 | 97.7 | 4.9 | 100 | 4.9 | 98.9 | 4.9 | 97.6 | 4.9 | 100 | 4.9 | 99 | 4.9 | 100 | 4.9 | 99.1 | 4.9 | 99.4 | 4.9 | 73.4 | 3.6 | 100 | 4.9 | 84.6 | 4.2 | |
| LLaVA-1.5-7B + Post-hoc LoRAModel Category=Safety Fine-Tuned, Backbone=LLaVA-1.5-7B, Fine-tuning Strategy=Post-hoc LoRA2026.03 | 100 | 4 | 6.2 | 0.2 | 11.7 | 0.4 | 100 | 4 | 4.3 | 0.1 | 8.3 | 0.2 | 100 | 4 | 2.3 | 0.1 | 4.5 | 0.2 | 100 | 4 | 0 | 0 | 0 | 0 | 100 | 4 | 1.3 | 0 | 2.6 | 0 | 100 | 3.9 | 4.6 | 0.2 | 8.8 | 0.4 | |
| LLaVA-1.5-7B + Mixed LoRAModel Category=Safety Fine-Tuned, Backbone=LLaVA-1.5-7B, Fine-tuning Strategy=Mixed LoRA2026.03 | 100 | 4 | 3.1 | 0.1 | 6 | 0.2 | 100 | 4 | 4.3 | 0.1 | 8.3 | 0.2 | 100 | 4 | 0 | 0 | 0 | 0 | 100 | 4 | 2.1 | 0.1 | 4.1 | 0.2 | 100 | 4 | 1.3 | 0 | 2.6 | 0 | 100 | 3.8 | 3.7 | 0.1 | 7.1 | 0.2 | |
| GPT-4o-MiniModel Category=Proprietary, Judge Model=gpt-5-mini2026.03 | 97.9 | 1.1 | 90.7 | 3.7 | 94.1 | 1.7 | 82.2 | 1.2 | 96.3 | 4.1 | 88.7 | 1.9 | 81.8 | 0.9 | 97.7 | 3.8 | 89 | 1.5 | 76.4 | 0.8 | 91 | 3.7 | 83.1 | 1.3 | 83.1 | 1 | 96.8 | 4 | 89.4 | 1.6 | 46.8 | 0.9 | 89.9 | 3.4 | 61.6 | 1.4 | |
| Qwen2-VL-7BModel Category=Open-Source, Judge Model=gpt-5-mini2026.03 | 96.9 | 4.7 | 100 | 2.6 | 98.4 | 3.4 | 87.1 | 4 | 99.4 | 2.7 | 92.9 | 3.2 | 86.4 | 4 | 100 | 2.6 | 92.7 | 3.2 | 79.9 | 3.7 | 99.3 | 2.6 | 88.4 | 3 | 95.5 | 4.6 | 100 | 2.6 | 97.7 | 3.3 | 47.7 | 2 | 87.2 | 2.2 | 61.7 | 2.1 | |
| InternVL2-8BModel Category=Open-Source, Judge Model=gpt-5-mini2026.03 | 76.3 | 2.7 | 100 | 3.7 | 86.6 | 3.1 | 66.9 | 2.6 | 100 | 4.1 | 79.7 | 3.2 | 34.1 | 1 | 95.5 | 3.4 | 50 | 1.6 | 45.8 | 1.6 | 99.3 | 3.7 | 63.6 | 2.3 | 60.4 | 2.4 | 100 | 3.9 | 75.3 | 3 | 21.1 | 0.7 | 99.1 | 3.5 | 34.7 | 1.1 | |
| Qwen2.5-VL-7BModel Category=Open-Source, Judge Model=gpt-5-mini2026.03 | 38.1 | 1.9 | 100 | 3.8 | 55.2 | 2.5 | 51.5 | 2.5 | 100 | 4 | 68 | 3.1 | 4.6 | 0.2 | 100 | 3 | 8.8 | 0.4 | 20.1 | 1 | 100 | 3.9 | 33.5 | 1.6 | 29.9 | 1.4 | 100 | 3.8 | 46 | 2 | 25.7 | 1.1 | 99.1 | 3.5 | 40.8 | 1.7 | |
| LLaVA-NeXT-7BModel Category=Open-Source, Judge Model=gpt-5-mini2026.03 | 8.3 | 0.4 | 100 | 3.4 | 15.3 | 0.7 | 23.9 | 1.1 | 100 | 3.8 | 38.6 | 1.7 | 4.6 | 0.2 | 100 | 3.1 | 8.8 | 0.4 | 4.2 | 0.2 | 100 | 3.5 | 8 | 0.4 | 3.9 | 0.2 | 100 | 3.6 | 7.5 | 0.4 | 11.9 | 0.5 | 100 | 3.4 | 21.4 | 0.9 | |
| LLaVA-1.5-7BModel Category=Open-Source, Judge Model=gpt-5-mini2026.03 | 5.2 | 0.3 | 100 | 3.1 | 9.9 | 0.6 | 17.8 | 0.8 | 99.4 | 3.4 | 30.1 | 1.2 | 4.6 | 0.2 | 100 | 2.8 | 8.8 | 0.4 | 4.2 | 0.2 | 100 | 3.1 | 8 | 0.4 | 4.6 | 0.2 | 100 | 3.1 | 8.8 | 0.4 | 10.1 | 0.4 | 100 | 3.1 | 18.4 | 0.7 |