Refusal Rate Evaluation on VQA v2
0.77Refusal RateImageProtector
Evaluation Results
| Method | Links | |
|---|---|---|
| ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 0.77 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 0.8 | |
| ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 0.83 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 0.85 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 0.86 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 0.87 | |
| ImageProtectorTarget MLLM=InstructBLIP, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 0.89 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 0.9 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 0.9 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 0.91 | |
| ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 0.91 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 0.91 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 0.92 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 0.93 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 0.93 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 0.94 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 0.94 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 0.94 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 0.94 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 0.94 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 0.95 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 0.96 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 0.99 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 1 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 88 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 88 | |
| ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 89 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 93 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 94 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 94 |