Refusal Rate Evaluation on GQA
77Refusal RateImageProtector
Evaluation Results
| Method | Links | |
|---|---|---|
| ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 77 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 81 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 82 | |
| ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 84 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 87 | |
| ImageProtectorTarget MLLM=InstructBLIP, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 87 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 90 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 90 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 91 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 92 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 92 | |
| ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 93 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 93 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 93 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 93 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 93 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 94 | |
| ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 94 | |
| ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 94 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 95 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 95 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 96 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 96 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.04 | 97 | |
| ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.04 | 97 | |
| ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.04 | 97 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 98 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Probing Question Type=exact, Question Relevance=image-irrelevant2026.04 | 100 | |
| ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 100 | |
| ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=exact, Question Relevance=image-relevant2026.04 | 100 |