List your company
ResearchFormerly WizWand

Refusal Rate Evaluation on VQA v2

0.77Refusal RateImageProtector
-2.959222.212947.38572.5571Apr 10, 2026
Updated 7mo ago

Evaluation results

MethodLinks
ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.040.77
ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.040.8
ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.040.83
ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.040.85
ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=exact, Question Relevance=image-relevant2026.040.86
ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.040.87
ImageProtectorTarget MLLM=InstructBLIP, Probing Question Type=exact, Question Relevance=image-irrelevant2026.040.89
ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.040.9
ImageProtectorTarget MLLM=MiniGPT-4, Probing Question Type=exact, Question Relevance=image-irrelevant2026.040.9
ImageProtectorTarget MLLM=LLaVA-1.5, Probing Question Type=exact, Question Relevance=image-irrelevant2026.040.91
ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=exact, Question Relevance=image-relevant2026.040.91
ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.040.91
ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.040.92
ImageProtectorTarget MLLM=Qwen-VL-Chat, Probing Question Type=exact, Question Relevance=image-irrelevant2026.040.93
ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.040.93
ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=general probing questions, Question Relevance=image-irrelevant2026.040.94
ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=exact, Question Relevance=image-relevant2026.040.94
ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=exact, Question Relevance=image-relevant2026.040.94
ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.040.94
ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=similar, Question Relevance=image-irrelevant2026.040.94
ImageProtectorTarget MLLM=Qwen2.5-VL, Probing Question Type=exact, Question Relevance=image-irrelevant2026.040.95
ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=exact, Question Relevance=image-relevant2026.040.96
ImageProtectorTarget MLLM=Phi-4-multimodal, Probing Question Type=exact, Question Relevance=image-irrelevant2026.040.99
ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=exact, Question Relevance=image-relevant2026.041
ImageProtectorTarget MLLM=LLaVA-1.5, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.0488
ImageProtectorTarget MLLM=MiniGPT-4, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.0488
ImageProtectorTarget MLLM=InstructBLIP, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.0489
ImageProtectorTarget MLLM=Phi-4-multimodal, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.0493
ImageProtectorTarget MLLM=Qwen-VL-Chat, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.0494
ImageProtectorTarget MLLM=Qwen2.5-VL, Shadow Question Type=Similar probing questions, Question Relevance=Image-relevant2026.0494