Multimodal Prompt Injection Attack on TextVQA
88.24Attack Success Rate (ASR)CrossMPI
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CrossMPITarget Model=BLIVA, Best Source Model=InstructBLIP2026.05 | 88.24 | 53.51 | |
| CrossMPITarget Model=InstructBLIP, Best Source Model=MiniGPT4-vicuna2026.05 | 82.35 | 44.93 | |
| CrossMPITarget Model=BLIP-2, Best Source Model=MiniGPT4-vicuna2026.05 | 81.37 | 54.26 | |
| CITarget Model=InstructBLIP, Best Source Model=MiniGPT4-vicuna2026.05 | 77.08 | 64.74 | |
| CITarget Model=BLIVA, Best Source Model=InstructBLIP2026.05 | 77.08 | 47.36 | |
| CITarget Model=BLIP-2, Best Source Model=MiniGPT4-vicuna2026.05 | 70.83 | 45.87 | |
| CrossMPITarget Model=MiniGPT4-llama2, Best Source Model=MiniGPT4-vicuna2026.05 | 52.94 | 51.89 | |
| CrossMPITarget Model=MiniGPT4-vicuna, Best Source Model=MiniGPT4-llama22026.05 | 51.82 | 51.51 | |
| CITarget Model=MiniGPT4-llama2, Best Source Model=MiniGPT4-vicuna2026.05 | 41.67 | 55.34 | |
| ARE-BTarget Model=MiniGPT4-vicuna, Best Source Model=MiniGPT4-llama22026.05 | 35.29 | 48.42 | |
| CITarget Model=MiniGPT4-vicuna, Best Source Model=MiniGPT4-llama22026.05 | 31.25 | 43.65 | |
| ARE-BTarget Model=InstructBLIP, Best Source Model=MiniGPT4-vicuna2026.05 | 25.49 | 42.3 | |
| ARE-BTarget Model=BLIVA, Best Source Model=InstructBLIP2026.05 | 23.53 | 43.92 | |
| CrossMPITarget Model=Qwen2.5-VL, Best Source Model=Qwen2.5-VL2026.05 | 20.59 | 37.99 | |
| ARE-WTarget Model=InstructBLIP, Best Source Model=MiniGPT4-vicuna2026.05 | 20.41 | 39.04 | |
| ARE-WTarget Model=BLIVA, Best Source Model=InstructBLIP2026.05 | 18.18 | 39.16 | |
| ARE-BTarget Model=MiniGPT4-llama2, Best Source Model=MiniGPT4-vicuna2026.05 | 17.65 | 45.12 | |
| ARE-WTarget Model=MiniGPT4-vicuna, Best Source Model=MiniGPT4-llama22026.05 | 14.29 | 40.73 | |
| ARE-BTarget Model=BLIP-2, Best Source Model=MiniGPT4-vicuna2026.05 | 13.73 | 26.19 | |
| ARE-WTarget Model=BLIP-2, Best Source Model=MiniGPT4-vicuna2026.05 | 12.25 | 29.5 | |
| ATPITarget Model=MiniGPT4-llama2, Best Source Model=MiniGPT4-vicuna2026.05 | 9.3 | 41.53 | |
| ARE-BTarget Model=Qwen2.5-VL, Best Source Model=Qwen2.5-VL2026.05 | 7.84 | 46.28 | |
| ARE-WTarget Model=MiniGPT4-llama2, Best Source Model=MiniGPT4-vicuna2026.05 | 7.69 | 41.43 | |
| ATPITarget Model=InstructBLIP, Best Source Model=MiniGPT4-vicuna2026.05 | 6.98 | 37.09 | |
| CITarget Model=Qwen2.5-VL, Best Source Model=Qwen2.5-VL2026.05 | 6.25 | 42.25 | |
| ATPITarget Model=BLIVA, Best Source Model=InstructBLIP2026.05 | 5.13 | 30.7 | |
| ATPITarget Model=MiniGPT4-vicuna, Best Source Model=MiniGPT4-llama22026.05 | 4.88 | 34.28 | |
| ATPITarget Model=BLIP-2, Best Source Model=MiniGPT4-vicuna2026.05 | 4.65 | 18.09 | |
| ARE-WTarget Model=Qwen2.5-VL, Best Source Model=Qwen2.5-VL2026.05 | 3.03 | 42.33 | |
| ATPITarget Model=Qwen2.5-VL, Best Source Model=Qwen2.5-VL2026.05 | 2.44 | 37.4 |