Object Visual Question Answering (Open Ended) on RIO-Bench Obj-VQA (OE) Attack
60.1Accuracy (Easy)IT-RT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| IT-RTBackbone=Llama-3.2-Vision-11B, Category=text-agnostic2025.12 | 60.1 | 58.9 | 44.6 | 54.5 | |
| RIO-RTBackbone=Qwen-2.5-VL-7B, Category=selective2025.12 | 59.3 | 58.4 | 44.2 | 53.9 | |
| IT-RTBackbone=Qwen-3-VL-7B, Category=text-agnostic2025.12 | 59.3 | 58 | 44.8 | 54 | |
| IT-RTBackbone=Qwen-2.5-VL-7B, Category=text-agnostic2025.12 | 59.2 | 58.1 | 44 | 53.7 | |
| RIO-RTBackbone=Llama-3.2-Vision-11B, Category=selective2025.12 | 59 | 57.5 | 43.2 | 53.2 | |
| IT-RTBackbone=Llava-1.5-7B, Category=text-agnostic2025.12 | 58.7 | 57.8 | 43.6 | 53.4 | |
| RIO-RTBackbone=Llava-1.5-7B, Category=selective2025.12 | 58.7 | 56.9 | 43.1 | 52.9 | |
| RIO-RTBackbone=Qwen-3-VL-7B, Category=selective2025.12 | 58 | 56.9 | 43.5 | 52.8 | |
| IT-RTBackbone=Llava-1.5-13B, Category=text-agnostic2025.12 | 57.8 | 56.2 | 41.2 | 51.7 | |
| RIO-RTBackbone=Llava-1.5-13B, Category=selective2025.12 | 56.8 | 55.2 | 40.8 | 50.9 | |
| IT-RTBackbone=SmolVLM-2B, Category=text-agnostic2025.12 | 53.2 | 48.9 | 36 | 46 | |
| RIO-RTBackbone=SmolVLM-2B, Category=selective2025.12 | 51 | 47.8 | 32.1 | 43.6 | |
| CoT-defenseBackbone=Llava-1.5-13B, Category=text-agnostic2025.12 | 50.9 | 48.2 | 34.8 | 44.6 | |
| CoT-defenseBackbone=Llava-1.5-7B, Category=text-agnostic2025.12 | 50.4 | 47.3 | 34.1 | 43.9 | |
| pretrainedBackbone=Llava-1.5-13B2025.12 | 49.2 | 45.9 | 31.9 | 42.3 | |
| CoT-defenseBackbone=SmolVLM-2B, Category=text-agnostic2025.12 | 45.2 | 43.5 | 33.4 | 40.7 | |
| CoT-defenseBackbone=Qwen-2.5-VL-7B, Category=text-agnostic2025.12 | 42.8 | 41.2 | 29.1 | 37.7 | |
| pretrainedBackbone=Llava-1.5-7B2025.12 | 42 | 39.1 | 24.9 | 35.3 | |
| CoT-defenseBackbone=Llama-3.2-Vision-11B, Category=text-agnostic2025.12 | 40.4 | 36.5 | 22.1 | 33 | |
| pretrainedBackbone=Qwen-3-VL-7B2025.12 | 38.9 | 33.5 | 15.8 | 29.4 | |
| pretrainedBackbone=SmolVLM-2B2025.12 | 38.9 | 35.5 | 21.3 | 31.9 | |
| CoT-defenseBackbone=Qwen-3-VL-7B, Category=text-agnostic2025.12 | 38.4 | 34.6 | 22.3 | 31.8 | |
| pretrainedBackbone=Qwen-2.5-VL-7B2025.12 | 35.3 | 28.6 | 14 | 26 | |
| pretrainedBackbone=Llama-3.2-Vision-11B2025.12 | 19.2 | 16.5 | 3.2 | 13 |