Object Visual Question Answering (Multiple Choice) on RIO-Bench Obj-VQA (MC) (Attack)
99Accuracy (Average)IT-RT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| IT-RTBackbone=Qwen-2.5-VL-7B, Category=text-agnostic2025.12 | 99 | 98.5 | 98.7 | 99.7 | |
| IT-RTBackbone=Qwen2.5-VL-7B, Training Strategy=IT-RT2025.12 | 99 | — | — | — | |
| IT-RTBackbone=Llama-3.2-Vision-11B, Category=text-agnostic2025.12 | 98.8 | 98.5 | 98.5 | 99.3 | |
| IT-RTBackbone=Llama-3.2-11B-Vision, Training Strategy=IT-RT2025.12 | 98.8 | — | — | — | |
| IT-RTBackbone=Qwen-3-VL-7B, Category=text-agnostic2025.12 | 98.7 | 98.2 | 98.4 | 99.5 | |
| RIO-RTBackbone=Qwen-3-VL-7B, Category=selective2025.12 | 98.7 | 98.3 | 98.4 | 99.4 | |
| RIO-RTBackbone=Llama-3.2-Vision-11B, Category=selective2025.12 | 98.6 | 98.5 | 98.3 | 99.1 | |
| RIO-RTBackbone=Llama-3.2-11B-Vision, Training Strategy=RIO-RT2025.12 | 98.6 | — | — | — | |
| RIO-RTBackbone=Qwen-2.5-VL-7B, Category=selective2025.12 | 98.5 | 97.9 | 98.4 | 99.2 | |
| RIO-RTBackbone=Qwen2.5-VL-7B, Training Strategy=RIO-RT2025.12 | 98.5 | — | — | — | |
| IT-RTBackbone=Llava-1.5-7B, Category=text-agnostic2025.12 | 98.1 | 97.6 | 97.7 | 98.9 | |
| IT-RTBackbone=LLaVA-1.5-7B, Training Strategy=IT-RT2025.12 | 98.1 | — | — | — | |
| RIO-RTBackbone=Llava-1.5-7B, Category=selective2025.12 | 97.9 | 97.3 | 97.6 | 98.6 | |
| RIO-RTBackbone=LLaVA-1.5-7B, Training Strategy=RIO-RT2025.12 | 97.9 | — | — | — | |
| IT-RTBackbone=Llava-1.5-13B, Category=text-agnostic2025.12 | 97.6 | 97.3 | 97.5 | 97.9 | |
| IT-RTBackbone=LLaVA-1.5-13B, Training Strategy=IT-RT2025.12 | 97.6 | — | — | — | |
| RIO-RTBackbone=Llava-1.5-13B, Category=selective2025.12 | 97 | 96.9 | 96.8 | 97.1 | |
| RIO-RTBackbone=LLaVA-1.5-13B, Training Strategy=RIO-RT2025.12 | 97 | — | — | — | |
| IT-RTBackbone=SmolVLM-2B, Category=text-agnostic2025.12 | 94.9 | 94.7 | 94.7 | 95.3 | |
| RIO-RTBackbone=SmolVLM-2B, Category=selective2025.12 | 93.3 | 93.8 | 94 | 92.2 | |
| CoT-defenseBackbone=Llava-1.5-13B, Category=text-agnostic2025.12 | 78.7 | 80.7 | 80.2 | 75.3 | |
| CoTBackbone=LLaVA-1.5-13B, Training Strategy=CoT2025.12 | 78.7 | — | — | — | |
| CoT-defenseBackbone=Qwen-2.5-VL-7B, Category=text-agnostic2025.12 | 76.1 | 80.8 | 77 | 70.3 | |
| CoTBackbone=Qwen2.5-VL-7B, Training Strategy=CoT2025.12 | 76.1 | — | — | — | |
| CoT-defenseBackbone=Llama-3.2-Vision-11B, Category=text-agnostic2025.12 | 73.8 | 78.2 | 75.8 | 67.3 | |
| CoTBackbone=Llama-3.2-11B-Vision, Training Strategy=CoT2025.12 | 73.8 | — | — | — | |
| pretrainedBackbone=Llava-1.5-7B2025.12 | 73.7 | 75.9 | 73.7 | 71.5 | |
| pretrainedBackbone=LLaVA-1.5-7B, Training Strategy=pretrained2025.12 | 73.7 | — | — | — | |
| pretrainedBackbone=Qwen-3-VL-7B2025.12 | 72.4 | 76.8 | 74.2 | 66.2 | |
| CoT-defenseBackbone=Llava-1.5-7B, Category=text-agnostic2025.12 | 71.4 | 72.8 | 72.1 | 69.2 | |
| CoTBackbone=LLaVA-1.5-7B, Training Strategy=CoT2025.12 | 71.4 | — | — | — | |
| pretrainedBackbone=Llava-1.5-13B2025.12 | 68.3 | 70.2 | 68.4 | 66.2 | |
| pretrainedBackbone=LLaVA-1.5-13B, Training Strategy=pretrained2025.12 | 68.3 | — | — | — | |
| pretrainedBackbone=Qwen-2.5-VL-7B2025.12 | 65.9 | 71.5 | 68.2 | 58 | |
| pretrainedBackbone=Qwen2.5-VL-7B, Training Strategy=pretrained2025.12 | 65.9 | — | — | — | |
| CoT-defenseBackbone=Qwen-3-VL-7B, Category=text-agnostic2025.12 | 64 | 66.8 | 64.8 | 60.4 | |
| pretrainedBackbone=Llama-3.2-Vision-11B2025.12 | 59.2 | 63.4 | 61 | 53.3 | |
| pretrainedBackbone=Llama-3.2-11B-Vision, Training Strategy=pretrained2025.12 | 59.2 | — | — | — | |
| CoT-defenseBackbone=SmolVLM-2B, Category=text-agnostic2025.12 | 57.6 | 60.7 | 60 | 52.2 | |
| pretrainedBackbone=SmolVLM-2B2025.12 | 51.3 | 56 | 53.1 | 44.9 |