Text Visual Question Answering (Open Ended) on RIO-Bench Text-VQA (OE) Attack
85.3Acc (AVG)RIO-RT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RIO-RTBackbone=Qwen-2.5-VL-7B, Category=selective2025.12 | 85.3 | 85.2 | 85.5 | |
| RIO-RTBackbone=Qwen2.5-VL-7B, Training Strategy=RIO-RT2025.12 | 85.3 | — | — | |
| RIO-RTBackbone=Qwen-3-VL-7B, Category=selective2025.12 | 85.1 | 85.1 | 85.1 | |
| RIO-RTBackbone=Llama-3.2-Vision-11B, Category=selective2025.12 | 81.7 | 81.7 | 81.7 | |
| RIO-RTBackbone=Llama-3.2-11B-Vision, Training Strategy=RIO-RT2025.12 | 81.7 | — | — | |
| pretrainedBackbone=Qwen-3-VL-7B2025.12 | 78.9 | 79.3 | 78.6 | |
| pretrainedBackbone=Qwen-2.5-VL-7B2025.12 | 78.1 | 78.9 | 77.4 | |
| pretrainedBackbone=Qwen2.5-VL-7B, Training Strategy=pretrained2025.12 | 78.1 | — | — | |
| IT-RTBackbone=Qwen-2.5-VL-7B, Category=text-agnostic2025.12 | 72.8 | 72.9 | 72.7 | |
| IT-RTBackbone=Qwen2.5-VL-7B, Training Strategy=IT-RT2025.12 | 72.8 | — | — | |
| CoT-defenseBackbone=Qwen-3-VL-7B, Category=text-agnostic2025.12 | 70.3 | 70.4 | 70.1 | |
| CoT-defenseBackbone=Qwen-2.5-VL-7B, Category=text-agnostic2025.12 | 68.4 | 69.1 | 67.7 | |
| CoTBackbone=Qwen2.5-VL-7B, Training Strategy=CoT2025.12 | 68.4 | — | — | |
| RIO-RTBackbone=SmolVLM-2B, Category=selective2025.12 | 66.5 | 66.7 | 66.4 | |
| pretrainedBackbone=SmolVLM-2B2025.12 | 66.3 | 66.5 | 66 | |
| CoT-defenseBackbone=SmolVLM-2B, Category=text-agnostic2025.12 | 64.6 | 65 | 64.2 | |
| CoT-defenseBackbone=Llama-3.2-Vision-11B, Category=text-agnostic2025.12 | 63 | 63.6 | 62.4 | |
| CoTBackbone=Llama-3.2-11B-Vision, Training Strategy=CoT2025.12 | 63 | — | — | |
| IT-RTBackbone=Qwen-3-VL-7B, Category=text-agnostic2025.12 | 60.9 | 61.3 | 60.6 | |
| pretrainedBackbone=Llama-3.2-Vision-11B2025.12 | 56.2 | 57 | 55.4 | |
| pretrainedBackbone=Llama-3.2-11B-Vision, Training Strategy=pretrained2025.12 | 56.2 | — | — | |
| RIO-RTBackbone=Llava-1.5-7B, Category=selective2025.12 | 53.3 | 53.6 | 53.1 | |
| RIO-RTBackbone=LLaVA-1.5-7B, Training Strategy=RIO-RT2025.12 | 53.3 | — | — | |
| RIO-RTBackbone=Llava-1.5-13B, Category=selective2025.12 | 50.5 | 50.9 | 50 | |
| RIO-RTBackbone=LLaVA-1.5-13B, Training Strategy=RIO-RT2025.12 | 50.5 | — | — | |
| pretrainedBackbone=Llava-1.5-13B2025.12 | 49.7 | 51.1 | 48.2 | |
| pretrainedBackbone=LLaVA-1.5-13B, Training Strategy=pretrained2025.12 | 49.7 | — | — | |
| CoT-defenseBackbone=Llava-1.5-13B, Category=text-agnostic2025.12 | 46.2 | 46.9 | 45.4 | |
| CoTBackbone=LLaVA-1.5-13B, Training Strategy=CoT2025.12 | 46.2 | — | — | |
| pretrainedBackbone=Llava-1.5-7B2025.12 | 45.1 | 46.2 | 44.1 | |
| pretrainedBackbone=LLaVA-1.5-7B, Training Strategy=pretrained2025.12 | 45.1 | — | — | |
| IT-RTBackbone=Llama-3.2-Vision-11B, Category=text-agnostic2025.12 | 44.4 | 44.7 | 44.1 | |
| IT-RTBackbone=Llama-3.2-11B-Vision, Training Strategy=IT-RT2025.12 | 44.4 | — | — | |
| CoT-defenseBackbone=Llava-1.5-7B, Category=text-agnostic2025.12 | 42.1 | 43.3 | 41 | |
| CoTBackbone=LLaVA-1.5-7B, Training Strategy=CoT2025.12 | 42.1 | — | — | |
| IT-RTBackbone=SmolVLM-2B, Category=text-agnostic2025.12 | 28.3 | 28.5 | 28.2 | |
| IT-RTBackbone=Llava-1.5-7B, Category=text-agnostic2025.12 | 22.6 | 24.2 | 21.1 | |
| IT-RTBackbone=LLaVA-1.5-7B, Training Strategy=IT-RT2025.12 | 22.6 | — | — | |
| IT-RTBackbone=Llava-1.5-13B, Category=text-agnostic2025.12 | 12 | 12.6 | 11.4 | |
| IT-RTBackbone=LLaVA-1.5-13B, Training Strategy=IT-RT2025.12 | 12 | — | — |