Object-centric Typographic Robustness (Binary Choice) on Typo-D Attack
99.5Character Robustness ScoreIT-RT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| IT-RTBackbone=Qwen2.5-VL-7B, Training Strategy=IT-RT2025.12 | 99.5 | 96.8 | 95.5 | 100 | 97.9 | |
| RIO-RTBackbone=Qwen2.5-VL-7B, Training Strategy=RIO-RT2025.12 | 99 | 97 | 96.6 | 99.6 | 98 | |
| IT-RTBackbone=Llama-3.2-11B-Vision, Training Strategy=IT-RT2025.12 | 98.5 | 92.3 | 97.4 | 98.5 | 96.7 | |
| RIO-RTBackbone=Llama-3.2-11B-Vision, Training Strategy=RIO-RT2025.12 | 98.5 | 91.5 | 98.2 | 99.6 | 96.9 | |
| RIO-RTBackbone=LLaVA-1.5-7B, Training Strategy=RIO-RT2025.12 | 94.9 | 87.3 | 87.1 | 97.7 | 91.7 | |
| CoTBackbone=Qwen2.5-VL-7B, Training Strategy=CoT2025.12 | 91.8 | 70.1 | 90.5 | 94.3 | 86.7 | |
| CoTBackbone=Llama-3.2-11B-Vision, Training Strategy=CoT2025.12 | 88.2 | 57 | 79.7 | 89.2 | 78.5 | |
| CoTBackbone=LLaVA-1.5-13B, Training Strategy=CoT2025.12 | 84.1 | 62.4 | 68.9 | 77.8 | 73.3 | |
| pretrainedBackbone=Qwen2.5-VL-7B, Training Strategy=pretrained2025.12 | 82.1 | 58.6 | 69.2 | 85.6 | 73.9 | |
| pretrainedBackbone=LLaVA-1.5-7B, Training Strategy=pretrained2025.12 | 81.5 | 52.3 | 63.2 | 51.9 | 62.2 | |
| pretrainedBackbone=LLaVA-1.5-13B, Training Strategy=pretrained2025.12 | 60 | 49.9 | 40 | 44.1 | 48.5 | |
| RIO-RTBackbone=LLaVA-1.5-13B, Training Strategy=RIO-RT2025.12 | 57.4 | 54.9 | 48.2 | 89.6 | 62.5 | |
| IT-RTBackbone=LLaVA-1.5-7B, Training Strategy=IT-RT2025.12 | 54.9 | 40.6 | 63.9 | 12.9 | 43.1 | |
| pretrainedBackbone=Llama-3.2-11B-Vision, Training Strategy=pretrained2025.12 | 53.8 | 45.9 | 64.2 | 69.9 | 58.5 | |
| CoTBackbone=LLaVA-1.5-7B, Training Strategy=CoT2025.12 | 48.7 | 43.8 | 62.6 | 78.2 | 58.3 | |
| IT-RTBackbone=LLaVA-1.5-13B, Training Strategy=IT-RT2025.12 | 41 | 64.2 | 41.6 | 97 | 61 |