Visual Question Answering on PlotQA
96.4Accuracy (v1)VL-T5
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| VL-T5Gold Table Access=yes2022.12 | 96.4 | — | 84.7 | 90.6 | |
| T5Gold Table Access=yes2022.12 | 93.2 | — | 85.6 | 89.4 | |
| MATCHAGold Table Access=no2022.12 | 92.3 | — | 90.7 | 91.5 | |
| MATCHASupervision=Fully-supervised2022.12 | 92.3 | — | 90.7 | 91.5 | |
| VisionTaPasGold Table Access=yes2022.12 | 80.2 | — | 58.3 | 69.3 | |
| CRCTGold Table Access=no2022.12 | 76.9 | — | 34.4 | 55.7 | |
| CRCTSupervision=Fully-supervised2022.12 | 76.9 | — | 34.4 | 55.7 | |
| VL-T5-OCRGold Table Access=no2022.12 | 75.9 | — | 56 | 66 | |
| VL-T5-OCRSupervision=Fully-supervised2022.12 | 75.9 | — | 56 | 66 | |
| Pix2StructGold Table Access=no2022.12 | 73.2 | — | 71.9 | 72.5 | |
| Pix2StructSupervision=Fully-supervised2022.12 | 73.2 | — | 71.9 | 72.5 | |
| T5-OCRGold Table Access=no2022.12 | 72.6 | — | 56.2 | 64.4 | |
| T5-OCRSupervision=Fully-supervised2022.12 | 72.6 | — | 56.2 | 64.4 | |
| VisionTaPas-OCRGold Table Access=no2022.12 | 65.3 | — | 42.5 | 53.9 | |
| VisionTapas-OCRSupervision=Fully-supervised2022.12 | 65.3 | — | 42.5 | 53.9 | |
| PaLI-17BGold Table Access=no, Input Resolution=5882022.12 | 64.5 | — | 15.2 | 39.8 | |
| PaLI-17BSupervision=Fully-supervised, Resolution=5882022.12 | 64.5 | — | 15.2 | 39.8 | |
| DEPLOT+FlanPaLM+Codex POT SCSupervision=One-shot, Strategy=Program-of-thought prompting + Self-consistency2022.12 | 62.2 | — | 71 | 66.6 | |
| DEPLOT+Codex POT SCSupervision=One-shot, Strategy=Program-of-thought prompting + Self-consistency2022.12 | 58.8 | — | 69.8 | 64.3 | |
| DEPLOT+FlanPaLM SCSupervision=One-shot, Strategy=Self-consistency2022.12 | 57.8 | — | 50.1 | 53.9 | |
| PaLI-17BGold Table Access=no, Input Resolution=2242022.12 | 56.9 | — | 13.1 | 35 | |
| PaLI-17BSupervision=Fully-supervised, Resolution=2242022.12 | 56.9 | — | 13.1 | 35 | |
| DEPLOT+FlanPaLM COTSupervision=One-shot, Strategy=Chain-of-thought prompting2022.12 | 51.3 | — | 44.9 | 48.1 | |
| DEPLOT+GPT3 SCSupervision=One-shot, Strategy=Self-consistency2022.12 | 35 | — | 51.6 | 43.3 | |
| DEPLOT+GPT3 COTSupervision=One-shot, Strategy=Chain-of-thought prompting2022.12 | 31.9 | — | 51.3 | 41.6 | |
| BAN2019.09 | — | 0.01 | — | — | |
| ChartSketcher-2BModel Scale=2B2026.01 | — | 41.12 | — | — | |
| IMG2019.09 | — | 4.84 | — | — | |
| LoRRA2019.09 | — | 0.02 | — | — | |
| Our Model2019.09 | — | 22.52 | — | — | |
| QUES2019.09 | — | 5.35 | — | — | |
| Qwen2.5VL-3BModel Scale=3B2026.01 | — | 42.88 | — | — | |
| Qwen2.5VL-7BModel Scale=7B2026.01 | — | 63.44 | — | — | |
| SAN2019.09 | — | 7.76 | — | — | |
| SketchVL-3BModel Scale=3B, Training Strategy=FinePO (Full Model)2026.01 | — | 48.32 | — | — | |
| SketchVL-3BModel Scale=3B, Training Strategy=Cold Start only2026.01 | — | 35.44 | — | — | |
| SketchVL-3BModel Scale=3B, Training Strategy=Naive GRPO2026.01 | — | 44.72 | — | — | |
| SketchVL-3BModel Scale=3B, Reward Mechanism=Random scores2026.01 | — | 46.4 | — | — | |
| SketchVL-3BModel Scale=3B, KL Action Regularization=Disabled2026.01 | — | 48.16 | — | — | |
| SketchVL-3BModel Scale=3B, Reasoning Method=Zero GRPO (No Sketch)2026.01 | — | 31.12 | — | — | |
| SketchVL-3BModel Scale=3B, Training Strategy=SFT (No RL)2026.01 | — | 27.44 | — | — | |
| SketchVL-7BModel Scale=7B, Training Strategy=FinePO2026.01 | — | 55.84 | — | — | |
| VLM-R12026.01 | — | 54.4 | — | — |