Chart Question Answering on ChartQAPro 67% held-out (test)
46.88Factoid AccuracyCOGS
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| COGSBase Model=Qwen2.5-VL-7B, Data Synthesis Approach=COGS2025.10 | 46.88 | 65.73 | 51.16 | 61.85 | 58.25 | 52.02 | |
| In-Context Q ExampleBase Model=Qwen2.5-VL-7B, Data Synthesis Approach=In-Context Q Example2025.10 | 46.33 | 62.94 | 46.91 | 61.11 | 61.72 | 50.58 | |
| GPT-5-nanoModel Category=Proprietary Models2025.10 | 45.95 | 63.64 | 49.4 | 63.58 | 49.82 | 50.74 | |
| GPT-4o-miniModel Category=Proprietary Models2025.10 | 43.63 | 66.43 | 45.48 | 59.88 | 45.2 | 48.32 | |
| Claude Haiku 3.5Model Category=Proprietary Models2025.10 | 43.44 | 65.03 | 39.84 | 61.79 | 38.77 | 46.74 | |
| Self-ConsistencyBase Model=Qwen2.5-VL-7B, Prompting Strategy=Self-Consistency2025.10 | 43.44 | 61.54 | 44 | 59.82 | 41.76 | 47.22 | |
| InternVL3.5-GPT-OSSModel Category=Opensource Models (7B+)2025.10 | 43.02 | 58.74 | 42.86 | 58.02 | 54.48 | 46.86 | |
| Chart-R1Base Model=Qwen2.5-VL-7B, Data Synthesis Approach=Chart-R12025.10 | 42.17 | 46.85 | 50.53 | 61.11 | 55.55 | 47.32 | |
| Decompositional CoTBase Model=Qwen2.5-VL-7B, Prompting Strategy=Decompositional CoT2025.10 | 42.08 | 65.03 | 42.57 | 56.53 | 45.55 | 46.36 | |
| Qwen2.5-VL-7B (base)Model Category=Opensource Models (7B+)2025.10 | 42.07 | 62.59 | 44.88 | 60.78 | 50.72 | 47.36 | |
| Gemini 2.5 Flash-LiteModel Category=Proprietary Models2025.10 | 40.42 | 19.96 | 48.77 | 37.43 | 16.66 | 38.72 | |
| Tree of ThoughtsBase Model=Qwen2.5-VL-7B, Prompting Strategy=Tree of Thoughts2025.10 | 40.01 | 57.94 | 41.55 | 54.13 | 53.35 | 44.44 | |
| ChartQA-TrainBase Model=Qwen2.5-VL-7B, Data Synthesis Approach=ChartQA-Train2025.10 | 38.77 | 60.14 | 49.72 | 61.11 | 53.12 | 46.64 | |
| Phi-4-14BModel Category=Opensource Models (7B+)2025.10 | 23.18 | 34.27 | 40.93 | 46.91 | 36.31 | 31.61 | |
| ChartMoEModel Category=Chart Specialist Models2025.10 | 19.03 | 35.66 | 32.97 | 45.68 | 27.08 | 27.28 | |
| ChartLLaMAModel Category=Chart Specialist Models2025.10 | 8.11 | 23.08 | 18.37 | 45.06 | 29.55 | 17.19 |