Chart Question Answering on ChartQA (OA, RA, VA, RVC)
94.7Overall Accuracy (OA)Gemini 2.5 Pro
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemini 2.5 ProModel Group=Proprietary models2026.05 | 94.7 | 92.7 | 89.7 | -5.2 | |
| GPT-5.4 miniModel Group=Proprietary models2026.05 | 94 | 94 | 91.7 | -2.5 | |
| Gemini 2.5 FlashModel Group=Proprietary models2026.05 | 93.3 | 93.3 | 89.1 | -4.5 | |
| Claude Sonnet 4.6Model Group=Proprietary models2026.05 | 92.7 | 93.3 | 90.5 | -2.3 | |
| GPT-4.1Model Group=Proprietary models2026.05 | 92.7 | 90 | 86.9 | -6.3 | |
| GPT-5.4Model Group=Proprietary models2026.05 | 92.7 | 94.7 | 91.6 | -1.2 | |
| Claude Haiku 4.5Model Group=Proprietary models2026.05 | 90.7 | 93.3 | 87.8 | -3.2 | |
| Qwen3-VL-8B-InstructModel Group=Open-source models2026.05 | 90.7 | 90.7 | 88.2 | -2.7 | |
| GPT-4.1 miniModel Group=Proprietary models2026.05 | 88 | 90 | 86.1 | -2.1 | |
| Qwen2.5-VL-7BToken Retention Rate=100%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 86.2 | — | — | — | |
| GPT-4oModel Group=Proprietary models2026.05 | 86 | 88.7 | 84.1 | -2.2 | |
| Gemma 4 E4B ITModel Group=Open-source models2026.05 | 86 | 85.3 | 83.9 | -2.5 | |
| InternVL3-8BModel Group=Open-source models2026.05 | 84.7 | 85.3 | 81.6 | -3.6 | |
| Qwen2.5-VL-7B-InstructModel Group=Open-source models2026.05 | 84 | 88 | 79 | -6 | |
| FastVToken Retention Rate=50%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 83.3 | — | — | — | |
| Pixtral-12BModel Group=Open-source models2026.05 | 82.7 | 78.7 | 76.7 | -7.3 | |
| ERAToken Retention Rate=50%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 82.7 | — | — | — | |
| VisionZipToken Retention Rate=50%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 78.1 | — | — | — | |
| LLaVA-OneVision-Qwen2-7BModel Group=Open-source models2026.05 | 70.7 | 68 | 61.2 | -13.4 | |
| ERAToken Retention Rate=20%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 70.2 | — | — | — | |
| CDPrunerToken Retention Rate=50%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 69.5 | — | — | — | |
| FastVToken Retention Rate=20%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 57.8 | — | — | — | |
| VisionZipToken Retention Rate=20%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 55 | — | — | — | |
| ERAToken Retention Rate=10%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 54.5 | — | — | — | |
| CDPrunerToken Retention Rate=20%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 52 | — | — | — | |
| CDPrunerToken Retention Rate=10%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 42 | — | — | — | |
| VisionZipToken Retention Rate=10%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 38.3 | — | — | — | |
| FastVToken Retention Rate=10%, Backbone=Qwen2.5-VL-7B, Resolution Strategy=Dynamic Resolution2026.06 | 34.6 | — | — | — |