Chart Understanding on FinChart-Bench
97.86TF Scoreo3
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| o3Category=Closed Source2026.06 | 97.86 | 92.17 | 60.79 | 83.89 | |
| o4 miniCategory=Closed Source2026.06 | 97.61 | 91.96 | 60.18 | 83.53 | |
| Gemini 2.5 ProCategory=Closed Source2026.06 | 97.27 | 89.7 | 63.46 | 83.73 | |
| MOCA-AGENTBackbone=Qwen3.6-27B, Input=VLM transcription2026.06 | 97.27 | 84.85 | 74.65 | 85.59 | |
| Claude Sonnet 4Category=Closed Source2026.06 | 96.94 | 91.66 | 63.59 | 84.32 | |
| GPT 4.1Category=Closed Source2026.06 | 96.18 | 83.19 | 62.54 | 80.88 | |
| Qwen2.5 VLParameters=7B, Category=Open Source2026.06 | 95.09 | 82.81 | 37.29 | 72.16 | |
| GPT 4oCategory=Closed Source2026.06 | 93.92 | 77.57 | 57.59 | 76.62 | |
| Mistral 3.1Parameters=24B, Category=Open Source2026.06 | 92.95 | 84.17 | 44.9 | 74.37 | |
| Qwen2 VLParameters=7B, Category=Open Source2026.06 | 91.86 | 19.4 | 24.77 | 45.76 | |
| GPT 4.1 miniCategory=Closed Source2026.06 | 91.61 | 86.77 | 60.3 | 79.8 | |
| LLaMa 4Parameters=17B, Category=Open Source2026.06 | 89.56 | 59.7 | 59.78 | 69.89 | |
| Gemma 3Parameters=12B, Category=Open Source2026.06 | 89.14 | 70.94 | 27.22 | 62.89 | |
| Sa2VAParameters=8B, Category=Open Source2026.06 | 86.62 | 75.79 | 19.43 | 61.12 | |
| CosmosParameters=7B, Category=Open Source2026.06 | 70.93 | 43.49 | 20.18 | 45.22 | |
| LLaMa 3.2Parameters=11B, Category=Open Source2026.06 | 63.46 | 64 | 5.12 | 44.65 | |
| LLaVa v1.6Parameters=7B, Category=Open Source2026.06 | 63.3 | 36.94 | 1.84 | 34.47 | |
| ChartInstruct-T5Parameters=3B, Category=Chart Fine-tuned2026.06 | 62.12 | 2.3 | 1.71 | 22.43 | |
| DeepSeek VL2Parameters=2.8B, Category=Open Source2026.06 | 57.97 | 6.77 | 4.73 | 23.5 | |
| nanoVLMParameters=222M, Category=Open Source2026.06 | 53.9 | 2.43 | 0.22 | 19.19 | |
| MedGemmaParameters=4B, Category=Open Source2026.06 | 53.27 | 46.3 | 3.72 | 34.81 | |
| ChartInstruct-llamaParameters=7B, Category=Chart Fine-tuned2026.06 | 51.09 | 0.17 | 1.97 | 18.05 | |
| UniChartParameters=201M, Category=Chart Fine-tuned2026.06 | 51.05 | 0 | 0.57 | 17.52 | |
| Blip 2Parameters=2.7B, Category=Open Source2026.06 | 49.79 | 0.77 | 0 | 17.17 | |
| MatchaParameters=282M, Category=Chart Fine-tuned2026.06 | 49.54 | 0 | 0.7 | 17.05 | |
| ChartGemmaParameters=3B, Category=Chart Fine-tuned2026.06 | 48.7 | 0.29 | 0.7 | 16.87 |