# Chart Understanding and Reasoning on CharXiv Source: [Chart Understanding and Reasoning on CharXiv]() Collection index: [llms.txt]() Data updated: 2026-06-02T06:33:32.635Z Evaluation of model performance on the CharXiv dataset for Chart Understanding and Reasoning tasks, measured using the reported Score metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Chart Understanding and Reasoning | | Dataset (including any stated split) | CharXiv | | Metric | Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 37 of 37 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | Score | Result date | Report paper | | --- | --- | --- | --- | | Gemini 2.5 Pro | 67.8 | Not recorded | 691171652828b7cc6937d492 | | Claude-3.7-Sonnet | 64.2 | Not recorded | 69520ae353fdd5c36173d472 | | Vero Mi-7B | 63.5 | Not recorded | 69d48fe77d0172140ec74799 | | MiMo-VL-7B-SFT | 62.6 | Not recorded | 6a1e63dbdbd7150506f854b9 | | MiMoVL 7B-RL | 60.9 | Not recorded | 69d48fe77d0172140ec74799 | | Vero Q3T-8B | 59.9 | Not recorded | 69d48fe77d0172140ec74799 | | MiMo-VL-7B-SFT | 58.7 | Not recorded | 6a1e63dbdbd7150506f854b9 | | Vero Q3I-8B | 53.7 | Not recorded | 69d48fe77d0172140ec74799 | | Q3VL 8B-Thk | 53 | Not recorded | 69d48fe77d0172140ec74799 | | GPT-5 Nano | 51.2 | Not recorded | 69d48fe77d0172140ec74799 | | BiPS-General-7B | 50.6 | Not recorded | 69520ae353fdd5c36173d472 | | BiPS-Chart-7B | 49.4 | Not recorded | 69520ae353fdd5c36173d472 | | DeepEyesV2 | 48.9 | Not recorded | 691171652828b7cc6937d492 | | Qwen2.5-VL | 48 | Not recorded | 691171652828b7cc6937d492 | | GPT-4o | 47.5 | Not recorded | 691171652828b7cc6937d492 | | GPT-4o | 47.1 | Not recorded | 69520ae353fdd5c36173d472 | | Vero Q25-7B | 47.1 | Not recorded | 69d48fe77d0172140ec74799 | | Q3VL 8B-Ins | 46.4 | Not recorded | 69d48fe77d0172140ec74799 | | Chart-R1-7B | 46.2 | Not recorded | 69520ae353fdd5c36173d472 | | GRPO | 45.4 | Not recorded | 69520ae353fdd5c36173d472 | | DeepEyes-7B | 42.9 | Not recorded | 69520ae353fdd5c36173d472 | | Qwen2.5-VL-7B | 42.5 | Not recorded | 69520ae353fdd5c36173d472 | | Vision-R1-7B | 42.5 | Not recorded | 69520ae353fdd5c36173d472 | | Q25VL 7B-Ins | 42.5 | Not recorded | 69d48fe77d0172140ec74799 | | Qwen3-VL-4B-Instruct | 42.4 | Not recorded | 6a1e63dbdbd7150506f854b9 | | BigCharts-R1-7B | 41.3 | Not recorded | 69520ae353fdd5c36173d472 | | Qwen2.5-VL | 40.2 | Not recorded | 691171652828b7cc6937d492 | | Qwen3-VL-4B-Instruct | 39.8 | Not recorded | 6a1e63dbdbd7150506f854b9 | | Qwen2.5-VL-7B-Instruct | 38.9 | Not recorded | 6a1e63dbdbd7150506f854b9 | | InternVL3-8B | 37.6 | Not recorded | 69520ae353fdd5c36173d472 | | InternVL3 | 37.6 | Not recorded | 691171652828b7cc6937d492 | | Qwen2.5-VL-7B-Instruct | 37.4 | Not recorded | 6a1e63dbdbd7150506f854b9 | | Mo2-O 7B | 35.6 | Not recorded | 69d48fe77d0172140ec74799 | | R1-OneVision-7B | 33.8 | Not recorded | 69520ae353fdd5c36173d472 | | ChartLlama-13B | 14.2 | Not recorded | 69520ae353fdd5c36173d472 | | ChartGemma-3B | 12.5 | Not recorded | 69520ae353fdd5c36173d472 | | TinyChart-3B | 8.3 | Not recorded | 69520ae353fdd5c36173d472 | ### Gemini 2.5 Pro Score: 67.8. Result ID: 69b2780d6dbb7075b95d014b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Tool":"Code","Param Size":"-"}. This record does not assert independent reproduction. ### Claude-3.7-Sonnet Score: 64.2. Result ID: 699808965698665ec462208f. [Reporting paper]() Recorded evaluation configuration: {"Data":"-"}. This record does not assert independent reproduction. ### Vero Mi-7B Score: 63.5. Result ID: 69d4e5c647f34ce924bd96a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"MiMoVL","Initial Model":"7B-SFT"}. This record does not assert independent reproduction. ### MiMo-VL-7B-SFT Score: 62.6. Result ID: 6a1e7937fbc373592c17e7f0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Run":"+TRON"}. This record does not assert independent reproduction. ### MiMoVL 7B-RL Score: 60.9. Result ID: 69d4e5c647f34ce924bd96ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"MiMoVL","Initial Model":"7B-SFT"}. This record does not assert independent reproduction. ### Vero Q3T-8B Score: 59.9. Result ID: 69d4e5c647f34ce924bd96a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"Qwen3VL","Initial Model":"8B Think"}. This record does not assert independent reproduction. ### MiMo-VL-7B-SFT Score: 58.7. Result ID: 6a1e7937fbc373592c17e7ef. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Run":"Base"}. This record does not assert independent reproduction. ### Vero Q3I-8B Score: 53.7. Result ID: 69d4e5c647f34ce924bd96a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"Qwen3VL","Initial Model":"8B Inst"}. This record does not assert independent reproduction. ### Q3VL 8B-Thk Score: 53. Result ID: 69d4e5c647f34ce924bd96a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"N/A","Initial Model":"N/A"}. This record does not assert independent reproduction. ### GPT-5 Nano Score: 51.2. Result ID: 69d4e5c647f34ce924bd96ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"N/A","Initial Model":"N/A"}. This record does not assert independent reproduction. ### BiPS-General-7B Score: 50.6. Result ID: 699808965698665ec462209c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Data":"13K+39K"}. This record does not assert independent reproduction. ### BiPS-Chart-7B Score: 49.4. Result ID: 699808965698665ec462209b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Data":"13K"}. This record does not assert independent reproduction. ### DeepEyesV2 Score: 48.9. Result ID: 69b2780d6dbb7075b95d014c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Tool":"General","Param Size":"7B"}. This record does not assert independent reproduction. ### Qwen2.5-VL Score: 48. Result ID: 69b2780d6dbb7075b95d0148. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Tool":"✗","Param Size":"32B"}. This record does not assert independent reproduction. ### GPT-4o Score: 47.5. Result ID: 69b2780d6dbb7075b95d014a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Tool":"Code","Param Size":"-"}. This record does not assert independent reproduction. ### GPT-4o Score: 47.1. Result ID: 699808965698665ec462208e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"-"}. This record does not assert independent reproduction. ### Vero Q25-7B Score: 47.1. Result ID: 69d4e5c647f34ce924bd96a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"Qwen25VL","Initial Model":"7B Inst"}. This record does not assert independent reproduction. ### Q3VL 8B-Ins Score: 46.4. Result ID: 69d4e5c647f34ce924bd96a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"N/A","Initial Model":"N/A"}. This record does not assert independent reproduction. ### Chart-R1-7B Score: 46.2. Result ID: 699808965698665ec4622099. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"258K"}. This record does not assert independent reproduction. ### GRPO Score: 45.4. Result ID: 699808965698665ec462209a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Data":"13K + 39K"}. This record does not assert independent reproduction. ### DeepEyes-7B Score: 42.9. Result ID: 699808965698665ec4622097. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"14K + 33K"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B Score: 42.5. Result ID: 699808965698665ec4622090. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"-"}. This record does not assert independent reproduction. ### Vision-R1-7B Score: 42.5. Result ID: 699808965698665ec4622096. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"73K + 137K"}. This record does not assert independent reproduction. ### Q25VL 7B-Ins Score: 42.5. Result ID: 69d4e5c647f34ce924bd96aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"N/A","Initial Model":"N/A"}. This record does not assert independent reproduction. ### Qwen3-VL-4B-Instruct Score: 42.4. Result ID: 6a1e7937fbc373592c17e7ec. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Run":"+TRON"}. This record does not assert independent reproduction. ### BigCharts-R1-7B Score: 41.3. Result ID: 699808965698665ec4622098. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"1.7M"}. This record does not assert independent reproduction. ### Qwen2.5-VL Score: 40.2. Result ID: 69b2780d6dbb7075b95d0147. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Tool":"✗","Param Size":"7B"}. This record does not assert independent reproduction. ### Qwen3-VL-4B-Instruct Score: 39.8. Result ID: 6a1e7937fbc373592c17e7eb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Run":"Base"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct Score: 38.9. Result ID: 6a1e7937fbc373592c17e7ee. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Run":"+TRON"}. This record does not assert independent reproduction. ### InternVL3-8B Score: 37.6. Result ID: 699808965698665ec4622091. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"-"}. This record does not assert independent reproduction. ### InternVL3 Score: 37.6. Result ID: 69b2780d6dbb7075b95d0149. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Tool":"✗","Param Size":"8B"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct Score: 37.4. Result ID: 6a1e7937fbc373592c17e7ed. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Run":"Base"}. This record does not assert independent reproduction. ### Mo2-O 7B Score: 35.6. Result ID: 69d4e5c647f34ce924bd96ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL":"SFT Only","Initial Model":"N/A"}. This record does not assert independent reproduction. ### R1-OneVision-7B Score: 33.8. Result ID: 699808965698665ec4622095. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"67K + 98K"}. This record does not assert independent reproduction. ### ChartLlama-13B Score: 14.2. Result ID: 699808965698665ec4622092. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"160K"}. This record does not assert independent reproduction. ### ChartGemma-3B Score: 12.5. Result ID: 699808965698665ec4622093. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"163K"}. This record does not assert independent reproduction. ### TinyChart-3B Score: 8.3. Result ID: 699808965698665ec4622094. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Data":"1.3M"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.