# Chart Question Answering on MC v1 (test) Source: [Chart Question Answering on MC v1 (test)]() Collection index: [llms.txt]() Data updated: 2026-03-31T12:30:33.185Z Evaluation of model performance on the test split of the MC v1 dataset for Chart Question Answering capabilities, measured using Accuracy (Acc), Word Match (WM), and Word Omission (WO) metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Chart Question Answering | | Dataset (including any stated split) | MC v1 (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 11 of 11 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Word Match](): Higher is better; unit: PERCENT. - [Word Omission](): Lower is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | ChartCynics | 74.43 | Not recorded | 69cb55840636e5bf4afca459 | | ChartCynics | 71.8 | Not recorded | 69cb55840636e5bf4afca459 | | ChartCynics | 71.67 | Not recorded | 69cb55840636e5bf4afca459 | | Gemini-3.1-Pro | 70.49 | Not recorded | 69cb55840636e5bf4afca459 | | Gemini-2.5-Flash | 67.21 | Not recorded | 69cb55840636e5bf4afca459 | | ChartCynics | 60.66 | Not recorded | 69cb55840636e5bf4afca459 | | GPT-o4-mini | 55.08 | Not recorded | 69cb55840636e5bf4afca459 | | Table-based QA | 49.18 | Not recorded | 69cb55840636e5bf4afca459 | | Qwen3-VL-8B | 45.57 | Not recorded | 69cb55840636e5bf4afca459 | | Redrawn | 44.26 | Not recorded | 69cb55840636e5bf4afca459 | | ChartMoE | 33.11 | Not recorded | 69cb55840636e5bf4afca459 | ### ChartCynics Accuracy: 74.43. Result ID: 69cbbe6747f34ce924b55e3b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B","Configuration":"SFT + GRPO (Full)"}. This record does not assert independent reproduction. ### ChartCynics Accuracy: 71.8. Result ID: 69cbbe6747f34ce924b55e3d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-o4-mini","Configuration":"Train-free"}. This record does not assert independent reproduction. ### ChartCynics Accuracy: 71.67. Result ID: 69cbbe6747f34ce924b55e3f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Gemini-2.5-Flash","Configuration":"Train-free"}. This record does not assert independent reproduction. ### Gemini-3.1-Pro Accuracy: 70.49. Result ID: 69cbbe6747f34ce924b55e36. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Standard Multimodal Baseline"}. This record does not assert independent reproduction. ### Gemini-2.5-Flash Accuracy: 67.21. Result ID: 69cbbe6747f34ce924b55e3e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Configuration":"Base Model"}. This record does not assert independent reproduction. ### ChartCynics Accuracy: 60.66. Result ID: 69cbbe6747f34ce924b55e3a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B","Configuration":"Train-free"}. This record does not assert independent reproduction. ### GPT-o4-mini Accuracy: 55.08. Result ID: 69cbbe6747f34ce924b55e3c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Configuration":"Base Model"}. This record does not assert independent reproduction. ### Table-based QA Accuracy: 49.18. Result ID: 69cbbe6747f34ce924b55e37. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B","category":"Structural Mitigation Heuristic"}. This record does not assert independent reproduction. ### Qwen3-VL-8B Accuracy: 45.57. Result ID: 69cbbe6747f34ce924b55e39. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Configuration":"Base Model"}. This record does not assert independent reproduction. ### Redrawn Accuracy: 44.26. Result ID: 69cbbe6747f34ce924b55e38. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B","category":"Structural Mitigation Heuristic"}. This record does not assert independent reproduction. ### ChartMoE Accuracy: 33.11. Result ID: 69cbbe6747f34ce924b55e35. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Standard Multimodal Baseline"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.