# Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Framework

Source: [Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Framework](<https://www.sota2.com/research/paper/6a434da0ba73165dbb743d8c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-01T05:00:12.177Z

Authors: Yuchen He, Peizhi Ying, Liqi Cheng, Kuilin Peng, Yuan Tian, Dazhen Deng, Yingcai Wu

Published: 2026-06-29T05:40:35.000Z

## Abstract

Chart data extraction, which reverse-engineers data tables from chart images, is essential for reproducibility, analysis, retrieval, and redesign. Existing interactive tools are reliable but tedious, and mixed-initiative systems, while more efficient, lack generalizability. Recent multimodal large language models (MLLMs) offer a unified interface for chart interpretation, yet their ability to extract accurate data tables, especially without visible labels, remains unclear. We build a benchmark featuring diverse real-world charts without data labels to evaluate this capability. Results show that, while current MLLMs reliably reconstruct table structures, they struggle with precise value recovery. To address this, we revisit chart data extraction from a human-centered perspective and argue that extraction should follow a progressive learning process similar to how people read charts. Our training framework substantially improves numerical accuracy, achieving state-of-the-art performance with a 7B-parameter model. A user study further shows that our model effectively supports mixed-initiative workflows for reliable chart data extraction.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Chart Data Extraction Benchmark](<https://www.sota2.com/research/sota/chart-data-extraction-benchmark.md>): Chart Data Extraction; Chart Data Extraction Benchmark; RNSS: 75.92.

- [Chart Data Extraction on ExChart-Bench 1.0 (test)](<https://www.sota2.com/research/sota/chart-data-extraction-on-exchart-bench-1-0-test.md>): Chart Data Extraction; ExChart-Bench 1.0 (test); Format Success Rate (All): 99.11.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.29808>)

[PDF](<https://arxiv.org/pdf/2606.29808.pdf>)
