# Document Understanding on ChartXiv-DQ (Accuracy)

Source: [Document Understanding on ChartXiv-DQ (Accuracy)](<https://www.sota2.com/research/sota/document-understanding-on-chartxiv-dq-accuracy>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:31:29.368Z

Evaluation of model performance on the ChartXiv-DQ dataset for document understanding tasks, measured using Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Document Understanding |
| Dataset (including any stated split) | ChartXiv-DQ |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/document-understanding-on-chartxiv-dq-accuracy.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| Gemini 3-Pro | 95.95 | Not recorded | 69842436a8b01e76cc8ee882 |
| Gemini 2.5-Pro | 93.38 | Not recorded | 69842436a8b01e76cc8ee882 |
| Qwen3-VL | 92.92 | Not recorded | 69842436a8b01e76cc8ee882 |
| GPT-5 | 91.17 | Not recorded | 69842436a8b01e76cc8ee882 |
| MMFineReason-8B | 90.8 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| Qwen3-VL 32B | 90.2 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| Gemini-2.5 Flash | 90.1 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| GPT5 mini | 89.4 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| ERNIE 5.0 | 89.05 | Not recorded | 69842436a8b01e76cc8ee882 |
| MMFineReason-4B | 87.7 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| Qwen3-VL 30B-A3B | 86.9 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| Qwen3-VL 8B | 85.9 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| MMR1 8B | 81.5 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| HoneyBee 8B | 75.8 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| MMFineReason-2B | 74.3 | Not recorded | 697c3b66a8b01e76cc8da6e1 |
| OMR 7B | 73.5 | Not recorded | 697c3b66a8b01e76cc8da6e1 |

### Gemini 3-Pro

Accuracy: 95.95. Result ID: 69964b4b5698665ec48f966c.

[Reporting paper](<https://www.sota2.com/research/paper/69842436a8b01e76cc8ee882.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Gemini 2.5-Pro

Accuracy: 93.38. Result ID: 69964b4b5698665ec48f966a.

[Reporting paper](<https://www.sota2.com/research/paper/69842436a8b01e76cc8ee882.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Qwen3-VL

Accuracy: 92.92. Result ID: 69964b4b5698665ec48f9669.

[Reporting paper](<https://www.sota2.com/research/paper/69842436a8b01e76cc8ee882.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6927db54d6b0dd3342930f66.md>)

Recorded evaluation configuration: {"mode":"Thinking"}. This record does not assert independent reproduction.

### GPT-5

Accuracy: 91.17. Result ID: 69964b4b5698665ec48f966b.

[Reporting paper](<https://www.sota2.com/research/paper/69842436a8b01e76cc8ee882.md>)

Recorded evaluation configuration: {"tier":"High"}. This record does not assert independent reproduction.

### MMFineReason-8B

Accuracy: 90.8. Result ID: 6997797c5698665ec4cdd0d9.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

Recorded evaluation configuration: {"Model Source Category":"Ours","Model Parameters":"8B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### Qwen3-VL 32B

Accuracy: 90.2. Result ID: 6997797c5698665ec4cdd0d3.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6927db54d6b0dd3342930f66.md>)

Recorded evaluation configuration: {"Model Source Category":"Open-weight VLMs","Model Parameters":"32B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### Gemini-2.5 Flash

Accuracy: 90.1. Result ID: 6997797c5698665ec4cdd0cf.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c311e209131cea8cef5.md>)

Recorded evaluation configuration: {"Model Source Category":"Closed-source VLMs","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### GPT5 mini

Accuracy: 89.4. Result ID: 6997797c5698665ec4cdd0d0.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

Recorded evaluation configuration: {"Model Source Category":"Closed-source VLMs","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### ERNIE 5.0

Accuracy: 89.05. Result ID: 69964b4b5698665ec48f966d.

[Reporting paper](<https://www.sota2.com/research/paper/69842436a8b01e76cc8ee882.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69842436a8b01e76cc8ee882.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### MMFineReason-4B

Accuracy: 87.7. Result ID: 6997797c5698665ec4cdd0d8.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

Recorded evaluation configuration: {"Model Source Category":"Ours","Model Parameters":"4B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### Qwen3-VL 30B-A3B

Accuracy: 86.9. Result ID: 6997797c5698665ec4cdd0d2.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6927db54d6b0dd3342930f66.md>)

Recorded evaluation configuration: {"Model Source Category":"Open-weight VLMs","Model Parameters":"30B-A3B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### Qwen3-VL 8B

Accuracy: 85.9. Result ID: 6997797c5698665ec4cdd0d1.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6927db54d6b0dd3342930f66.md>)

Recorded evaluation configuration: {"Model Source Category":"Open-weight VLMs","Model Parameters":"8B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### MMR1 8B

Accuracy: 81.5. Result ID: 6997797c5698665ec4cdd0d4.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b8f1e209131cea8920f.md>)

Recorded evaluation configuration: {"Model Source Category":"Open-source VLMs","Model Parameters":"8B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### HoneyBee 8B

Accuracy: 75.8. Result ID: 6997797c5698665ec4cdd0d5.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c1a1e209131cea8c598.md>)

[Implementation](<https://github.com/facebookresearch/HoneyBee_VLM>)

Recorded evaluation configuration: {"Model Source Category":"Open-source VLMs","Model Parameters":"8B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### MMFineReason-2B

Accuracy: 74.3. Result ID: 6997797c5698665ec4cdd0d7.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

Recorded evaluation configuration: {"Model Source Category":"Ours","Model Parameters":"2B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

### OMR 7B

Accuracy: 73.5. Result ID: 6997797c5698665ec4cdd0d6.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b66a8b01e76cc8da6e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/691ff1f4d6b0dd334292e589.md>)

[Implementation](<https://github.com/EvolvingLMMs-Lab/OpenMMReasoner>)

Recorded evaluation configuration: {"Model Source Category":"Open-source VLMs","Model Parameters":"7B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
