Chart Question Answering on ChartQA (val)
77.8Relaxed Acc (avg.)GT+GPT-4
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GT+GPT-4Schema=STR, Perception=Ground Truth2023.09 | 77.8 | 89.7 | 65.9 | — | |
| Qwen2VL-7B-InstructLatency=0.35692025.12 | 77.38 | — | — | 80.8 | |
| Qwen2VL-7B-Instruct + LUVCLatency=0.2335, Speedup=1.53x2025.12 | 76.52 | — | — | 80.08 | |
| GT+GPT-4Schema=LCT, Perception=Ground Truth2023.09 | 76 | 87.9 | 64.1 | — | |
| StructChart+GPT-4Train Data=ChartQA+SimChart9K, Schema=STR2023.09 | 75.5 | 85.6 | 64.5 | — | |
| Qwen2VL-7B-Instruct + PACTLatency=0.2179, Speedup=1.64x2025.12 | 74 | — | — | 76 | |
| StructChart+GPT-4Train Data=ChartQA+SimChart9K, Schema=LCT2023.09 | 73.6 | 84.1 | 63.1 | — | |
| Qwen2VL-7B-Instruct + FastVLatency=0.3594, Speedup=0.99x2025.12 | 73.2 | — | — | 75.32 | |
| StructChart+Llama3.1-70bTrain Data=ChartQA+SimChart9K, Schema=STR2023.09 | 72.8 | 84.9 | 60.6 | — | |
| StructChart+Llama3.1-70bTrain Data=ChartQA+SimChart9K, Schema=LCT2023.09 | 71.1 | 83.2 | 58.9 | — | |
| GPT-4VTrain Data=C.D., Schema=Baseline2023.09 | 70.3 | 76.1 | 64.5 | — | |
| ChartLlamaTrain Data=C.D., Schema=Baseline2023.09 | 69.7 | 90.4 | 49 | — | |
| MatChaTrain Data=ChartQA/PlotQA/C.D., Schema=Baseline2023.09 | 64.2 | 90.2 | 38.2 | — | |
| StructChart+Llama3.1-8bTrain Data=ChartQA+SimChart9K, Schema=STR2023.09 | 64 | 72.7 | 55.3 | — | |
| StructChart+Vicuna-13bTrain Data=ChartQA+SimChart9K, Schema=STR2023.09 | 57.1 | 79.7 | 34.4 | — | |
| Pix2StructTrain Data=ChartQA, Schema=Baseline2023.09 | 56 | 81.6 | 30.5 | — | |
| StructChart+Llama3.1-8bTrain Data=ChartQA+SimChart9K, Schema=LCT2023.09 | 55.8 | 69.8 | 41.7 | — | |
| DeplotTrain Data=ChartQA/PlotQA/C.D., Schema=Baseline2023.09 | 52.9 | 69.3 | 36.6 | — | |
| StructChart+Llama2-13bTrain Data=ChartQA+SimChart9K, Schema=STR2023.09 | 52.2 | 71.2 | 33.2 | — | |
| Qwen2VL-7B-Instruct + VTWLatency=0.2437, Speedup=1.46x2025.12 | 52.07 | — | — | 18.12 | |
| PaLI-17BTrain Data=ChartQA, Schema=Baseline2023.09 | 47.6 | 64.9 | 30.4 | — | |
| StructChart+Vicuna-13bTrain Data=ChartQA+SimChart9K, Schema=LCT2023.09 | 46.5 | 64.6 | 28.4 | — | |
| Tapas-OCRTrain Data=ChartQA, Schema=Baseline2023.09 | 45.5 | — | — | — | |
| StructChart+Llama2-13bTrain Data=ChartQA+SimChart9K, Schema=LCT2023.09 | 45.4 | 61.2 | 29.5 | — | |
| VL-T5-OCRTrain Data=ChartQA, Schema=Baseline2023.09 | 41.6 | — | — | — |