Chart Question Answering on ChartQA human questions subset (test)
67.6Relaxed AccuracyDEPLOT + FlanPaLM + Codex
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DEPLOT + FlanPaLM + Codexparams=540B2024.03 | 67.6 | — | |
| MATCHA + LAMENDAparams=282M, resolution=4096 tokens2024.03 | 53.92 | 44.88 | |
| MATCHA + LAMENDAparams=282M, resolution=1024 tokens2024.03 | 51.76 | 41.92 | |
| Pix2Struct + LAMENDAparams=282M, resolution=4096 tokens2024.03 | 51.67 | 40.1 | |
| Pix2Struct + LAMENDAparams=282M, resolution=1024 tokens2024.03 | 46.25 | 34.69 | |
| MATCHAparams=282M, resolution=4096 tokens2024.03 | 38.2 | — | |
| MATCHAparams=282M, resolution=4096 tokens, implementation=re-implemented2024.03 | 37.76 | 31.04 | |
| Pix2Structparams=282M, resolution=4096 tokens, implementation=re-implemented2024.03 | 32 | 24.64 | |
| Pix2Structparams=282M, resolution=4096 tokens2024.03 | 30.5 | — |