Plot Question Answering on PlotQA V1 & V2 (test)
92.89Relaxed Accuracy (avg)MATCHA4096 + LAMENDA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MATCHA4096 + LAMENDAResolution=4096, LAMENDA=true2024.03 | 92.89 | 51.4 | 76.42 | 26.38 | 93.94 | 91.84 | |
| MATCHA4096Resolution=4096, reimpl.=true2024.03 | 91.97 | 50.89 | 76.14 | 25.64 | 92.64 | 91.3 | |
| MATCHA2024.03 | 91.5 | — | — | — | 92.3 | 90.7 | |
| MATCHA1024 + LAMENDAResolution=1024, LAMENDA=true2024.03 | 78.41 | 43.04 | 68.48 | 17.6 | 78.44 | 78.38 | |
| MATCHA1024Resolution=1024, reimpl.=true2024.03 | 74.95 | 41.58 | 66.66 | 16.5 | 73.88 | 76.02 | |
| Pix2Struct2024.03 | 72.5 | — | — | — | 73.2 | 71.9 | |
| DEPLOT+FlanPaLM(540B)+CodexParameters=540B2024.03 | 66.6 | — | — | — | 62.2 | 71 | |
| VL-T5-OCR2024.03 | 65.96 | — | — | — | 75.9 | 56.02 | |
| VisionTapas-OCR2024.03 | 53.9 | — | — | — | 65.3 | 42.5 |