Document Image Question Answering on InfographicVQA (test)
0.612ANLSTILT
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TILTParadigm=Fine-tuning2023.06 | 0.612 | — | 67.65 | 64.19 | 43.91 | 38.32 | 59.17 | 79.16 | 45.45 | 56.54 | 44.8 | 48.01 | 49.58 | 26.52 | |
| Claude + LATIN-PromptParadigm=Zero-shot2023.06 | 0.5451 | 0.4653 | 59.92 | 58.61 | 39.85 | 35.44 | 54.21 | 67.25 | 48.97 | 50.27 | 49.82 | 45.98 | 43.11 | 27.08 | |
| ChatGPT-3.5 + LATIN-PromptParadigm=Zero-shot2023.06 | 0.4898 | 0.1563 | 54.57 | 56.39 | 34.58 | 27.98 | 49.17 | 60.16 | 44.91 | 45.85 | 36.14 | 43.12 | 31.57 | 26.6 | |
| pix2structParadigm=Fine-tuning2023.06 | 0.4001 | — | 43.08 | 48.39 | 20.59 | 31.73 | 38.33 | 52.56 | 25.72 | 37.26 | 32.83 | 27.62 | 41.98 | 20.17 | |
| ChatGPT-3.5 + Plain PromptParadigm=Zero-shot2023.06 | 0.3335 | — | 37.49 | 45.05 | 9.5 | 18.22 | 34.81 | 38.93 | 36.7 | 31.14 | 18.43 | 23.49 | 14.66 | 23.2 | |
| BROSParadigm=Fine-tuning2023.06 | 0.3219 | — | 39.97 | 23.17 | 10.64 | 10.68 | 26.53 | 44.88 | 18.78 | 30.95 | 32.31 | 20.2 | 14.8 | 6.95 | |
| LayoutLMv2Paradigm=Fine-tuning2023.06 | 0.2829 | — | 34.3 | 27.63 | 6.41 | 11.14 | 24.49 | 38.55 | 14.4 | 26.01 | 31.1 | 18.97 | 11.3 | 11.58 | |
| LayoutLMParadigm=Fine-tuning2023.06 | 0.272 | — | 32.78 | 23.86 | 4.5 | 13.71 | 24 | 36.26 | 17.05 | 25.51 | 22.05 | 18.36 | 15.59 | 11.4 | |
| BERTParadigm=Fine-tuning2023.06 | 0.2078 | — | 26.25 | 23.33 | 7.39 | 2.59 | 18.52 | 29.95 | 8.96 | 19.42 | 17.09 | 18.05 | 1.6 | 4.36 | |
| Claude + Plain PromptParadigm=Zero-shot2023.06 | 0.0798 | — | 9.51 | 9.13 | 2.03 | 2.8 | 8.49 | 10.99 | 8.58 | 6.95 | 4.96 | 5.89 | 2.71 | 3.68 |