Multi-page Document Question Answering on MP-DocVQA (test)
0.8458ANLSAVIR
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| AVIROCR=false, Params.=3B2026.01 | 0.8458 | 81.55 | |
| M3DOCRAGYear=2024, OCR=false, Params.=8B2026.01 | 0.8444 | 81.05 | |
| Qwen2.5-3B-AWQYear=2025, OCR=false, Params.=3B2026.01 | 0.8405 | — | |
| Arctic-TILTYear=2024, OCR=true, Params.=800M2026.01 | 0.8122 | — | |
| GRAMYear=2024, OCR=true, Params.=859M2026.01 | 0.8032 | — | |
| ScreenAIYear=2024, OCR=true, Params.=5B2026.01 | 0.7711 | 77.88 | |
| ScreenAIOCR Usage=With OCR2024.02 | 0.771 | — | |
| ScreenAIOCR Usage=Without OCR2024.02 | 0.729 | — | |
| RMT5Year=2024, OCR=true, Params.=312M2026.01 | 0.6401 | 88.32 | |
| Hi-VT5Paradigm=Fine-tuning, Setup=Multipage2023.06 | 0.6201 | — | |
| Hi-VT5Year=2023, OCR=true, Params.=316M2026.01 | 0.6201 | 79.23 | |
| SelfAttnScoringYear=2024, OCR=false, Params.=273M2026.01 | 0.6199 | 81.55 | |
| Hi-VT5OCR Usage=SOTA Overall2024.02 | 0.618 | — | |
| Hi-VT5OCR Usage=With OCR2024.02 | 0.618 | — | |
| Claude + LATIN-PromptParadigm=Zero-shot, Setup=Max Conf2023.06 | 0.6129 | — | |
| Big BirdParadigm=Fine-tuning, Setup=Max Conf2023.06 | 0.5854 | — | |
| Big BirdYear=2020, OCR=true, Params.=131M2026.01 | 0.5854 | 72.27 | |
| LayoutLMv3Paradigm=Fine-tuning, Setup=Max Conf2023.06 | 0.5513 | — | |
| LayoutLMv3Year=2022, OCR=true, Params.=125M2026.01 | 0.5513 | 74.02 | |
| LongformerParadigm=Fine-tuning, Setup=Max Conf2023.06 | 0.5506 | — | |
| LongformerYear=2020, OCR=true, Params.=148M2026.01 | 0.5506 | 70.37 | |
| BERTParadigm=Fine-tuning, Setup=Max Conf2023.06 | 0.5347 | — | |
| BERTYear=2018, OCR=true, Params.=334M2026.01 | 0.5347 | 71.24 | |
| LongformerParadigm=Fine-tuning, Setup=Concat2023.06 | 0.5287 | — | |
| T5Paradigm=Fine-tuning, Setup=Concat2023.06 | 0.505 | — | |
| Big BirdParadigm=Fine-tuning, Setup=Concat2023.06 | 0.4929 | — | |
| LayoutLMv3Paradigm=Fine-tuning, Setup=Concat2023.06 | 0.4538 | — | |
| BERTParadigm=Fine-tuning, Setup=Concat2023.06 | 0.4183 | — | |
| T5Paradigm=Fine-tuning, Setup=Max Conf2023.06 | 0.4028 | — | |
| T5Year=2020, OCR=true, Params.=223M2026.01 | 0.4028 | 46.05 |