Visual Question Answering on TextVQA (test)
81.1AccuracySMoLA-PaLI-XFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SMoLA-PaLI-XFTOCR pipeline input=true, Model Category=Generalist, Backbone=PaLI-X2023.12 | 81.1 | — | |
| PaLI-X (Specialist)OCR pipeline input=true, Model Category=Specialist, Reference Citation=[7]2023.12 | 80.8 | — | |
| PaLI-3 (Specialist)OCR pipeline input=false, Model Category=Specialist, Reference Citation=[8]2023.12 | 79.5 | — | |
| FP16 (Baseline)Bits=FP162026.02 | 76.1 | — | |
| QuEPTBits=W32026.02 | 74.1 | — | |
| MiaVocabulary setting=Open-vocabulary generation, base_model=T5-XL2022.09 | 73.67 | — | |
| Mia#Model type=Fine-tuned T5-3B2022.05 | 73.67 | — | |
| Mia2022.05 | 73.67 | — | |
| Miamodel_base=T5-3B, config=fine-tuned, status=winner entry of TextVQA Challenge 20212022.05 | 73.67 | — | |
| MBQBits=W32026.02 | 73.3 | — | |
| PaLi-17Bpre-train data=1.6B, proprietary_data=true, #param=17B2023.06 | 73.1 | — | |
| PaLI-17BVocabulary setting=Open-vocabulary generation2022.09 | 73.06 | — | |
| QuEPTBits=W4A82026.02 | 71.5 | — | |
| InfiMM-HDResolution=dynamic, In-house data=false2024.03 | 70.7 | — | |
| SMoLA-PaLI-XFTOCR pipeline input=false, Model Category=Generalist, Backbone=PaLI-X2023.12 | 70.5 | — | |
| AWQBits=W32026.02 | 70 | — | |
| MBQBits=W4A82026.02 | 68.3 | — | |
| CogVLMOCR pipeline input=false, Model Category=Generalist2023.12 | 68.1 | — | |
| MonkeyResolution=1344x768, In-house data=true2024.03 | 67.6 | — | |
| GIT2+pre-train data=12.9B, proprietary_data=true, extra_data=8 VQA datasets, #param=5.1B2023.06 | 67.3 | — | |
| GIT2Parameters=5.1B, Vocabulary setting=Closed-vocabulary classification2022.09 | 67.27 | — | |
| GIT2Parameters=5.1B2022.05 | 67.27 | — | |
| QuEPTBits=W4A42026.02 | 64.4 | — | |
| DocFormerv2-largepre-train data=64M, #param=750M2023.06 | 64 | — | |
| QuEPTBits=W22026.02 | 64 | — | |
| Qwen-VLOCR pipeline input=false, Model Category=Generalist2023.12 | 63.8 | — | |
| Prophet++backbone=mPLUG2023.03 | 61.8 | — | |
| Prophet++VQA model=mPLUG2023.03 | 61.8 | — | |
| LaTr-largepre-train data=64M, #param=856M2023.06 | 61.6 | — | |
| LaTr2022.05 | 61.6 | — | |
| Qwen-VL-ChatResolution=448x448, In-house data=true2024.03 | 61.5 | — | |
| Prophetbackbone=mPLUG2023.03 | 61.3 | — | |
| ProphetVQA model=mPLUG2023.03 | 61.3 | — | |
| LLaVA-Next-7BRetained Tokens=2880 (100%)2026.02 | 61.3 | — | |
| Sphinx-2KResolution=768x768, In-house data=false2024.03 | 61.2 | — | |
| DocPediaResolution=2560x2560, In-house data=false2024.03 | 60.2 | — | |
| DocFormerv2-basepre-train data=64M, #param=232M2023.06 | 60 | — | |
| GITpre-train data=800M, proprietary_data=true, #param=681M2023.06 | 59.8 | — | |
| GITParameters=0.7B, Vocabulary setting=Closed-vocabulary classification2022.09 | 59.75 | — | |
| GITParameters=0.7B2022.05 | 59.75 | — | |
| GIT2022.05 | 59.75 | — | |
| GIT2022.05 | 59.75 | — | |
| LaTr-basepre-train data=64M, #param=311M, combined_training_sets=true2023.06 | 59.6 | — | |
| LaTr2023.03 | 59.6 | — | |
| LaTr2023.03 | 59.6 | — | |
| mPLUG-Owl-7B + OursModel Params=7.2B, Trainable Params=96M, Pre-training Data=1.1B, Fine-tuning Data=650K2024.11 | 59.2 | — | |
| LaTr-basepre-train data=64M, #param=311M2023.06 | 58.9 | — | |
| EntropyPruneRetained Tokens=320 (↓ 88.9%)2026.02 | 58.5 | — | |
| DARTRetained Tokens=320 (↓ 88.9%)2026.02 | 58 | — | |
| mPLUG-Owl-7B + UReaderModel Params=7.2B, Trainable Params=86M, Pre-training Data=1.1B, Fine-tuning Data=650K2024.11 | 57.6 | — | |
| CDPrunerRetained Tokens=320 (↓ 88.9%)2026.02 | 57.4 | — | |
| BLIP-2-OPT-2.7B + OursModel Params=3.8B, Trainable Params=14M, Pre-training Data=129M, Fine-tuning Data=650K2024.11 | 57.3 | — | |
| SmoothQBits=W4A82026.02 | 56.9 | — | |
| PreSTUpre-train data=13M, #param=237M2023.06 | 56.3 | — | |
| TAP Two-Stagepre-train data=200M2023.06 | 55.3 | — | |
| Flamingopre-train data=2.3B, video-text data=true, #param=80B2023.06 | 54.1 | — | |
| FlamingoParameters=80B, Vocabulary setting=Open-vocabulary generation2022.09 | 54.1 | — | |
| Flamingo-80Bnumber of parameters=80B2023.03 | 54.1 | — | |
| Flamingo-80BParameters=80B2023.03 | 54.1 | — | |
| Flamingo2022.05 | 54.1 | — | |
| Flamingo2022.05 | 54.1 | — | |
| Flamingo2022.05 | 54.1 | — | |
| TAPpre-train data=200M2023.06 | 54 | — | |
| TAP2023.03 | 54 | — | |
| TAP2023.03 | 54 | — | |
| TAP2022.05 | 53.97 | — | |
| TAP2022.05 | 53.97 | — | |
| TAP2022.05 | 53.97 | — | |
| TAP + TAG2023.06 | 53.7 | — | |
| TAP + TAGOCR system=Microsoft-OCR, Extra Data=ST-VQA2022.08 | 53.69 | — | |
| mPLUG2023.03 | 53.5 | — | |
| mPLUG2023.03 | 53.5 | — | |
| PDropRetained Tokens=320 (↓ 88.9%)2026.02 | 53.2 | — | |
| mPLUG-DocOwlOCR pipeline input=false, Model Category=Generalist2023.12 | 52.6 | — | |
| TAP + TAGOCR system=Microsoft-OCR, Extra Data=None2022.08 | 52.57 | — | |
| PromptCap2023.03 | 51.9 | — | |
| PromptCap2023.03 | 51.9 | — | |
| DivPruneRetained Tokens=320 (↓ 88.9%)2026.02 | 51.4 | — | |
| LOGOS2023.06 | 51.1 | — | |
| LOGOSOCR system=Microsoft-OCR, Extra Data=ST-VQA2022.08 | 51.08 | — | |
| MTVBackbone=LLaMA-3-8B, Method=Multimodal Task Vectors2024.06 | 51 | — | |
| TAPOCR system=Microsoft-OCR, Extra Data=ST-VQA2022.08 | 50.71 | — | |
| LOGOSOCR system=Microsoft-OCR, Extra Data=None2022.08 | 50.65 | — | |
| TAPOCR system=Microsoft-OCR, Extra Data=None2022.08 | 49.71 | — | |
| FastVRetained Tokens=320 (↓ 88.9%)2026.02 | 49 | — | |
| LLaVA 1.5Resolution=336x336, In-house data=false2024.03 | 48.5 | — | |
| 8-shot ICLBackbone=LLaMA-3-8B, Number of shots=8, Method=In-Context Learning2024.06 | 47.1 | — | |
| M4C† + TAGOCR system=Microsoft-OCR, Extra Data=ST-VQA2022.08 | 46.38 | — | |
| M4C† + TAGOCR system=Microsoft-OCR, Extra Data=None2022.08 | 45.96 | — | |
| SC-Net2023.06 | 45.7 | — | |
| SSBaselineOCR system=SBD-Trans OCR, Extra Data=ST-VQA2022.08 | 45.66 | — | |
| SMAOCR system=SBD-Trans OCR, Extra Data=ST-VQA2022.08 | 45.51 | — | |
| SMA2022.05 | 45.51 | — | |
| SMA2022.05 | 45.51 | — | |
| SMA2022.05 | 45.51 | — | |
| SMA2023.06 | 45.5 | — | |
| 4-shot ICLBackbone=LLaMA-3-8B, Number of shots=4, Method=In-Context Learning2024.06 | 45.4 | — | |
| M4C†OCR system=Microsoft-OCR, Extra Data=None2022.08 | 44.75 | — | |
| SSBaselineOCR system=SBD-Trans OCR, Extra Data=None2022.08 | 44.72 | — | |
| SA-M4COCR system=Google-OCR, Extra Data=ST-VQA2022.08 | 44.6 | — |