Visual Question Answering on TextVQA v1.0 (val)
85.5AccuracyQwen2-VL-72B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen2-VL-72B2024.12 | 85.5 | |
| Human2019.04 | 85.01 | |
| InternVL2-Llama3-76B2024.12 | 84.4 | |
| Qwen2-VL-7B2024.12 | 84.3 | |
| InternVL2.5-78B2024.12 | 83.4 | |
| Molmo-72B2024.12 | 83.1 | |
| InternVL2-40B2024.12 | 83 | |
| InternVL2.5-38B2024.12 | 82.7 | |
| InternVL2.5-26B2024.12 | 82.4 | |
| InternVL2-26B2024.12 | 82.3 | |
| NVLM-D-72B2024.12 | 82.1 | |
| Molmo-7B-D2024.12 | 81.7 | |
| InternVL-Chat-V1.52024.12 | 80.6 | |
| LLaVA-OneVision-72B2024.12 | 80.5 | |
| MiniCPM-V2.62024.12 | 80.1 | |
| Qwen2-VL-2B2024.12 | 79.7 | |
| InternVL2.5-8B2024.12 | 79.1 | |
| Gemini-1.5-Pro2024.12 | 78.8 | |
| GPT-4V2024.12 | 78 | |
| InternVL2-8B2024.12 | 77.4 | |
| GPT-4o-202405132024.12 | 77.4 | |
| InternVL2.5-4B2024.12 | 76.8 | |
| Cambrian-34B2024.12 | 76.7 | |
| Aquila-VL-2B2024.12 | 76.4 | |
| InternVL2-4B2024.12 | 74.4 | |
| InternVL2.5-2B2024.12 | 74.3 | |
| Claude-3.5-Sonnet2024.12 | 74.1 | |
| VILA-1.5-40B2024.12 | 73.6 | |
| InternVL2-2B2024.12 | 73.4 | |
| InternVL2.5-1B2024.12 | 72 | |
| Phi-3.5-Vision-4B2024.12 | 72 | |
| InternVL2-1B2024.12 | 70.5 | |
| LA+OCR UBVocab=LA2019.04 | 67.56 | |
| Claude-3-Opus2024.12 | 67.5 | |
| MobileVLM V2 7BLLM=Vicuna-7B, #Samples=3.6M2026.03 | 62.3 | |
| LaTr+-LargeOCR System=Amazon-OCR, Pre-Training Data=IDL, Extra Finetune=ST-VQA2021.12 | 61.05 | |
| ShareGPT4VLLM=Vicuna-7B, #Samples=1.9M2026.03 | 60.4 | |
| LaTr-LargeOCR System=Amazon-OCR, Pre-Training Data=IDL, Extra Finetune=None2021.12 | 59.76 | |
| LaTr+-BaseOCR System=Amazon-OCR, Pre-Training Data=IDL, Extra Finetune=ST-VQA2021.12 | 59.53 | |
| LLaVA-1.5LLM=Vicuna-7B, #Samples=1.2M2026.03 | 58.2 | |
| LaTr-BaseOCR System=Amazon-OCR, Pre-Training Data=IDL, Extra Finetune=None2021.12 | 58.03 | |
| TAPOCR System=Microsoft-OCR, Pre-Training Data=TextVQA, ST-VQA, TextCaps, OCR-CC, Extra Finetune=ST-VQA2021.12 | 54.71 | |
| Cosine KD w/ Beta-KD (Instance)LLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 54.2 | |
| Align-KD w/ Beta-KD (Instance)LLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 53.9 | |
| Cosine KD w/ Beta-KD (Task)LLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 53.9 | |
| Align-KD w/ Beta-KD (Task)LLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 52.9 | |
| Align-KDLLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 52.8 | |
| LaTr-BaseOCR System=Amazon-OCR, Pre-Training Data=None, Extra Finetune=None2021.12 | 52.29 | |
| Cosine KDLLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 52.2 | |
| LOGOSOCR System=Microsoft-OCR, Pre-Training Data=None, Extra Finetune=ST-VQA2021.12 | 51.53 | |
| MobileVLM -V2 1.7BLLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 51.2 | |
| TAPOCR System=Microsoft-OCR, Pre-Training Data=TextVQA, ST-VQA, Extra Finetune=ST-VQA2021.12 | 50.57 | |
| TAPOCR System=Microsoft-OCR, Pre-Training Data=TextVQA, Extra Finetune=None2021.12 | 49.91 | |
| LA UBVocab=LA2019.04 | 48.46 | |
| LaTr-BaseOCR System=Rosetta-en, Pre-Training Data=IDL, Extra Finetune=None2021.12 | 48.38 | |
| MoE-LLaVA-2BLLM=Qwen-1.5-1.8B, #Samples=2.2M2026.03 | 48 | |
| M4COCR System=Amazon-OCR, Pre-Training Data=None, Extra Finetune=None2021.12 | 47.84 | |
| MoE-LLaVA-1.6BLLM=StableLM-1.6B, #Samples=2.2M2026.03 | 47.8 | |
| MobileVLM 3BLLM=MobileLLaMA 2.7B, #Samples=3.6M2026.03 | 47.5 | |
| SA-M4COCR System=Google-OCR, Pre-Training Data=None, Extra Finetune=ST-VQA2021.12 | 45.4 | |
| LLAVADILLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 45.3 | |
| M4COCR System=Microsoft-OCR, Pre-Training Data=None, Extra Finetune=ST-VQA2021.12 | 45.22 | |
| TAPOCR System=Rosetta-en, Pre-Training Data=TextVQA, Extra Finetune=None2021.12 | 44.06 | |
| LaTr-BaseOCR System=Rosetta-en, Pre-Training Data=None, Extra Finetune=None2021.12 | 44.06 | |
| MobileVLM 1.7BLLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 41.5 | |
| LaAP-NetOCR System=Rosetta-en, Pre-Training Data=None, Extra Finetune=None2021.12 | 40.68 | |
| CRNOCR System=Rosetta-en, Pre-Training Data=None, Extra Finetune=None2021.12 | 40.39 | |
| SMAOCR System=Rosetta-en, Pre-Training Data=None, Extra Finetune=None2021.12 | 40.05 | |
| M4COCR System=Rosetta-en, Pre-Training Data=None, Extra Finetune=None2021.12 | 39.4 | |
| OCR UBDescription=Upper bound from OCR tokens2019.04 | 37.12 | |
| LoRRA (Pythia backbone)Vocab=SA2019.04 | 26.56 | |
| LoRRA (Pythia backbone)Vocab=LA2019.04 | 26.23 | |
| Pythia (I+Q) + O + CVocab=n/a, Features=Image + Question + OCR + Copy Mechanism2019.04 | 20.06 | |
| LoRRA (BAN backbone)Vocab=SA2019.04 | 18.41 | |
| Pythia (I+Q) + OVocab=LA, Features=Image + Question + OCR features2019.04 | 18.35 | |
| Pythia (I+Q)Vocab=LA, Features=Image + Question2019.04 | 13.04 | |
| BAN (I+Q)Vocab=LA2019.04 | 12.3 | |
| OCR Max2019.04 | 9.76 | |
| QVocab=LA, Features=Question only2019.04 | 8.09 | |
| Random OCR2019.04 | 7.72 | |
| IVocab=LA, Features=Image only2019.04 | 6.29 | |
| Majority Ans2019.04 | 4.48 | |
| Wt. Rand 1002019.04 | 0.27 | |
| Rand 1002019.04 | 0.22 |