Visual Question Answering on AI2D (test)
96.4AccuracyMolmoPoint-8B
Evaluation Results
| Method | Links | |
|---|---|---|
| MolmoPoint-8BAccess=MolmoPoint2026.03 | 96.4 | |
| Molmo-72BAccess=Open weights, data, code, Number of crops=36, Style Prompting=AI2D with transparent boxes2024.09 | 96.3 | |
| Gemini 2.5 FlashAccess=API call only2026.03 | 95.9 | |
| Molmo2-8BAccess=Fully open2026.03 | 95.8 | |
| Molmo2-4BAccess=Fully open2026.03 | 95.6 | |
| Claude-3.5 SonnetAccess=API2024.09 | 94.7 | |
| Gemini 1.5 ProAccess=API2024.09 | 94.4 | |
| Gemini 2.5 ProAccess=API call only2026.03 | 94.3 | |
| GPT-4o-0513Access=API2024.09 | 94.2 | |
| MolmoPoint-8B-O-7BAccess=Fully open2026.03 | 93.7 | |
| Molmo-7B-DAccess=Open weights, data, code, Number of crops=36, Style Prompting=AI2D with transparent boxes2024.09 | 93.2 | |
| PLM-8BAccess=Fully open2026.03 | 92.7 | |
| Llama-3.2V-90B-InstructAccess=Open Weights2024.09 | 92.3 | |
| Gemini 1.5 FlashAccess=API2024.09 | 91.7 | |
| Claude Sonnet 4.5Access=API call only2026.03 | 91.5 | |
| Llama-3.2V-11B-InstructAccess=Open Weights2024.09 | 91.1 | |
| PLM-3BAccess=Fully open2026.03 | 90.9 | |
| Molmo-7B-OAccess=Open weights, data, code, Number of crops=36, Style Prompting=AI2D with transparent boxes2024.09 | 90.7 | |
| Zamba2-VL-7BLanguage-backbone scale=7–8B2026.05 | 90.6 | |
| GPT-5Access=API call only2026.03 | 89.5 | |
| Keye-VL-1.5-8BAccess=Open weights only2026.03 | 89.5 | |
| GPT-4VAccess=API2024.09 | 89.4 | |
| Claude-3 OpusAccess=API2024.09 | 88.1 | |
| Qwen2-VL-72BAccess=Open Weights2024.09 | 88.1 | |
| GLM-4.1V-9BAccess=Open weights only2026.03 | 87.9 | |
| InternVL2-Llama-3-76BAccess=Open Weights2024.09 | 87.6 | |
| Claude-3 HaikuAccess=API2024.09 | 86.7 | |
| GPT-5 miniAccess=API call only2026.03 | 86.7 | |
| MiniCPM-V-4.5-8BAccess=Open weights only2026.03 | 86.5 | |
| MolmoE-1BAccess=Open weights, data, code, Number of crops=36, Style Prompting=AI2D with transparent boxes2024.09 | 86.4 | |
| Zamba2-VL-2.7BLanguage-backbone scale=2–4B2026.05 | 85.8 | |
| Qwen3-VL-8BAccess=Open weights only2026.03 | 85.7 | |
| LLaVA OneVision-72BAccess=Open Weights + Data, distilled=true2024.09 | 85.6 | |
| Eagle2.5-8BAccess=Open weights only2026.03 | 84.5 | |
| Qwen3-VL-4BAccess=Open weights only2026.03 | 84.1 | |
| InternVL3.5-8BAccess=Open weights only2026.03 | 84 | |
| InternVL2-8BAccess=Open Weights2024.09 | 83.8 | |
| Mamba-VL-2.8BLanguage-backbone scale=2–4B2026.05 | 83.7 | |
| Qwen2-VL-7BAccess=Open Weights2024.09 | 83 | |
| InternVL3.5-4BAccess=Open weights only2026.03 | 82.6 | |
| SMoLA-PaLI-X_FT (Specialist)Base model=PaLI-X, Per-task LoRA tuning=true, LoRA rank=42023.12 | 82.5 | |
| jina-vlm2025.12 | 82 | |
| Pythia-VL-2.8BLanguage-backbone scale=2–4B2026.05 | 81.6 | |
| Zamba2-VL-1.2BLanguage-backbone scale=~1B2026.05 | 81.5 | |
| PaLI-XOCR pipeline input=true2023.05 | 81.4 | |
| SMoLA-PaLI-XFTOCR pipeline input=false, Model Category=Generalist, Backbone=PaLI-X2023.12 | 81.4 | |
| PaLI-X (Specialist)OCR pipeline input=true, Model Category=Specialist, Reference Citation=[7]2023.12 | 81.4 | |
| SMoLA-PaLI-XFTOCR pipeline input=true, Model Category=Generalist, Backbone=PaLI-X2023.12 | 81.4 | |
| PaLI-XBase model=PaLI-X2023.12 | 81.4 | |
| SOTA [7]2023.12 | 81.4 | |
| LLaVA OneVision-7BAccess=Open Weights + Data, distilled=true2024.09 | 81.4 | |
| PaLI-XOCR pipeline input=false2023.05 | 81.2 | |
| PaLI-X (Specialist)OCR pipeline input=false, Model Category=Specialist, Reference Citation=[7]2023.12 | 81.2 | |
| Mamba-VL-1.4BLanguage-backbone scale=~1B2026.05 | 80.9 | |
| Cambrian-1-34BAccess=Open Weights + Data, distilled=true2024.09 | 79.7 | |
| Mamba-VL-0.8BLanguage-backbone scale=~1B2026.05 | 79.3 | |
| Pythia-VL-1.4BLanguage-backbone scale=~1B2026.05 | 79.3 | |
| Pixtral-12BAccess=Open Weights2024.09 | 79 | |
| InternVL3.5-2BParameter Count=2B2025.12 | 78.8 | |
| InternVL3-2BParameter Count=2B2025.12 | 78.6 | |
| Phi3.5-Vision-4BAccess=Open Weights2024.09 | 78.1 | |
| Pythia-VL-1BLanguage-backbone scale=~1B2026.05 | 77.6 | |
| Qwen3-VL-2BParameter Count=2B2025.12 | 76.9 | |
| CogVLMOCR pipeline input=false, Model Category=Generalist2023.12 | 75.7 | |
| SMoLA-PaLI-3FTOCR pipeline input=false, Model Category=Generalist, Backbone=PaLI-32023.12 | 75.6 | |
| SMoLA-PaLI-3FTOCR pipeline input=true, Model Category=Generalist, Backbone=PaLI-32023.12 | 75.5 | |
| Qwen2-VL-2BParameter Count=2B2025.12 | 74.7 | |
| xGen-MM-interleave-4BAccess=Open Weights + Data, distilled=true2024.09 | 74.2 | |
| Cambrian-1-8BAccess=Open Weights + Data, distilled=true2024.09 | 73 | |
| PaliGemma-mix-3BAccess=Open Weights2024.09 | 72.3 | |
| Qwen-VLOCR pipeline input=false, Model Category=Generalist2023.12 | 62.3 | |
| LLaVA-1.5-13BAccess=Open Weights + Data2024.09 | 61.1 | |
| LLaVA-1.5-7BAccess=Open Weights + Data2024.09 | 55.5 | |
| Method [27]OCR pipeline input=false2023.05 | 42.1 | |
| Pix2Struct LargePretraining=Screenshot parsing, Pixel only=true2022.10 | 42.1 | |
| Pix2Struct BasePretraining=Screenshot parsing, Pixel only=true2022.10 | 40.9 | |
| Method [45]OCR pipeline input=true2023.05 | 38.5 | |
| DQANtype=State of the art w/ pipelines2022.10 | 38.5 | |
| DonutPretraining=OCR, Pixel only=true2022.10 | 30.8 | |
| LAMMSetting=Zero-shot2023.06 | 20.92 | |
| mPLUG-owlSetting=Zero-shot2023.06 | 19.31 | |
| LLaVASetting=Zero-shot2023.06 | 18.13 |