Diagram Question Answering on AI2D (test)
94.7AccuracyClaude 3.5
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude 3.5evaluation_protocol=transparent bounding box2024.07 | 94.7 | |
| Claude 3.5 Sonnet2024.12 | 94.7 | |
| Gemini 1.5 Proevaluation_protocol=transparent bounding box2024.07 | 94.4 | |
| Gemini-1.5-Pro2024.12 | 94.4 | |
| Gemini 1.5 Pro2024.12 | 94.4 | |
| GPT-4oevaluation_protocol=transparent bounding box2024.07 | 94.2 | |
| GPT-4o2024.12 | 94.2 | |
| GPT-4o2024.12 | 94.2 | |
| NVLM-D-1.0# Parameters=78B2024.12 | 94.2 | |
| Llama 3-V 405Bevaluation_protocol=transparent bounding box2024.07 | 94.1 | |
| NVILA# Parameters=15B2024.12 | 94.1 | |
| GenRecalstudent=Qwen2-VL-7B, teacher=InternVL2.5-78B2025.06 | 93.9 | |
| Llama 3-V 70Bevaluation_protocol=transparent bounding box2024.07 | 93 | |
| GenRecalstudent=InternVL2.5-8B, teacher=InternVL2.5-78B2025.06 | 93 | |
| NVILA# Parameters=8B2024.12 | 92.3 | |
| Llama 3.2# Parameters=90B2024.12 | 92.3 | |
| NVILA-Lite# Parameters=15B2024.12 | 92 | |
| Llama 3.2# Parameters=11B2024.12 | 91.9 | |
| GenRecalstudent=InternVL2.5-2B, teacher=InternVL2.5-78B2025.06 | 91.8 | |
| NVILA-Lite# Parameters=8B2024.12 | 91 | |
| GenRecalstudent=InternVL2.5-4B, teacher=InternVL2.5-78B2025.06 | 90 | |
| GenRecalstudent=Qwen2-VL-2B, teacher=InternVL2.5-78B2025.06 | 89.1 | |
| VILA-1.5# Parameters=40B2024.12 | 88.9 | |
| InternVL2# Parameters=40B2024.12 | 87.1 | |
| LLaVA-OneVision# Parameters=72B2024.12 | 85.6 | |
| Qwen3-VL-32B-InstructContext Window=32K2026.03 | 85 | |
| InternVL2.5size=8B2025.06 | 84.8 | |
| Phi-4-reasoning-vision-15B2026.03 | 84.8 | |
| Qwen3-VL-32B-InstructContext Window=4K2026.03 | 84.8 | |
| Phi-4-reasoning-vision-15BProtocol=force nothink2026.03 | 84.7 | |
| GPT-4oZero-shot=true, Decoding Strategy=greedy2024.09 | 84.6 | |
| Kimi-VL-A3B-Instruct2026.03 | 84.6 | |
| Llama 3-V 8Bevaluation_protocol=transparent bounding box2024.07 | 84.4 | |
| Qwen2.5VLSize=7B, Type=AR, Samples=>9M2025.12 | 83.9 | |
| InternVL2Size=8B, #token /image tile=2562024.12 | 83.8 | |
| PVC InternVL2 (Ours)Size=8B, #token /image tile=642024.12 | 83.8 | |
| InternVL2# Parameters=8B2024.12 | 83.8 | |
| InternVL2size=8B2025.06 | 83.8 | |
| Qwen2-VL-InstructLLM=Qwen2-7B, Vision Encoder=DFN-CLIP-H2024.12 | 83 | |
| Qwen2-VLSize=7B, #token /image tile=Native resolution (unfixed)2024.12 | 83 | |
| Qwen2-VL# Parameters=8B2024.12 | 83 | |
| InternVL-2-26BModel=InternVL-2-26B2026.01 | 83 | |
| Qwen3-VL-8B-InstructContext Window=32K2026.03 | 83 | |
| GenRecalstudent=InternVL2.5-1B, teacher=InternVL2.5-78B2025.06 | 82.9 | |
| Qwen3-VL-8B-InstructContext Window=4K2026.03 | 82.7 | |
| DiffusionVLSize=7B, Type=Diff., Samples=738K2025.12 | 82.2 | |
| InternVL-2-8BModel=InternVL-2-8B2026.01 | 82.2 | |
| MiniCPM-V2.6size=8B2025.06 | 82.1 | |
| LLaVA-OV (SI)LLM=Qwen2-7B, Vision Encoder=SigLIP-SO400M2024.12 | 81.6 | |
| Qwen2.5VLSize=3B, Type=AR, Samples=>9M2025.12 | 81.6 | |
| IXC-2.5Size=7B, #token /image tile=4002024.12 | 81.5 | |
| LLaVA-OVLLM=Qwen2-7B, Vision Encoder=SigLIP-SO400M2024.12 | 81.4 | |
| LLaVA-OVSize=7B, #token /image tile=7292024.12 | 81.4 | |
| LLaVA-OneVision# Parameters=8B2024.12 | 81.4 | |
| LLaVA-OVSize=7B, Type=AR, Samples=7.8M2025.12 | 81.4 | |
| LLaVA-OneVisionsize=7B2025.06 | 81.4 | |
| InternVL2.5size=4B2025.06 | 81.4 | |
| gemma-3-12b-it2026.03 | 80.4 | |
| VILA-1.5# Parameters=13B2024.12 | 79.9 | |
| Cambrian-34BModel Scale=30B, Zero-shot=true, Decoding Strategy=greedy2024.09 | 79.7 | |
| Cambrian-1# Parameters=34B2024.12 | 79.7 | |
| Gemini-1.5-ProZero-shot=true, Decoding Strategy=greedy2024.09 | 79.1 | |
| Pixtral# Parameters=12B2024.12 | 79 | |
| InternVL2size=4B2025.06 | 78.9 | |
| LLaVA-NeXT-INST-ITLLM=Qwen2-7B, Vision Encoder=SigLIP-SO4002024.12 | 78.7 | |
| Oryx MLLMSize=7B2024.12 | 78.5 | |
| DiffusionVLSize=3B, Type=Diff., Samples=738K2025.12 | 78.4 | |
| MiniCPM-V2.5size=8B2025.06 | 78.4 | |
| GPT-4Vevaluation_protocol=transparent bounding box2024.07 | 78.2 | |
| GPT-4V2024.12 | 78.2 | |
| LLaVA-OV-7B w/ CLIModel=LLaVA-OV-7B w/ CLI2026.01 | 77.9 | |
| LLaDA-VSize=8B, Type=Diff., Samples=16.5M2025.12 | 77.8 | |
| Phi-3.5-Visionsize=4B2025.06 | 77.8 | |
| LLaVA-OV-7B w/ SLIModel=LLaVA-OV-7B w/ SLI2026.01 | 77.6 | |
| LLaVA-OV-7BModel=LLaVA-OV-7B2026.01 | 77.5 | |
| Qwen2-VLsize=7B2025.06 | 77.5 | |
| MM1.5-30BModel Scale=30B, Zero-shot=true, Decoding Strategy=greedy2024.09 | 77.2 | |
| Phi-3-Vision-4BModel Scale=3B, Zero-shot=true, Decoding Strategy=greedy2024.09 | 76.7 | |
| VILA-1.5# Parameters=8B2024.12 | 76.6 | |
| PVC InternVL2 (Ours)Size=2B, #token /image tile=642024.12 | 76.3 | |
| Eaglesize=8B2025.06 | 76.1 | |
| LLaVA-OV-7B w/ DeepStackModel=LLaVA-OV-7B w/ DeepStack2026.01 | 76 | |
| GPT-4VZero-shot=true, Decoding Strategy=greedy2024.09 | 75.9 | |
| Aquila-VLsize=2B2025.06 | 75 | |
| InternVL2.5size=2B2025.06 | 74.9 | |
| Qwen2-VLSize=2B, #token /image tile=Native resolution (unfixed)2024.12 | 74.7 | |
| DimpleSize=7B, Type=Diff., Samples=1.3M2025.12 | 74.4 | |
| Florence-VL# Parameters=8B2024.12 | 74.2 | |
| InternVL2-2BModel Scale=1B, Zero-shot=true, Decoding Strategy=beam search2024.09 | 74.1 | |
| InternVL2Size=2B, #token /image tile=2562024.12 | 74.1 | |
| InternVL2size=2B2025.06 | 74.1 | |
| Eaglesize=13B2025.06 | 74 | |
| Cambrian-1# Parameters=13B2024.12 | 73.6 | |
| Cambrian-1size=13B2025.06 | 73.6 | |
| LLaVA-NeXT-Qwen22024.06 | 73.5 | |
| CogVLM2size=19B2025.06 | 73.4 | |
| MM1-30BModel Scale=30B, Zero-shot=true, Decoding Strategy=greedy2024.09 | 73.3 | |
| LongVAResolution Strategy=AnyRes2024.06 | 73.1 | |
| Cambrian-1# Parameters=8B2024.12 | 73 | |
| Cambrian-1Size=8B, Type=AR, Samples=-2025.12 | 73 |