Diagram Understanding on AI2D 1.0 (test)
96.3AccuracyMolmo-72B
Evaluation Results
| Method | Links | |
|---|---|---|
| Molmo-72Bmasking=without2024.12 | 96.3 | |
| InternVL2.5-78Bmasking=without2024.12 | 95.7 | |
| InternVL2.5-38Bmasking=without2024.12 | 95.1 | |
| InternVL2-Llama3-76Bmasking=without2024.12 | 94.8 | |
| Claude-3.5-Sonnetmasking=without2024.12 | 94.7 | |
| InternVL2-40Bmasking=without2024.12 | 94.5 | |
| InternVL2.5-26Bmasking=without2024.12 | 94.4 | |
| Gemini-1.5-Promasking=without2024.12 | 94.4 | |
| GPT-4o-20240513masking=without2024.12 | 94.2 | |
| NVLM-D-72Bmasking=without2024.12 | 94.2 | |
| Molmo-7B-Dmasking=without2024.12 | 93.2 | |
| InternVL2.5-8Bmasking=without2024.12 | 92.8 | |
| InternVL2-26Bmasking=without2024.12 | 92.5 | |
| Qwen2-VL-7Bmasking=without2024.12 | 92.1 | |
| InternVL2-8Bmasking=without2024.12 | 91.7 | |
| InternVL2.5-4Bmasking=without2024.12 | 90.5 | |
| InternVL-Chat-V1.5masking=without2024.12 | 89.8 | |
| GPT-4Vmasking=without2024.12 | 89.4 | |
| InternVL2.5-78Bmasking=with2024.12 | 89.1 | |
| Claude-3-Opusmasking=without2024.12 | 88.1 | |
| Qwen2-VL-72Bmasking=with2024.12 | 88.1 | |
| InternVL2-4Bmasking=without2024.12 | 87.8 | |
| Phi-3.5-Vision-4Bmasking=without2024.12 | 87.6 | |
| InternVL2.5-38Bmasking=with2024.12 | 87.6 | |
| InternVL2-Llama3-76Bmasking=with2024.12 | 87.6 | |
| InternVL2-40Bmasking=with2024.12 | 86.6 | |
| InternVL2.5-26Bmasking=with2024.12 | 86.4 | |
| LLaVA-OneVision-72Bmasking=with2024.12 | 85.6 | |
| NVLM-D-72Bmasking=with2024.12 | 85.2 | |
| Qwen2-VL-2Bmasking=without2024.12 | 84.6 | |
| GPT-4o-20240513masking=with2024.12 | 84.6 | |
| InternVL2.5-8Bmasking=with2024.12 | 84.5 | |
| InternVL2-26Bmasking=with2024.12 | 84.5 | |
| Ovis1.6-Gemma2-9Bmasking=with2024.12 | 84.4 | |
| InternVL2-8Bmasking=with2024.12 | 83.8 | |
| InternVL2.5-2Bmasking=without2024.12 | 83.5 | |
| Qwen2-VL-7Bmasking=with2024.12 | 83 | |
| InternVL2-2Bmasking=without2024.12 | 82.3 | |
| MiniCPM-V2.6masking=with2024.12 | 82.1 | |
| InternVL2.5-4Bmasking=with2024.12 | 81.4 | |
| Claude-3.5-Sonnetmasking=with2024.12 | 81.2 | |
| InternVL-Chat-V1.5masking=with2024.12 | 80.7 | |
| Cambrian-34Bmasking=with2024.12 | 79.5 | |
| Gemini-1.5-Promasking=with2024.12 | 79.1 | |
| InternVL2-4Bmasking=with2024.12 | 78.9 | |
| GPT-4Vmasking=with2024.12 | 78.2 | |
| InternVL2.5-1Bmasking=without2024.12 | 77.8 | |
| Phi-3.5-Vision-4Bmasking=with2024.12 | 77.8 | |
| Aquila-VL-2Bmasking=with2024.12 | 75 | |
| InternVL2.5-2Bmasking=with2024.12 | 74.9 | |
| Qwen2-VL-2Bmasking=with2024.12 | 74.7 | |
| InternVL2-2Bmasking=with2024.12 | 74.1 | |
| Claude-3-Opusmasking=with2024.12 | 70.6 | |
| InternVL2-1Bmasking=without2024.12 | 70.5 | |
| VILA-1.5-40Bmasking=with2024.12 | 69.9 | |
| InternVL2.5-1Bmasking=with2024.12 | 69.3 | |
| InternVL2-1Bmasking=with2024.12 | 64.1 | |
| LLaVA-OneVision-0.5Bmasking=with2024.12 | 57.1 |