Multimodal Reasoning on MMMU-Pro
55Std-10 ScoreClaude-3.5-Sonnet
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Claude-3.5-Sonnet2024.12 | 55 | 48 | 51.5 | |
| GPT-4o-202405132024.12 | 54 | 49.7 | 51.9 | |
| InternVL2.5-78Bparameters=78B2024.12 | 51.4 | 45.9 | 48.6 | |
| Gemini-1.5-Pro2024.12 | 49.4 | 44.4 | 46.9 | |
| Qwen2-VL-72Bparameters=72B2024.12 | 49.2 | 43.3 | 46.2 | |
| InternVL2.5-38Bparameters=38B2024.12 | 48 | 44 | 46 | |
| InternVL2-Llama3-76Bparameters=76B, backbone=Llama32024.12 | 41.9 | 38 | 40 | |
| InternVL2.5-26Bparameters=26B2024.12 | 41.6 | 32.6 | 37.1 | |
| InternVL2.5-8Bparameters=8B2024.12 | 38.2 | 30.4 | 34.3 | |
| LLaVA-OneVision-72Bparameters=72B2024.12 | 38 | 24 | 31 | |
| InternVL2.5-4Bparameters=4B2024.12 | 36.4 | 29 | 32.7 | |
| InternVL2-40Bparameters=40B2024.12 | 36.3 | 32.1 | 34.2 | |
| VILA-1.5-40Bparameters=40B2024.12 | 35.9 | 14.1 | 25 | |
| Qwen2-VL-7Bparameters=7B2024.12 | 34.1 | 27 | 30.5 | |
| InternVL2-26Bparameters=26B2024.12 | 32.8 | 27.1 | 30 | |
| InternVL2-8Bparameters=8B2024.12 | 32.5 | 25.4 | 29 | |
| MiniCPM-V2.62024.12 | 30.2 | 24.2 | 27.2 | |
| InternVL-Chat-V1.52024.12 | 29.5 | 19.9 | 24.7 | |
| InternVL2-4Bparameters=4B2024.12 | 28.2 | 21.3 | 24.7 | |
| InternVL2.5-2Bparameters=2B2024.12 | 27.3 | 20.1 | 23.7 | |
| Phi-3.5-Vision-4Bparameters=4B2024.12 | 26.3 | 13.1 | 19.7 | |
| Qwen2-VL-2Bparameters=2B2024.12 | 25.3 | 17.2 | 21.2 | |
| InternVL2.5-1Bparameters=1B2024.12 | 23.3 | 15.5 | 19.4 | |
| InternVL2-2Bparameters=2B2024.12 | 21.6 | 14.9 | 18.2 | |
| InternVL2-1Bparameters=1B2024.12 | 16 | 13.6 | 14.8 |