Multimodal Understanding on MMMU (val)
85.2MMMU ScoreGemini 3 Pro
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 3 ProModel Category=API call only2026.01 | 85.2 | — | — | — | — | — | — | — | — | — | — | |
| GPT-5Model Category=API call only2026.01 | 81.8 | — | — | — | — | — | — | — | — | — | — | |
| Gemini 2.5 ProModel Category=API call only2026.01 | 79.6 | — | — | — | — | — | — | — | — | — | — | |
| Gemini 2.5 FlashModel Category=API call only2026.01 | 79 | — | — | — | — | — | — | — | — | — | — | |
| GPT-5 miniModel Category=API call only2026.01 | 78.7 | — | — | — | — | — | — | — | — | — | — | |
| Claude Sonnet 4.5Model Category=API call only2026.01 | 77.8 | — | — | — | — | — | — | — | — | — | — | |
| Claude Sonnet 4Release=20252026.05 | 74.4 | — | — | — | — | — | — | — | — | — | — | |
| InternVL3.5-8BModel Category=Open weights only2026.01 | 73.4 | — | — | — | — | — | — | — | — | — | — | |
| Keye-VL-1.5-8BModel Category=Open weights only2026.01 | 71.4 | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-8BModel Category=Open weights only2026.01 | 69.6 | — | — | — | — | — | — | — | — | — | — | |
| GLM-4.1V-9BModel Category=Open weights only2026.01 | 68 | — | — | — | — | — | — | — | — | — | — | |
| MiniCPM-V-4.5-8BModel Category=Open weights only2026.01 | 67.7 | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-4BModel Category=Open weights only2026.01 | 67.4 | — | — | — | — | — | — | — | — | — | — | |
| InternVL3.5-4BModel Category=Open weights only2026.01 | 66.6 | — | — | — | — | — | — | — | — | — | — | |
| FastMMoE†Vision Token Retention Ratio=75%, lv=6, Kv=5, Saving FLOPs. (%)=37.592025.11 | 61.11 | — | — | — | — | — | — | — | — | — | 1.76 | |
| FastMMoE*Vision Token Retention Ratio=50%, Saving FLOPs. (%)=44.232025.11 | 61 | — | — | — | — | — | — | — | — | — | 2.8 | |
| FastMMoE†Vision Token Retention Ratio=75%, lv=3, Kv=4, Saving FLOPs. (%)=44.652025.11 | 60.78 | — | — | — | — | — | — | — | — | — | 2.31 | |
| FastMMoE*Vision Token Retention Ratio=25%, Saving FLOPs. (%)=64.302025.11 | 60.78 | — | — | — | — | — | — | — | — | — | 5.63 | |
| BaselineVision Token Retention Ratio=100%, Saving FLOPs. (%)=02025.11 | 60.67 | — | — | — | — | — | — | — | — | — | 0 | |
| FastMMoE*Vision Token Retention Ratio=75%, Saving FLOPs. (%)=21.992025.11 | 60.67 | — | — | — | — | — | — | — | — | — | 1.6 | |
| FastVVision Token Retention Ratio=75%, Saving FLOPs. (%)=21.992025.11 | 60.22 | — | — | — | — | — | — | — | — | — | 2.22 | |
| FastMMoE†Vision Token Retention Ratio=50%, lv=6, Kv=5, Saving FLOPs. (%)=55.022025.11 | 60.11 | — | — | — | — | — | — | — | — | — | 3.53 | |
| FastVVision Token Retention Ratio=50%, Saving FLOPs. (%)=44.232025.11 | 60 | — | — | — | — | — | — | — | — | — | 4.82 | |
| Claude 3 Opus (Opus 3)Release=20232026.05 | 59.4 | — | — | — | — | — | — | — | — | — | — | |
| FastMMoE†Vision Token Retention Ratio=50%, lv=3, Kv=3, Saving FLOPs. (%)=64.552025.11 | 59.33 | — | — | — | — | — | — | — | — | — | 4.47 | |
| FastMMoE†Vision Token Retention Ratio=25%, lv=6, Kv=5, Saving FLOPs. (%)=70.472025.11 | 59 | — | — | — | — | — | — | — | — | — | 7.27 | |
| SparseVLMVision Token Retention Ratio=75%, Saving FLOPs. (%)=21.992025.11 | 58.89 | — | — | — | — | — | — | — | — | — | 9.09 | |
| VanillaMode=Upper bound2024.12 | 58.6 | — | — | — | — | — | — | — | — | — | — | |
| Gemini 1.5 ProZero-shot=true2024.05 | 58.5 | — | — | — | — | — | — | — | — | — | — | |
| SparseVLMImage token reduction ratio=66.7%2024.12 | 58.4 | — | — | — | — | — | — | — | — | — | — | |
| VisionZipImage token reduction ratio=66.7%2024.12 | 58.3 | — | — | — | — | — | — | — | — | — | — | |
| iLLaVAImage token reduction ratio=66.7%2024.12 | 58.2 | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-VL2026.01 | 58 | — | — | — | — | — | — | — | — | — | — | |
| InternVL3.5-8BParameter Scale=8B2026.05 | 58 | — | — | — | — | — | — | — | — | — | — | |
| FasterVLMImage token reduction ratio=66.7%2024.12 | 57.9 | — | — | — | — | — | — | — | — | — | — | |
| PyramidDropImage token reduction ratio=66.7%2024.12 | 57.9 | — | — | — | — | — | — | — | — | — | — | |
| iLLaVAImage token reduction ratio=77.8%2024.12 | 57.9 | — | — | — | — | — | — | — | — | — | — | |
| FastVVision Token Retention Ratio=25%, Saving FLOPs. (%)=64.302025.11 | 57.89 | — | — | — | — | — | — | — | — | — | 9.95 | |
| InternVL3.5-4BParameter Scale=4B2026.05 | 57.2 | — | — | — | — | — | — | — | — | — | — | |
| SparseVLMVision Token Retention Ratio=50%, Saving FLOPs. (%)=44.232025.11 | 57.11 | — | — | — | — | — | — | — | — | — | 16.6 | |
| iLLaVAImage token reduction ratio=88.9%2024.12 | 56.9 | — | — | — | — | — | — | — | — | — | — | |
| GPT-4V2024.07 | 56.8 | 65.3 | 64.3 | 48.4 | 63.5 | 76.3 | 41.7 | — | — | — | — | |
| FasterVLMImage token reduction ratio=77.8%2024.12 | 56.8 | — | — | — | — | — | — | — | — | — | — | |
| SparseVLMImage token reduction ratio=77.8%2024.12 | 56.7 | — | — | — | — | — | — | — | — | — | — | |
| VisionZipImage token reduction ratio=77.8%2024.12 | 56.7 | — | — | — | — | — | — | — | — | — | — | |
| VisionZipImage token reduction ratio=88.9%2024.12 | 56 | — | — | — | — | — | — | — | — | — | — | |
| FasterVLMImage token reduction ratio=88.9%2024.12 | 55.9 | — | — | — | — | — | — | — | — | — | — | |
| Eagle2.5-8BModel Category=Open weights only2026.01 | 55.8 | — | — | — | — | — | — | — | — | — | — | |
| SparseVLMImage token reduction ratio=88.9%2024.12 | 55.6 | — | — | — | — | — | — | — | — | — | — | |
| PyramidDropImage token reduction ratio=77.8%2024.12 | 55.3 | — | — | — | — | — | — | — | — | — | — | |
| Bagel2026.04 | 55.3 | — | — | — | — | — | — | — | — | — | — | |
| BAGELType=Native Unified, # Params.=14B2026.05 | 55.3 | — | — | — | — | — | — | — | — | — | — | |
| BAGEL# Params.=14B2026.06 | 55.3 | — | — | — | — | — | — | — | — | — | — | |
| SenseChat-V2024.07 | 54.6 | 62.7 | 44.1 | 42.3 | 55.7 | 74.7 | 43.5 | — | — | — | — | |
| LatentUMBasequantized setting=false2026.04 | 54.6 | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-8BParameter Scale=8B2026.05 | 54.3 | — | — | — | — | — | — | — | — | — | — | |
| Molmo-72BModel Category=Molmo1 family2026.01 | 54.1 | — | — | — | — | — | — | — | — | — | — | |
| SparseVLMVision Token Retention Ratio=25%, Saving FLOPs. (%)=64.302025.11 | 53.89 | — | — | — | — | — | — | — | — | — | 24.43 | |
| GPT-4V-1106Size=> 8B2024.02 | 53.8 | — | — | — | — | — | — | — | — | — | — | |
| Claude 3 SonnetZero-shot=true2024.05 | 53.1 | — | — | — | — | — | — | — | — | — | — | |
| VILA2-40BModel Scale=40B2024.07 | 53 | 62 | 42.3 | 38.5 | 51.9 | 71.9 | 42.3 | — | — | — | — | |
| Molmo2-8BModel Category=Molmo2 family2026.01 | 53 | — | — | — | — | — | — | — | — | — | — | |
| VLM-UniDDT# Params.=4B+1B2026.06 | 52.6 | — | — | — | — | — | — | — | — | — | — | |
| PyramidDropImage token reduction ratio=88.9%2024.12 | 52.3 | — | — | — | — | — | — | — | — | — | — | |
| LatentUMBasequantized setting=true2026.04 | 52.3 | — | — | — | — | — | — | — | — | — | — | |
| Circle-RoPEParameters=3B, Supervised Fine-Tuning (SFT)=true2025.05 | 52.11 | — | — | — | — | — | — | — | — | — | — | |
| InternVL-Chat-V1.2Version=V1.22024.07 | 51.6 | 62.5 | 37.6 | 37.9 | 49.7 | 70.1 | 40.8 | — | — | — | — | |
| InternVL2.5Parameters=4B2025.05 | 51.56 | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-VL (SFT)Parameters=3B, Supervised Fine-Tuning (SFT)=true2025.05 | 51.56 | — | — | — | — | — | — | — | — | — | — | |
| Qwen-VL-MAX2024.07 | 51.4 | 64.2 | 39.8 | 36.3 | 52.5 | 70.4 | 40.7 | — | — | — | — | |
| Qwen3-VL-4BParameter Scale=4B2026.05 | 51.4 | — | — | — | — | — | — | — | — | — | — | |
| LLaVA-NeXTSize=34B, # tokens per image=2880, Zero-shot=true2024.05 | 51.1 | — | — | — | — | — | — | — | — | — | — | |
| Molmo2-4BModel Category=Molmo2 family2026.01 | 50.9 | — | — | — | — | — | — | — | — | — | — | |
| BLIP3o 8B2026.04 | 50.6 | — | — | — | — | — | — | — | — | — | — | |
| Qwen-2.5-VL-InstructType=Und. Only, # Params.=7B2026.05 | 50.6 | — | — | — | — | — | — | — | — | — | — | |
| BLIP3-oType=Composite Unified, # Params.=8B2026.05 | 50.6 | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-VLParameters=3B2025.05 | 50.22 | — | — | — | — | — | — | — | — | — | — | |
| Claude 3 HaikuZero-shot=true2024.05 | 50.2 | — | — | — | — | — | — | — | — | — | — | |
| InternVL3.5-2BParameter Scale=2B2026.05 | 49.9 | — | — | — | — | — | — | — | — | — | — | |
| TUNAType=Native Unified, # Params.=7B2026.05 | 49.8 | — | — | — | — | — | — | — | — | — | — | |
| Cambrian-1-34BModel Scale=34B2024.07 | 49.7 | — | — | — | — | — | — | — | — | — | — | |
| Molmo2026.01 | 49.1 | — | — | — | — | — | — | — | — | — | — | |
| Show-o22026.04 | 48.9 | — | — | — | — | — | — | — | — | — | — | |
| Show-o2Type=Native Unified, # Params.=7B2026.05 | 48.9 | — | — | — | — | — | — | — | — | — | — | |
| Show-o2# Params.=7B2026.06 | 48.9 | — | — | — | — | — | — | — | — | — | — | |
| Molmo2-4BParameter Scale=4B2026.05 | 48.8 | — | — | — | — | — | — | — | — | — | — | |
| LLaVA-OV# Params.=7B2026.06 | 48.8 | — | — | — | — | — | — | — | — | — | — | |
| LLaVA-1.62024.07 | 48.1 | 58.6 | 39.9 | 36 | 51.2 | 70.2 | 36.3 | — | — | — | — | |
| Mini-Gemini-HDSize=34B, # tokens per image=2880, Zero-shot=true2024.05 | 48 | — | — | — | — | — | — | — | — | — | — | |
| Mini-Gemini-HD-34BModel Scale=34B2024.07 | 48 | — | — | — | — | — | — | — | — | — | — | |
| Gemini 1.0 ProZero-shot=true2024.05 | 47.9 | — | — | — | — | — | — | — | — | — | — | |
| LLaVA-OneVision2026.01 | 47.9 | — | — | — | — | — | — | — | — | — | — | |
| Molmo2-8BParameter Scale=8B2026.05 | 47.7 | — | — | — | — | — | — | — | — | — | — | |
| Gemma3-4b-it2026.01 | 47.3 | — | — | — | — | — | — | — | — | — | — | |
| BLIP3o 4B2026.04 | 46.6 | — | — | — | — | — | — | — | — | — | — | |
| Marco-VL-Plus*2024.07 | 46.2 | 57.4 | 34.7 | 38.5 | 48.7 | 72.2 | 36.7 | — | — | — | — | |
| PLM-8BModel Category=Open models2026.01 | 46.1 | — | — | — | — | — | — | — | — | — | — | |
| Yi-VL2024.07 | 45.9 | 56.1 | 33.3 | 32.9 | 45.9 | 66.5 | 36 | — | — | — | — | |
| Molmo2-O-7BModel Category=Molmo2 family2026.01 | 45.8 | — | — | — | — | — | — | — | — | — | — | |
| Molmo-7B-DModel Category=Molmo1 family2026.01 | 45.3 | — | — | — | — | — | — | — | — | — | — |