Video Understanding on Video-MME (test)
88.6AccuracyGemini 3 Pro
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 3 ProAccess Type=API call only2026.01 | 88.6 | — | — | |
| Gemini 2.5 ProAccess Type=API call only2026.01 | 87.8 | — | — | |
| Gemini 2.5 FlashAccess Type=API call only2026.01 | 84.2 | — | — | |
| GPT-5Access Type=API call only2026.01 | 83.3 | — | — | |
| VideoChat-M1Frames=69.9, Inference Time=19.8s2025.11 | 83.2 | — | — | |
| GPT-5 miniAccess Type=API call only2026.01 | 77.3 | — | — | |
| Gemini-1.5-ProFrames=568, Inference Time=227.2s2025.11 | 75 | — | — | |
| Gemini 1.5 ProBackbone Model=Gemini 1.5 Pro, Method/Configuration=Base, Model Access Level=Closed-source2026.04 | 75 | — | — | |
| Claude Sonnet 4.5Access Type=API call only2026.01 | 74.2 | — | — | |
| Keye-VL-1.5-8BAccess Type=Open weights only2026.01 | 73 | — | — | |
| Eagle2.5-8BAccess Type=Open weights only2026.01 | 72.4 | — | — | |
| GPT-4oFrames=384, Inference Time=153.6s2025.11 | 71.9 | — | — | |
| GPT-4oBackbone Model=GPT-4o, Method/Configuration=Base, Model Access Level=Closed-source2026.04 | 71.9 | — | — | |
| Qwen3-VL-8BAccess Type=Open weights only2026.01 | 71.4 | — | — | |
| Qwen2-VL-72BFrames=568, Inference Time=90.5s2025.11 | 71.2 | — | — | |
| Molmo2-8BAccess Type=Molmo2 family2026.01 | 69.9 | — | — | |
| Molmo2-4BAccess Type=Molmo2 family2026.01 | 69.6 | — | — | |
| Qwen3-VL-4BAccess Type=Open weights only2026.01 | 69.3 | — | — | |
| GLM-4.1V-9BAccess Type=Open weights only2026.01 | 68.2 | — | — | |
| MiMo-VL-Miloco-7B2025.12 | 68 | — | — | |
| MiniCPM-V-4.5-8BAccess Type=Open weights only2026.01 | 67.9 | — | — | |
| MiMo-VL-7B-SFT-25082025.12 | 67.8 | — | — | |
| InternVL3-8B2025.12 | 66.3 | — | — | |
| InternVL3.5-8BAccess Type=Open weights only2026.01 | 66 | — | — | |
| InternVL3.5-4BAccess Type=Open weights only2026.01 | 65.4 | — | — | |
| VideoChat-Flash-7BAccess Type=Open models2026.01 | 65.3 | — | — | |
| Qwen2.5-VL-7B2025.12 | 65.1 | — | — | |
| Molmo2-O-7BAccess Type=Molmo2 family2026.01 | 64.9 | — | — | |
| LLaVA-Video-7BAccess Type=Open models2026.01 | 63.3 | — | — | |
| SeasonBackbone Model=Qwen3-VL-32B, Method/Configuration=+Season, Model Access Level=Open-source2026.04 | 60.9 | — | — | |
| Structured Reward Reinforcement LearningBackbone Model=Qwen3-VL-32B, Method/Configuration=+Ours, Model Access Level=Open-source2026.04 | 60.3 | — | — | |
| Qwen3-VL-32BBackbone Model=Qwen3-VL-32B, Method/Configuration=Base, Model Access Level=Open-source2026.04 | 60.1 | — | — | |
| PLM-8BAccess Type=Open models2026.01 | 58.3 | — | — | |
| Structured Reward Reinforcement LearningBackbone Model=Qwen3-VL-8B, Method/Configuration=+Ours, Model Access Level=Open-source2026.04 | 57.4 | — | — | |
| Qwen3-VL-8BBackbone Model=Qwen3-VL-8B, Method/Configuration=Base, Model Access Level=Open-source2026.04 | 56.7 | — | — | |
| SeasonBackbone Model=Qwen3-VL-8B, Method/Configuration=+Season, Model Access Level=Open-source2026.04 | 56.6 | — | — | |
| PLM-3BAccess Type=Open models2026.01 | 54.9 | — | — | |
| Structured Reward Reinforcement LearningBackbone Model=Qwen3-VL-4B, Method/Configuration=+Ours, Model Access Level=Open-source2026.04 | 54.4 | — | — | |
| SeasonBackbone Model=Qwen3-VL-4B, Method/Configuration=+Season, Model Access Level=Open-source2026.04 | 53.8 | — | — | |
| Qwen3-VL-4BBackbone Model=Qwen3-VL-4B, Method/Configuration=Base, Model Access Level=Open-source2026.04 | 53.6 | — | — | |
| Structured Reward Reinforcement LearningBackbone Model=LLaVA-OV-7B, Method/Configuration=+Ours, Model Access Level=Open-source2026.04 | 51.2 | — | — | |
| Structured Reward Reinforcement LearningBackbone Model=Qwen2.5-VL-7B, Method/Configuration=+Ours, Model Access Level=Open-source2026.04 | 50.9 | — | — | |
| SeasonBackbone Model=LLaVA-OV-7B, Method/Configuration=+Season, Model Access Level=Open-source2026.04 | 50.7 | — | — | |
| LLaVA-OV-7BBackbone Model=LLaVA-OV-7B, Method/Configuration=Base, Model Access Level=Open-source2026.04 | 50.6 | — | — | |
| DINO-HEALBackbone Model=LLaVA-OV-7B, Method/Configuration=+DINO-HEAL, Model Access Level=Open-source2026.04 | 50.6 | — | — | |
| TCDBackbone Model=LLaVA-OV-7B, Method/Configuration=+TCD, Model Access Level=Open-source2026.04 | 50.4 | — | — | |
| SeasonBackbone Model=Qwen2.5-VL-7B, Method/Configuration=+Season, Model Access Level=Open-source2026.04 | 50.4 | — | — | |
| Qwen2.5-VL-7BBackbone Model=Qwen2.5-VL-7B, Method/Configuration=Base, Model Access Level=Open-source2026.04 | 49.9 | — | — | |
| DINO-HEALBackbone Model=Qwen2.5-VL-7B, Method/Configuration=+DINO-HEAL, Model Access Level=Open-source2026.04 | 49.9 | — | — | |
| ArrowRLBackbone Model=Qwen2.5-VL-7B, Method/Configuration=+ArrowRL, Model Access Level=Open-source2026.04 | 49.6 | — | — | |
| TCDBackbone Model=Qwen2.5-VL-7B, Method/Configuration=+TCD, Model Access Level=Open-source2026.04 | 49.4 | — | — | |
| Demo-ICLSize=7B2026.02 | — | 65.2 | 69.7 | |
| Demo-ICL (SFT)Size=7B2026.02 | — | 65.6 | 70.2 | |
| Gemini 1.5-Pro2024.09 | — | 75 | 81.3 | |
| Gemini-1.5-ProSize=not specified2024.09 | — | 75 | 81.3 | |
| Gemini-1.5-ProSize=-2026.02 | — | 73.2 | 79.8 | |
| Gemini-2.5-ProSize=-2026.02 | — | 84.3 | 86.9 | |
| GPT-4o2024.09 | — | 71.9 | 77.2 | |
| GPT-4oSize=not specified2024.09 | — | 71.9 | 77.2 | |
| GPT-4oSize=-2026.02 | — | 71.9 | 77.2 | |
| GPT-4VSize=not specified2024.09 | — | 59.9 | 63.3 | |
| GPT-4VSize=-2026.02 | — | 59.9 | 63.3 | |
| InternVL3.5Size=8B2026.02 | — | 66 | 68.6 | |
| KangarooSize=8B2024.09 | — | 56 | 57.6 | |
| LLaVA-Next-VideoSize=34B2024.09 | — | 52 | 54.9 | |
| LLaVA-OneVisionSize=7B2024.09 | — | 58.2 | 61.5 | |
| LLaVA-OneVisionSize=72B2024.09 | — | 66.2 | 69.5 | |
| LLaVA-OneVisionSize=7B2026.02 | — | 58.2 | 61.5 | |
| LLaVA-OneVisionSize=72B2026.02 | — | 66.2 | 69.5 | |
| LLaVA-VideoSize=7B2026.02 | — | 63.3 | 69.7 | |
| LLaVA-VideoSize=72B2026.02 | — | 70.5 | 76.9 | |
| Ola-Video (Base)Size=7B2026.02 | — | 63 | 68.9 | |
| OryxSize=7B2024.09 | — | 58.3 | 62.6 | |
| OryxSize=34B2024.09 | — | 63.2 | 67.4 | |
| Oryx-1.5Size=7B2024.09 | — | 58.8 | 64.2 | |
| Oryx-1.5Size=32B2024.09 | — | 67.3 | 74.9 | |
| Previous SoTA2024.09 | — | 66.3 | 69.6 | |
| Qwen2-VL-2BParameters=2B2024.09 | — | 55.6 | 60.4 | |
| Qwen2-VL-72BParameters=72B2024.09 | — | 71.2 | 77.8 | |
| Qwen2-VL-7BParameters=7B2024.09 | — | 63.3 | 69 | |
| Qwen2.5-VLSize=7B2026.02 | — | 65.1 | 71.6 | |
| Qwen2.5-VLSize=72B2026.02 | — | 73.3 | 79.1 | |
| VideoCAMSize=9B2024.09 | — | 53.9 | 56.1 | |
| VideoChat2-HDSize=7B2024.09 | — | 45.3 | 55.7 | |
| VideoLLaMA2Size=7B2024.09 | — | 47.9 | 50.3 | |
| VideoLLaMA2Size=72B2024.09 | — | 61.4 | 63.1 | |
| VideoLLaMA2Size=7B2026.02 | — | 47.9 | 50.3 | |
| VideoLLaMA2Size=72B2026.02 | — | 61.4 | 63.1 | |
| VideoLLaMA3Size=7B2026.02 | — | 66.2 | 70.3 | |
| VILA-1.5Size=40B2024.09 | — | 60.1 | 61.1 | |
| VILA-1.5Size=40B2026.02 | — | 60.1 | 61.1 |