Video Understanding on Video-MMMU
87.6AccuracyGemini 3-Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini 3-Pro2026.02 | 87.6 | |
| GPT-5tier=High2026.02 | 84.6 | |
| Gemini 2.5-Pro2026.02 | 83.6 | |
| Gemini-2.5-ProSize=-2026.02 | 83.6 | |
| GPT-5-miniRelease Date=2025-08-072026.06 | 82.5 | |
| ERNIE 5.02026.02 | 81.11 | |
| Qwen3.5Model Parameters=35B-A3B2026.06 | 80.4 | |
| Qwen3-VLmode=Thinking2026.02 | 80 | |
| Keye-VL-2.0-30B-A3BModel Parameters=30B-A3B2026.06 | 80 | |
| Qwen3-VL ThinkingModel Parameters=235B-A22B, Thinking Mode=true2026.06 | 80 | |
| Qwen3-VL ThinkingModel Parameters=32B, Thinking Mode=true2026.06 | 79 | |
| Qwen3-VL ThinkingModel Parameters=30B-A3B, Thinking Mode=true2026.06 | 75 | |
| Gemini-1.5-ProSize=-2026.02 | 70.4 | |
| GPT-4oSize=-2026.02 | 61.2 | |
| Qwen2.5-VLSize=72B2026.02 | 60.2 | |
| Demo-ICLSize=7B2026.02 | 52.6 | |
| LLaVA-VideoSize=72B2026.02 | 49.7 | |
| Demo-ICL (SFT)Size=7B2026.02 | 48.8 | |
| LLaVA-OneVisionSize=72B2026.02 | 48.3 | |
| Qwen2.5-VLSize=7B2026.02 | 47.4 | |
| Qwen2.5-VL#S (Model Size)=7B, #F (Number of input frames)=64, Architecture=AR Baseline, reproduced=true2026.01 | 47.4 | |
| VideoLLaMA3Size=7B2026.02 | 47 | |
| VidLaDA#S (Model Size)=8B, #F (Number of input frames)=64, Architecture=DLM Baseline2026.01 | 46.6 | |
| Ola-Video (Base)Size=7B2026.02 | 46.2 | |
| LLaDA-V#S (Model Size)=8B, #F (Number of input frames)=32, Architecture=DLM Baseline, reproduced=true2026.01 | 43.3 | |
| LLaVA-Video#S (Model Size)=7B, #F (Number of input frames)=64, Architecture=AR Baseline, reproduced=true2026.01 | 37.1 | |
| VILA-1.5Size=40B2026.02 | 34 | |
| LLaVA-OneVisionSize=7B2026.02 | 33.9 | |
| LLaVA-OneVision#S (Model Size)=7B, #F (Number of input frames)=32, Architecture=AR Baseline, reproduced=true2026.01 | 33.9 |