Video Understanding on VideoMME (test)
86.9Overall ScoreGemini 2.5-Pro
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemini 2.5-ProAccess Type=Proprietary2025.05 | 86.9 | — | — | — | |
| Gemini 2.5-FlashAccess Type=Proprietary2025.05 | 81.5 | — | — | — | |
| Qwen3-VL-235B-A22BFrames=7682026.03 | 79 | — | — | — | |
| Qwen3-VL 32BAccess Type=Open-source2025.05 | 77.3 | — | — | — | |
| SiLVRAccess Type=Open-source2025.05 | 74.2 | — | — | — | |
| Qwen2.5-VL-72BFrames=7682026.03 | 73.3 | — | — | — | |
| InternVL3.5-241B-A28BFrames=–2026.03 | 72.9 | — | — | — | |
| Qwen3-VL 8BAccess Type=Open-source2025.05 | 71.8 | — | — | — | |
| Qwen2.5-VL 7BAccess Type=Open-source2025.05 | 71.6 | — | — | — | |
| Qwen3-VL-8B-Thinking + FrameRepeatFrames=128+82026.03 | 67.9 | — | — | — | |
| Qwen3-VL-8B-Thinking + Random-KFrames=128+82026.03 | 67.3 | — | — | — | |
| Qwen3-VL-8B-ThinkingFrames=1282026.03 | 67 | — | — | — | |
| Qwen3-VL-8B-Thinking + CLIP-KFrames=128+82026.03 | 67 | — | — | — | |
| Qwen3-VL-8B-Thinking + Bottom-KFrames=128+82026.03 | 66.3 | — | — | — | |
| PSASparsity=0.65, Base Model=Qwen2.5-VL-7B, Training-free=true2025.12 | 65.4 | 74.8 | 67.3 | 54.2 | |
| Full AttentionSparsity=0.0, Base Model=Qwen2.5-VL-7B, Training-free=true2025.12 | 65.1 | 75.2 | 66.3 | 53.7 | |
| XAttentionSparsity=0.58, Base Model=Qwen2.5-VL-7B, Training-free=true2025.12 | 65.1 | 74.8 | 66.1 | 54.4 | |
| SpargeAttentionSparsity=0.37, Base Model=Qwen2.5-VL-7B, Training-free=true2025.12 | 65 | 74.9 | 66.3 | 53.9 | |
| Qwen2.5-VL-7B-Instruct + FrameRepeatFrames=128+82026.03 | 63.4 | — | — | — | |
| Qwen3-VL-8B-Thinking + FrameRepeatFrames=64+42026.03 | 63.2 | — | — | — | |
| Qwen3-VL-8B-Thinking + CLIP-KFrames=64+42026.03 | 62.6 | — | — | — | |
| Qwen2.5-VL-7B-Instruct + CLIP-KFrames=128+82026.03 | 62.4 | — | — | — | |
| Qwen3-VL-8B-ThinkingFrames=642026.03 | 62.4 | — | — | — | |
| Qwen3-VL-8B-Thinking + Random-KFrames=64+42026.03 | 62.3 | — | — | — | |
| Qwen3-VL-8B-Thinking + Bottom-KFrames=64+42026.03 | 62 | — | — | — | |
| Qwen2.5-VL-7B-Instruct + Random-KFrames=128+82026.03 | 61.5 | — | — | — | |
| Video-R1 7BAccess Type=Open-source2025.05 | 61.4 | — | — | — | |
| Qwen2.5-VL-7B-Instruct + Bottom-KFrames=128+82026.03 | 61.3 | — | — | — | |
| Qwen2.5-VL-7B-Instruct + FrameRepeatFrames=64+42026.03 | 61 | — | — | — | |
| Qwen2.5-VL-7B-InstructFrames=1282026.03 | 60.9 | — | — | — | |
| Qwen2.5-VL-7B-Instruct + Random-KFrames=64+42026.03 | 60.3 | — | — | — | |
| Qwen2.5-VL-7B-Instruct + CLIP-KFrames=64+42026.03 | 60.2 | — | — | — | |
| VIDEOP2RBase Model=Qwen2.5-VL-7B2025.11 | 60 | — | — | — | |
| VideoRFTBase Model=Qwen2.5-VL-7B2025.11 | 59.8 | — | — | — | |
| Qwen2.5-VL-7B-InstructFrames=642026.03 | 59.6 | — | — | — | |
| Qwen2.5-VL-7B-Instruct + Bottom-KFrames=64+42026.03 | 59.6 | — | — | — | |
| Video-R1Base Model=Qwen2.5-VL-7B2025.11 | 59.3 | — | — | — | |
| Time-R1Base Model=Qwen2.5-VL-7B2025.11 | 59.3 | — | — | — | |
| VersaVid-R1Base Model=Qwen2.5-VL-7B2025.11 | 58.8 | — | — | — | |
| LLaVA-OneVision-7BModel Scale=7B2025.11 | 58.2 | — | — | — | |
| VideoChat-R1Base Model=Qwen2.5-VL-7B2025.11 | 57.7 | — | — | — | |
| Qwen2.5-VL-7BModel Scale=7B2025.11 | 56.6 | — | — | — | |
| LongVA-7BModel Scale=7B2025.11 | 52.6 | — | — | — | |
| Video-UTR-7BModel Scale=7B2025.11 | 52.6 | — | — | — | |
| VideoLLaMA2-7BModel Scale=7B2025.11 | 47.9 | — | — | — |