Video Understanding on VideoMME w/o sub. Long
67.4Long AccuracyGemini-1.5-Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-1.5-ProCategory=Proprietary Models, Size=-2026.06 | 67.4 | |
| GPT-4oCategory=Proprietary Models, Size=-2026.06 | 65.3 | |
| OmniAgentCategory=Ours, Size=7B, incorporate audio signals=true2026.06 | 59.6 | |
| LongVUCategory=Open-Source Non-Thinking Models, Size=7B2026.06 | 59.5 | |
| LongVILA-R1Category=Open-Source Thinking Models, Size=7B2026.06 | 55.2 | |
| Open-o3 VideoCategory=Open-Source Thinking Models, Size=7B2026.06 | 54.9 | |
| Zoom-ZeroCategory=Open-Source Agentic Models, Size=7B2026.06 | 54.8 | |
| Qwen2.5-OmniCategory=Open-Source Non-Thinking Models, Size=7B, incorporate audio signals=true2026.06 | 54.8 | |
| VITALCategory=Open-Source Agentic Models, Size=7B2026.06 | 54 | |
| LongVILACategory=Open-Source Non-Thinking Models, Size=7B2026.06 | 53 | |
| VISTACategory=Open-Source Non-Thinking Models, Size=7B2026.06 | 47.4 | |
| KangarooCategory=Open-Source Non-Thinking Models, Size=8B2026.06 | 46.7 | |
| LLaVA-OneVisionCategory=Open-Source Non-Thinking Models, Size=7B2026.06 | 46.7 | |
| LongVACategory=Open-Source Non-Thinking Models, Size=7B2026.06 | 46.1 | |
| LongVTCategory=Open-Source Agentic Models, Size=7B2026.06 | 44.4 | |
| VideoChat-TCategory=Open-Source Non-Thinking Models, Size=7B2026.06 | 41.9 |