Long Video Understanding on Video-MME (w/o sub.) Overall 1010s
75AccuracyGemini 1.5 Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini 1.5 ProSize=-, Tokens per frame=-2026.04 | 75 | |
| GPT-4oSize=-, Tokens per frame=-2026.04 | 71.9 | |
| Qwen3-VL*Size=8B, Tokens per frame=≤ 6402026.04 | 71.4 | |
| Molmo2Size=8B, Tokens per frame=832026.04 | 69.9 | |
| Tempo*Size=6B, Tokens per frame=0.5–16, Visual Budget=4K, Actual tokens/frame=3.62026.04 | 67.8 | |
| Tempo*Size=6B, Tokens per frame=0.5–16, Visual Budget=8K, Actual tokens/frame=4.32026.04 | 67.7 | |
| VideoLLaMA3*Size=7B, Tokens per frame=≤ 912026.04 | 66.2 | |
| InternVL3.5Size=8B, Tokens per frame=2562026.04 | 66 | |
| VideoChat-FlashSize=7B, Tokens per frame=162026.04 | 65.3 | |
| Qwen2.5-VLSize=7B, Tokens per frame=19242026.04 | 65.1 | |
| BIMBASize=7B, Tokens per frame=362026.04 | 64.7 | |
| StormSize=7B, Tokens per frame=642026.04 | 63.4 | |
| LLaVA-VideoSize=7B, Tokens per frame=6762026.04 | 63.3 | |
| Qwen3-VL*Size=2B, Tokens per frame=≤ 6402026.04 | 61.9 | |
| LongVUSize=7B, Tokens per frame=642026.04 | 60.6 | |
| LongVILASize=7B, Tokens per frame=1962026.04 | 60.1 | |
| LLaVA-OneVisionSize=7B, Tokens per frame=1962026.04 | 58.2 | |
| KangarooSize=8B, Tokens per frame=2562026.04 | 56 | |
| LongLLaVASize=A13B, Tokens per frame=1442026.04 | 53.8 | |
| LongVASize=7B, Tokens per frame=1442026.04 | 52.6 | |
| VideoChat2-HDSize=7B, Tokens per frame=722026.04 | 45.3 | |
| LLaMA-VIDSize=7B, Tokens per frame=22026.04 | 25.9 |