Video Question Answering on VideoMME (Length and Subtitle Breakdown)
77.4Accuracy (Overall, w/o Subtitles)Video-RAG
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Video-RAGSize=72B, # Frames=642026.04 | 77.4 | 82.8 | — | 76.3 | — | 73.1 | — | — | |
| Gemini 1.5 ProSize=-, # Frames=1fps2026.04 | 75 | 81.7 | 84.5 | 74.3 | 81 | 67.4 | 77.4 | 81.3 | |
| Qwen2.5-VLSize=72B, # Frames=32 / 768∗2026.04 | 74.1 | 79.4 | — | 71.2 | — | 71.8 | — | 79.1 | |
| InternVL 2.5Size=78B, # Frames=642026.04 | 72.1 | 82.8 | — | 70.9 | — | 62.6 | — | 74 | |
| GPT-4oSize=-, # Frames=1fps2026.04 | 71.9 | 80 | 82.8 | 70.3 | 76.6 | 65.3 | 72.1 | 77.2 | |
| LLaVA-VideoSize=72B, # Frames=32 / 64∗2026.04 | 70.3 | 80.7 | 81.8 | 68.7 | 73.8 | 62.1 | 72.2 | 75.9 | |
| VgentSize=7B, # Frames=3202026.04 | 68.9 | — | — | — | — | — | — | 74.3 | |
| SVAgentBackbone=Qwen3-VL, Size=8B, # Frames=82026.04 | 63.1 | 73.9 | 78 | 61.5 | 71.1 | 53.9 | 60.5 | 69.8 | |
| SVAgentBackbone=Qwen2.5-VL, Size=7B, # Frames=82026.04 | 61.2 | 72.1 | 75.8 | 58 | 67.5 | 53.3 | 57.3 | 66.9 | |
| SVAgentBackbone=Qwen2.5-VL, Size=3B, # Frames=82026.04 | 60.7 | 71 | 72.4 | 59.4 | 66.3 | 51.6 | 58.3 | 65.7 | |
| SVAgentBackbone=Qwen3-VL, Size=4B, # Frames=82026.04 | 60.1 | 69.7 | 74.5 | 57.1 | 66.6 | 53.5 | 57.5 | 66.2 | |
| VideoMindSize=7B, # Frames=322026.04 | 58.2 | — | — | — | — | 49.2 | — | — | |
| Qwen3-VLSize=8B, # Frames=82026.04 | 55.8 | 66.7 | 71.7 | 53.4 | 65.2 | 47.4 | 54.6 | 63.8 | |
| Qwen3-VLSize=4B, # Frames=82026.04 | 53.9 | 64.5 | 69.6 | 50.6 | 61.4 | 46.6 | 52.2 | 61.1 | |
| Qwen2.5-VLSize=3B, # Frames=82026.04 | 52.8 | 63.7 | 67.3 | 51.6 | 62 | 43.1 | 50.6 | 60 | |
| Qwen2.5-VLSize=7B, # Frames=82026.04 | 52.8 | 63.9 | 69 | 50.2 | 62 | 44.4 | 52.3 | 61.1 |