Video Question-Answering on VideoMME wo sub
88.6AccuracyGemini 3 Pro
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini 3 Pro2026.02 | 88.6 | — | — | — | — | |
| Gemini 2.5 Pro2026.02 | 87.8 | — | — | — | — | |
| GPT-52026.02 | 83.3 | — | — | — | — | |
| Gemini-1.5-Pro2025.01 | 75 | — | — | — | — | |
| Claude Sonnet 4.52026.02 | 74.2 | — | — | — | — | |
| GPT4-o2025.01 | 71.9 | — | — | — | — | |
| QwenVL2Size=72B2025.01 | 71.2 | — | — | — | — | |
| LLaVA-OneVisionSize=72B, #Tokens=1962025.01 | 66.2 | — | — | — | — | |
| InternVideo2.5 (InternVL2.5+LRC)Size=7B, #Tokens=162025.01 | 65.1 | — | — | — | — | |
| InternVL2.5Size=7B, #Tokens=2562025.01 | 64.2 | — | — | — | — | |
| VideoChat-FlashSize=7B, #Tokens=162025.01 | 64 | — | — | — | — | |
| QwenVL2Size=7B2025.01 | 63.3 | — | — | — | — | |
| LLaVA-Video-7BParameters=7B2026.02 | 63.3 | — | — | — | — | |
| VideoLLaMA2Size=72B, #Tokens=722025.01 | 62.4 | — | — | — | — | |
| CoPE-VideoLM-7BParameters=7B2026.02 | 61.7 | — | — | — | — | |
| Apollo-7BParameters=7B2026.02 | 61.3 | — | — | — | — | |
| InternVL2Size=76B, #Tokens=2562025.01 | 61.2 | — | — | — | — | |
| VILA-40BParameters=40B2026.02 | 60.1 | — | — | — | — | |
| GPT4-V2025.01 | 58.8 | — | — | — | — | |
| Oryx-7BParameters=7B2026.02 | 58.3 | — | — | — | — | |
| LLaVA-OneVisionSize=7B, #Tokens=1962025.01 | 58.2 | — | — | — | — | |
| LLaVAOneVision-7BTraining Mode=SFT, Video Data=true, Model Size=7B2024.09 | 58.2 | — | — | — | — | |
| LLaVA-OV-7BParameters=7B2026.02 | 58.2 | — | — | — | — | |
| LongVILASize=7B, #Tokens=1962025.01 | 57.5 | — | — | — | — | |
| IXC-2.5-7BParameters=7B2026.02 | 55.8 | — | — | — | — | |
| InternVL2Size=8B, #Tokens=2562025.01 | 54 | — | — | — | — | |
| MM1.5-Video-7BTraining Mode=SFT, Video Data=true, Model Size=7B2024.09 | 53.5 | — | — | — | — | |
| EMA-7BParameters=7B2026.02 | 53.4 | — | — | — | — | |
| LongVASize=7B, #Tokens=1442025.01 | 52.6 | — | — | — | — | |
| LongVA-7BParameters=7B2026.02 | 52.6 | — | — | — | — | |
| MM1.5-Video-7BTraining Mode=Training-free, Video Data=false, Model Size=7B2024.09 | 52.4 | — | — | — | — | |
| MM1.5-Video-3BTraining Mode=SFT, Video Data=true, Model Size=3B2024.09 | 49.5 | — | — | — | — | |
| InternVideo2-HDSize=7B, #Tokens=722025.01 | 49.4 | — | — | — | — | |
| MM1.5-Video-3BTraining Mode=Training-free, Video Data=false, Model Size=3B2024.09 | 48.4 | — | — | — | — | |
| VideoLLaMA2Size=7B, #Tokens=722025.01 | 47.9 | — | — | — | — | |
| Video-LLaMA2-7BTraining Mode=SFT, Video Data=true, Model Size=7B2024.09 | 47.9 | — | — | — | — | |
| LLaVA-NeXT-VideoSize=7B, #Tokens=1442025.01 | 46.5 | — | — | — | — | |
| VideoChat-TLLM Size=7B2024.10 | 46.3 | — | — | — | — | |
| MM1.5-Video-1BTraining Mode=SFT, Video Data=true, Model Size=1B2024.09 | 45.7 | — | — | — | — | |
| MM1.5-Video-1BTraining Mode=Training-free, Video Data=false, Model Size=1B2024.09 | 45.6 | — | — | — | — | |
| VideoChat2-HDSize=7B, #Tokens=722025.01 | 45.3 | — | — | — | — | |
| LLaVAOneVision-0.5BTraining Mode=SFT, Video Data=true, Model Size=0.5B2024.09 | 44 | — | — | — | — | |
| LongLLaVASize=9B, #Tokens=1442025.01 | 43.7 | — | — | — | — | |
| mPLUG-Owl3Size=7B2025.01 | 43.5 | — | — | — | — | |
| SlowFast-LLaVA-7BTraining Mode=Training-free, Video Data=false, Model Size=7B2024.09 | 40.7 | — | — | — | — | |
| Chat-UniViLLM Size=7B2024.10 | 40.6 | — | — | — | — | |
| Video-LLaVA-7BTraining Mode=SFT, Video Data=true, Model Size=7B2024.09 | 39.9 | — | — | — | — | |
| VideoChat2LLM Size=7B2024.10 | 39.5 | — | — | — | — | |
| MovieChatLLM Size=7B2024.10 | 38.2 | — | — | — | — | |
| TimeChatLLM Size=7B2024.10 | 30.2 | — | — | — | — | |
| LLaMA-VIDSize=7B, #Tokens=22025.01 | 25.9 | — | — | — | — | |
| CASA⊕ Q2.5VLMax Res.=448^22025.12 | — | 58.6 | — | — | — | |
| Chat-Univi-1.5LLM Size=7B, #Frames=64, #Tokens=4482025.03 | — | 40.6 | 45.7 | 40.3 | 35.8 | |
| Gemini-1.5Zero-shot=true2025.02 | — | — | 81.7 | 74.3 | 67.4 | |
| GPT-4VBackbone=GPT-4, Zero-shot=true2025.02 | — | — | 70.5 | 55.8 | 53.5 | |
| HICom (Ours)LLM Size=1.5B, #Frames=32, #Tokens=6802025.03 | — | 52.3 | 61.7 | 51.3 | 43.9 | |
| HICom (Ours)LLM Size=7B, #Frames=32, #Tokens=6802025.03 | — | 58.1 | 65.7 | 57.1 | 51.4 | |
| HICom (Ours)LLM Size=7B, #Frames=64, #Tokens=1328, inference_frames_sampling_32=true2025.03 | — | 58.9 | 67.6 | 58 | 51.2 | |
| HICom (Ours)LLM Size=7B, #Frames=128, #Tokens=2624, inference_frames_sampling_32=true2025.03 | — | 60.3 | 69 | 60.2 | 51.5 | |
| LLaMA-VIDLLM Size=7B, #Frames=1fps, #Tokens=2tps2025.03 | — | 25.9 | — | — | — | |
| LLaVA-Next-VideoLLM Size=7B, #Frames=32, #Tokens=46082025.03 | — | 42.8 | 49.9 | 41.4 | 37 | |
| LLaVA-OneVisionLLM Size=7B, #Frames=32, #Tokens=62722025.03 | — | 58.5 | 70.1 | 56.4 | 48.9 | |
| LLAVA-OVBackbone=Qwen2-7B, Zero-shot=true2025.02 | — | — | 68.1 | 54.9 | 47.8 | |
| LLaVA-VideoLLM Size=7B, #Frames=32, #Tokens=62722025.03 | — | 60.6 | 73.9 | 57.3 | 50.4 | |
| Long-Llava 9BBackbone=Long-Llava 9B, Zero-shot=true2025.02 | — | — | 52.4 | 42.2 | 36.4 | |
| LongVABackbone=Qwen2-7B, Zero-shot=true2025.02 | — | — | 61.1 | 50.4 | 46.2 | |
| LongVALLM Size=7B, #Frames=128, #Tokens=184322025.03 | — | 52.6 | 61.1 | 50.4 | 46.2 | |
| Magma-8BBackbone=LLaMA3-8B, Zero-shot=true2025.02 | — | — | 72.9 | 55.8 | 44.3 | |
| mPLUG-Owl3Number of Parameters=8B2024.08 | — | 53.5 | — | — | — | |
| mPlug-Owl3 8BMax Res.=384^2 x 62025.12 | — | 53.5 | — | — | — | |
| Qwen2.5VL 3BMax Res.=Native2025.12 | — | 61.5 | — | — | — | |
| ShareGPT4VideoNumber of Parameters=8B2024.08 | — | 39.9 | — | — | — | |
| ShareGPT4VideoBackbone=LLaMA3-8B, Zero-shot=true2025.02 | — | — | 48.3 | 36.3 | 35 | |
| Video-Chat2Backbone=Mistral 7B, Zero-shot=true2025.02 | — | — | 48.3 | 37 | 33.2 | |
| Video-Llama2Backbone=Llama2-7B, Zero-shot=true2025.02 | — | — | 55.9 | 45.4 | 42.1 | |
| Video-LLaMA2Number of Parameters=8B2024.08 | — | 47.9 | — | — | — | |
| Video-LlavaBackbone=Vicuna-7B, Zero-shot=true2025.02 | — | — | 45.3 | 38 | 36.2 | |
| Video-LLaVALLM Size=7B, #Frames=8, #Tokens=20482025.03 | — | 39.9 | 45.3 | 38 | 36.2 | |
| VideoChat2Number of Parameters=8B2024.08 | — | 43.8 | — | — | — | |
| VideoChat2-MistralLLM Size=7B, #Frames=16, #Tokens=15362025.03 | — | 39.5 | 48.3 | 36.3 | 35 | |
| VideoLLaMA2LLM Size=7B, #Frames=16, #Tokens=11522025.03 | — | 47.9 | 56 | 45.4 | 42.1 | |
| VITALLM Size=8x7B, #Frames=16, #Tokens=40962025.03 | — | 55.8 | 65.9 | 52.9 | 48.6 |