Video Understanding on MVBench zero-shot
69.7AccuracyVideoLLaMA3
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| VideoLLaMA3LLM=Qwen2.5-7B, Frames=1fps2025.07 | 69.7 | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-VLLLM=Qwen2.5-7B2025.07 | 69.6 | — | — | — | — | — | — | — | — | — | |
| InternVL3.5-REGATELLM=Qwen3-14B, Frames=16, Tokens=2.32B (↓ 41.41%)2025.07 | 69.6 | — | — | — | — | — | — | — | — | — | |
| InternVL3.5LLM=Qwen3-14B, Frames=16, Tokens=3.96B2025.07 | 68.3 | — | — | — | — | — | — | — | — | — | |
| Qwen-2.5-VLSize=3B, #V=31932025.08 | 65.2 | 68.3 | 67.7 | 48.8 | 91 | 63 | 75 | 51.5 | 75.5 | 56 | |
| GPT-4o2025.07 | 64.6 | — | — | — | — | — | — | — | — | — | |
| Fourier-Qwen-2.5Size=3B, #V=13282025.08 | 62.9 | 67.2 | 65.7 | 48.5 | 90.5 | 54.8 | 70.5 | 47.5 | 66 | 57.5 | |
| Qwen-2-VLSize=2B, #V=31932025.08 | 61.4 | 68.5 | 65.7 | 44.8 | 90.5 | 60.8 | 65.3 | 53.5 | 59 | 47.8 | |
| Gemini-1.5-Pro2025.07 | 60.5 | — | — | — | — | — | — | — | — | — | |
| Fourier-Qwen-2Size=2B, #V=13282025.08 | 59.8 | 67.7 | 63.3 | 40 | 88.5 | 55.5 | 64.8 | 51 | 57 | 50.2 | |
| LLaVA-OneVisionLLM=Qwen2-7B, Frames=322025.07 | 56.7 | — | — | — | — | — | — | — | — | — | |
| VideoChat2-REGATELLM=Mistral-7B, Frames=16, Tokens=2.22B (↓ 43.51%)2025.07 | 56.6 | — | — | — | — | — | — | — | — | — | |
| VideoChat2LLM=Mistral-7B, Frames=16, Tokens=3.93B2025.07 | 55.7 | — | — | — | — | — | — | — | — | — | |
| VideoLLaMA2-REGATELLM=Qwen2-7B, Frames=16, Tokens=49.27M (↓ 41.22%)2025.07 | 53.6 | — | — | — | — | — | — | — | — | — | |
| VideoLLaMA2LLM=Qwen2-7B, Frames=16, Tokens=83.82M2025.07 | 52 | — | — | — | — | — | — | — | — | — | |
| VideoChat2Size=7B, #V=40962025.08 | 51.1 | 61.3 | 57.3 | 23 | 88.5 | 40.5 | 51.3 | 49 | 36.5 | 47 | |
| LLaVA-NeXT-VideoLLM=Vicuna-7B, Frames=322025.07 | 46.5 | — | — | — | — | — | — | — | — | — | |
| LLaVA-v1.5Size=7B, #V=23042025.08 | 45.6 | 52.8 | 43.7 | 31.5 | 83.5 | 37 | 45.3 | 44.5 | 50 | 37.3 | |
| Fourier-LLaVASize=7B, #V=2882025.08 | 44 | 48.1 | 46.3 | 31 | 81 | 40.5 | 41.3 | 36 | 49 | 36.3 | |
| Video-LLaVASize=7B, #V=40962025.08 | 43.1 | 48 | 46.5 | 27.8 | 84.5 | 35.5 | 45.8 | 34 | 42.5 | 34.2 | |
| LLaMA-VIDLLM=Llama-2-7B, Frames=1fps2025.07 | 41.9 | — | — | — | — | — | — | — | — | — | |
| Video-LLaVALLM=Vicuna-7B, Frames=82025.07 | 41 | — | — | — | — | — | — | — | — | — | |
| VideoChatSize=7B, #V=40962025.08 | 35.5 | 38 | 41.2 | 26.3 | 48.5 | 27.8 | 44.3 | 26.5 | 41 | 27.7 | |
| Video-LLaMASize=7B, #V=402025.08 | 34.1 | 34.4 | 42.2 | 22.5 | 43 | 28.3 | 39 | 32.5 | 40 | 29.3 | |
| VideoChatGPTSize=7B, #V=3562025.08 | 32.7 | 32.1 | 40.7 | 21.5 | 31 | 28 | 44 | 29 | 33 | 30.3 |