Video Question Answering on VideoMME 2024 (test)
81.7Accuracy (Short, w/o Subtitles)Gemini-1.5-Pro
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Gemini-1.5-ProSize=-2025.12 | 81.7 | 74.3 | 67.4 | 75 | 84.5 | 81 | 77.4 | 81.3 | |
| GPT-4oSize=-2025.12 | 80 | 70.3 | 65.3 | 71.9 | 82.8 | 76.6 | 72.1 | 77.2 | |
| InternVL2Size=34B2025.12 | 72 | 59.1 | 52.6 | 61.2 | 72.8 | 61.3 | 53 | 62.4 | |
| MiniCPM-V 2.6Size=8B2025.12 | 71.3 | 59.4 | 51.8 | 60.9 | 73.5 | 61.1 | 56.3 | 63.7 | |
| Claude 3.5 SonnetSize=-2025.12 | 71 | 57.4 | 51.2 | 60 | 73.5 | 60.1 | 54.7 | 62.9 | |
| mPLUG-Owl3Size=7B2025.12 | 70 | 57.7 | 50.1 | 59.3 | 72.8 | 66.9 | 64.5 | 68.1 | |
| VideoLLaMA 2Size=72B2025.12 | 69.8 | 59.9 | 57.6 | 62.4 | 72 | 63 | 59 | 64.7 | |
| LongVILASize=7B2025.12 | 69 | 58.3 | 53 | 60.1 | 72.9 | 64.9 | 57.4 | 65.1 | |
| VideoDetectiveSize=7B2025.12 | 68.6 | 61.6 | 56 | 62.1 | 70.4 | 66.5 | 63.5 | 66.8 | |
| Video-XLSize=7B2025.12 | 64 | 53.2 | 49.2 | 55.5 | 67.4 | 60.7 | 54.9 | 61 | |
| LongLLavaSize=7B2025.12 | 61.9 | 51.4 | 45.4 | 52.9 | 66.2 | 54.7 | 50.3 | 57.1 | |
| LLaVA-Next-VideoSize=34B2025.12 | 61.7 | 50.1 | 44.3 | 52 | 65.1 | 52.2 | 47.2 | 54.9 | |
| LongVASize=7B2025.12 | 61.1 | 50.4 | 46.2 | 52.6 | 61.6 | 53.6 | 47.6 | 54.3 |