Long-context Video Understanding on VideoMME
81.3Accuracy (w subs)Gemini-1.5-Pro
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini-1.5-Procategory=Proprietary2024.10 | 81.3 | 75 | |
| Qwen2-VL-72Bactivated_params=72B, category=Open-source2024.10 | 77.8 | 71.2 | |
| GPT-4ocategory=Proprietary2024.10 | 77.2 | 71.9 | |
| Gemini-1.5-Flashcategory=Proprietary2024.10 | 75 | 60.3 | |
| ARIAactivated_params=3.9B (25.3B), category=Open-source2024.10 | 72.1 | 67.6 | |
| LLaVA-OneVision-72Bactivated_params=72B, category=Open-source2024.10 | 69.6 | 66.3 | |
| Qwen2-VL-7Bactivated_params=7B, category=Open-source2024.10 | 69 | 63.3 | |
| GPT-4o minicategory=Proprietary2024.10 | 68.9 | 64.8 | |
| MiniCPM-V-2.6activated_params=8B, category=Open-source2024.10 | 63.7 | 60.9 | |
| InternVL2-40Bactivated_params=40B, category=Open-source2024.10 | 62.4 | 61.2 | |
| InternVL-Chat-V1.5activated_params=26B, category=Open-source2024.10 | 52.4 | 50.7 | |
| Llama3.2-11Bactivated_params=11B, category=Open-source2024.10 | 49.5 | 46 | |
| Pixtral-12Bactivated_params=12B, category=Open-source2024.10 | 47.5 | 40.7 |