Multiple-Choice Video QA on VideoMME
75AccuracyGemini 1.5 Pro
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini 1.5 ProModel Scale=Pro2024.06 | 75 | 81.3 | |
| GPT4-O2024.06 | 71.9 | 77.2 | |
| Gemini 1.5 FlashModel Scale=Flash2024.06 | 70.3 | 75 | |
| LLaVA-OneVisionModel Scale=72B, #Frames=322024.06 | 66.3 | 69.6 | |
| VILA 1.5Model Scale=34B2024.06 | 62.3 | 64.1 | |
| VideoLLaMA2Model Scale=72B, #Frames=162024.06 | 61.4 | 63.1 | |
| LLaVA-NeXT-VideoModel Scale=32B, #Frames=322024.06 | 60.2 | 63 | |
| GPT4-V2024.06 | 59.9 | 63.3 | |
| VideoLLaMA2.1Model Scale=7B, #Frames=162024.06 | 54.9 | 56.4 | |
| VideoLLaMA2Model Scale=7B, #Frames=162024.06 | 47.9 | 50.3 | |
| VideoLLaMA2Model Scale=8x7B, #Frames=82024.06 | 47.9 | 49.7 | |
| VideoLLaMA2Model Scale=7B, #Frames=82024.06 | 45.1 | 46.6 | |
| Video-LLaVAModel Scale=7B, #Frames=82024.06 | 39.9 | 41.6 | |
| VideoChat2Model Scale=7B, #Frames=162024.06 | 33.7 | — | |
| LLAMA-VIDModel Scale=7B, #Frames=1 fps2024.06 | 25.9 | — |