Multi-choice Video Question Answering on VideoMME
75Accuracy (no subs)Gemini-1.5-Pro
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini-1.5-Prozero-shot=true2025.03 | 75 | 81.3 | — | |
| GPT-4ozero-shot=true2025.03 | 71.9 | 77.2 | — | |
| Gemini-1.5-Flashzero-shot=true2025.03 | 70.3 | 75 | — | |
| InternVL2.5-8B-ITG-32Frames Sampled=322025.07 | 67.3 | 69.6 | — | |
| Qwen2.5-VL-7B2025.07 | 65.1 | 71.6 | — | |
| InternVL2.5-8B2025.07 | 64.2 | 66.9 | — | |
| LLaVAction-7Bzero-shot=true2025.03 | 63.9 | 71.4 | — | |
| LLaVA-Video-7Bzero-shot=true2025.03 | 63.3 | 69.7 | — | |
| LLaVA-Video-7B2025.07 | 63.3 | 69.7 | — | |
| LongVU-7B2025.07 | 60.6 | — | — | |
| VILA-40B2025.07 | 60.1 | 61.1 | — | |
| GPT-4Vzero-shot=true2025.03 | 59.9 | 63.3 | — | |
| mPLUG-Owl3zero-shot=true2025.03 | 59.3 | 68.1 | — | |
| LLaVA-OV-7Bzero-shot=true2025.03 | 58.2 | 61.5 | — | |
| LLaVA-OV-7B2025.07 | 58.2 | 61.5 | — | |
| mPLUG-Owl3-8B2025.07 | 53.5 | — | — | |
| LongVA-7Bzero-shot=true2025.03 | 52.6 | 54.3 | — | |
| LongVA-7B2025.07 | 52.6 | 54.3 | — | |
| VideoLLaMA2-7Bzero-shot=true2025.03 | 47.9 | 50.3 | — | |
| VideoLLaMA2-7B2025.07 | 45.1 | 46.6 | — | |
| VideoChat2-7Bzero-shot=true2025.03 | 42.3 | 54.6 | — | |
| DYTOLLM Size=7B, Vis Encoder=CLIP-L, Training Regime=Training-Free2026.02 | — | — | 41.2 | |
| DYTOLLM Size=34B, Vis Encoder=CLIP-L, Training Regime=Training-Free2026.02 | — | — | 53.4 | |
| IG-VLMLLM Size=7B, Vis Encoder=CLIP-L, Training Regime=Training-Free2026.02 | — | — | 39.7 | |
| IG-VLMLLM Size=34B, Vis Encoder=CLIP-L, Training Regime=Training-Free2026.02 | — | — | 50.3 | |
| KTV-34B-denseLLM Size=34B, Vis Encoder=CLIP-L, Training Regime=Training-Free, Variant=dense2026.02 | — | — | 53.2 | |
| KTV-34B-normalLLM Size=34B, Vis Encoder=CLIP-L, Training Regime=Training-Free, Variant=normal2026.02 | — | — | 53 | |
| KTV-34B-sparseLLM Size=34B, Vis Encoder=CLIP-L, Training Regime=Training-Free, Variant=sparse2026.02 | — | — | 52.2 | |
| KTV-7B-denseLLM Size=7B, Vis Encoder=CLIP-L, Training Regime=Training-Free, Variant=dense2026.02 | — | — | 44 | |
| KTV-7B-normalLLM Size=7B, Vis Encoder=CLIP-L, Training Regime=Training-Free, Variant=normal2026.02 | — | — | 43.7 | |
| KTV-7B-sparseLLM Size=7B, Vis Encoder=CLIP-L, Training Regime=Training-Free, Variant=sparse2026.02 | — | — | 43.6 | |
| SF-LLaVA-7BLLM Size=7B, Vis Encoder=CLIP-L, Training Regime=Training-Free2026.02 | — | — | 39.4 | |
| SF-LLaVA-7BLLM Size=34B, Vis Encoder=CLIP-L, Training Regime=Training-Free2026.02 | — | — | 51.9 | |
| Video-LLaVALLM Size=7B, Vis Encoder=ViT-L, Training Regime=Training-Based2026.02 | — | — | 39.9 |