Short Video Question Answering on TempCompass
76MC AccuracyQwen2-VL-72B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2-VL-72BModel Group=Open-source (>10B), Parameters=72B2025.01 | 76 | 75.9 | 84.6 | 58.6 | |
| Tarsier2-7BParameters=7B2025.01 | 75.3 | 75.1 | 80.6 | 66.6 | |
| GPT-4oModel Group=Proprietary2025.01 | 71 | 73.7 | 80.8 | 70.8 | |
| LLaVA-Video-72BModel Group=Open-source (>10B), Parameters=72B2025.01 | 69.9 | 73 | 80.9 | 54.4 | |
| Tarsier-34BModel Group=Open-source (>10B), Parameters=34B2025.01 | 69.8 | 74 | 73 | 60.9 | |
| Qwen2-VL-7BModel Group=Open-source (<=10B), Parameters=7B2025.01 | 68.5 | 72.8 | 77.3 | 54.2 | |
| Previous SOTACitation=[106]2025.01 | 68.5 | 72.8 | 77.3 | 54.2 | |
| LLaVA-OV-72BModel Group=Open-source (>10B), Parameters=72B2025.01 | 67.6 | 72.6 | 78.2 | 52.6 | |
| LLaVA-OV-7BModel Group=Open-source (<=10B), Parameters=7B2025.01 | 64.8 | 69.7 | 73.8 | 49.9 | |
| Gemini-1.5-ProModel Group=Proprietary2025.01 | 63.9 | 70.3 | 77.5 | 57.9 | |
| Tarsier-7BModel Group=Open-source (<=10B), Parameters=7B2025.01 | 58.7 | 58 | 54.2 | 55.3 | |
| LLaVA-Video-7BModel Group=Open-source (<=10B), Parameters=7B2025.01 | 56.3 | 68.7 | 76.8 | 53 |