Video Multiple Choice Question Answering on ACaM real-world videos 1.0 (test)
99Accuracy (Static)Human Performance
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Human Performance2026.07 | 99 | 93.07 | 90.39 | 90.83 | 97.1 | 96.97 | 93.44 | 93.14 | |
| VLM-3R-7BModel Type=Spatial VLMs2026.07 | 94.5 | 20.69 | 28.55 | 35.83 | 51.43 | 89.39 | 40.22 | 35.02 | |
| Qwen3-VL-4BModel Type=General VLMs2026.07 | 93.5 | 53.47 | 34.21 | 61.67 | 56.52 | 80.3 | 53.01 | 46.96 | |
| LLaVa-OV-7BModel Type=General VLMs2026.07 | 92.5 | 28.71 | 25.79 | 24.17 | 62.32 | 75.76 | 39.55 | 34.85 | |
| Qwen3-VL-32BModel Type=General VLMs2026.07 | 92 | 64.11 | 47.93 | 64.17 | 57.97 | 86.36 | 61.95 | 58.66 | |
| ShotVL-7BModel Type=Camera Movement Specialized VLMs2026.07 | 91 | 60.4 | 51.07 | 32.5 | 68.12 | 98.48 | 60.52 | 57.09 | |
| Mega-SaMModel Type=Visual Geometry Models2026.07 | 89.55 | 74.94 | 54.16 | — | — | — | — | — | |
| InternVL3.5-8BModel Type=General VLMs2026.07 | 89 | 48.51 | 30.58 | 58.33 | 40.58 | 86.36 | 48.77 | 46.15 | |
| SFT Qwen3-VL-8BMode=SFT, Backbone=Qwen3-VL-8B2026.07 | 85.5 | 70.05 | 64.13 | 84.17 | 84.06 | 96.97 | 72.75 | 74.28 | |
| Qwen3-VL-8BModel Type=General VLMs2026.07 | 85 | 65.84 | 39.5 | 61.67 | 79.71 | 74.24 | 58.27 | 55.25 | |
| Gemini-3.1-ProModel Type=General VLMs2026.07 | 83.92 | 63.34 | 64 | 62.39 | 76.81 | 95.38 | 68.44 | 67.73 | |
| Gemini-3-ProModel Type=General VLMs2026.07 | 83.5 | 64.36 | 60.84 | 65 | 82.61 | 96.97 | 68.24 | 66.83 | |
| InternVideo2.5-8BModel Type=General VLMs2026.07 | 82.5 | 31.44 | 28.43 | 64.17 | 50.72 | 92.42 | 43.51 | 41.07 | |
| Qwen2.5-VL-7BModel Type=General VLMs2026.07 | 81 | 33.91 | 34.71 | 58.33 | 66.67 | 92.42 | 46.86 | 45.62 | |
| InternVL3.5-14BModel Type=General VLMs2026.07 | 81 | 43.56 | 39.83 | 63.33 | 55.07 | 89.39 | 51.37 | 47.21 | |
| CamReasoner-7BModel Type=Camera Movement Specialized VLMs2026.07 | 76.5 | 41.58 | 32.89 | 59.17 | 39.13 | 80.3 | 45.83 | 44.99 | |
| SFT Qwen3-VL-4BMode=SFT, Backbone=Qwen3-VL-4B2026.07 | 76 | 71.53 | 64.13 | 76.67 | 78.26 | 93.94 | 70.83 | 72.27 | |
| ViPEModel Type=Visual Geometry Models2026.07 | 61.81 | 49.71 | 73.17 | — | — | — | — | — | |
| GPT-5Model Type=General VLMs2026.07 | 61.5 | 66.58 | 54.55 | 61.67 | 68.12 | 80.3 | 61.2 | 61.54 | |
| Spatial-MLLMModel Type=Spatial VLMs2026.07 | 59 | 16.98 | 31.72 | 41.67 | 24.29 | 77.27 | 34.22 | 29.68 | |
| CameraModel-7BModel Type=Camera Movement Specialized VLMs2026.07 | 55.5 | 68.81 | 47.93 | 45.83 | 91.3 | 98.48 | 58.88 | 60.56 | |
| Random Guess2026.07 | 24.5 | 24.28 | 24 | 24.54 | 24.78 | 25 | 24.27 | 24.37 | |
| G2VLM-2BModel Type=Spatial VLMs2026.07 | 9.5 | 4.52 | 4.52 | 7.44 | 5.71 | 7.58 | 5.66 | 5.24 |