Camera Movement Understanding on ACaM synthetic videos (test)
99.44Static AccuracyVLM-3R
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| VLM-3RCategory=Spatial VLMs, Parameters=7B2026.07 | 99.44 | 18.9 | 43.45 | 51.92 | 47.27 | 89.29 | 48.68 | 38.94 | |
| Human Performance2026.07 | 98.88 | 97.72 | 98.04 | 96.15 | 100 | 97.62 | 98.05 | 97.61 | |
| Qwen3-VLCategory=General VLMs, Parameters=4B2026.07 | 97.77 | 54.42 | 42.92 | 73.08 | 40.74 | 72.62 | 58.02 | 53.68 | |
| LLaVa-OVCategory=General VLMs, Parameters=7B2026.07 | 96.09 | 39.6 | 37.25 | 36.54 | 51.85 | 86.9 | 51.06 | 41.59 | |
| InternVL3.5Category=General VLMs, Parameters=8B2026.07 | 96.09 | 47.29 | 33.99 | 67.31 | 35.19 | 73.81 | 51.74 | 47.73 | |
| ShotVLCategory=Camera Movement Specialized VLMs, Parameters=7B2026.07 | 96.09 | 69.8 | 63.62 | 63.46 | 29.63 | 98.81 | 71.33 | 64.49 | |
| Mega-SaMCategory=Visual Geometry Models2026.07 | 95.53 | 76.35 | 61.66 | — | — | — | — | — | |
| InternVideo2.5Category=General VLMs, Parameters=8B2026.07 | 94.97 | 44.73 | 31.15 | 75 | 27.78 | 91.67 | 50.98 | 46.71 | |
| Our SFT Qwen3-VL-8BMode=SFT, Base Model=Qwen3-VL, Parameters=8B2026.07 | 94.41 | 72.36 | 74.51 | 82.69 | 68.52 | 91.67 | 78.2 | 77.44 | |
| Gemini-3.1-ProCategory=General VLMs2026.07 | 92.07 | 62 | 74.07 | 69.23 | 55.56 | 85.37 | 73.01 | 69.58 | |
| Qwen3-VLCategory=General VLMs, Parameters=32B2026.07 | 91.62 | 68.95 | 55.77 | 82.69 | 51.85 | 67.86 | 67.01 | 64.17 | |
| Qwen2.5-VLCategory=General VLMs, Parameters=7B2026.07 | 91.06 | 52.99 | 50.54 | 82.69 | 61.11 | 94.05 | 62.43 | 58.5 | |
| Qwen3-VLCategory=General VLMs, Parameters=8B2026.07 | 90.5 | 62.39 | 45.1 | 80.77 | 55.56 | 83.33 | 61.92 | 60.91 | |
| Gemini-3-ProCategory=General VLMs2026.07 | 89.94 | 61.54 | 68.63 | 73.08 | 53.7 | 88.1 | 70.65 | 66.27 | |
| InternVL3.5Category=General VLMs, Parameters=14B2026.07 | 88.27 | 41.6 | 47.93 | 76.92 | 31.48 | 86.9 | 55.47 | 51.15 | |
| CamReasonerCategory=Camera Movement Specialized VLMs, Parameters=7B2026.07 | 84.92 | 51.28 | 42.7 | 67.31 | 55.56 | 77.38 | 55.81 | 53.74 | |
| Our SFT Qwen3-VL-4BMode=SFT, Base Model=Qwen3-VL, Parameters=4B2026.07 | 82.68 | 74.36 | 66.67 | 78.85 | 68.52 | 84.52 | 73.28 | 74.4 | |
| Spatial-MLLMCategory=Spatial VLMs2026.07 | 68.33 | 29.65 | 27.07 | 50 | 43.64 | 94.05 | 40.75 | 36.04 | |
| ViPECategory=Visual Geometry Models2026.07 | 68.16 | 52.42 | 78.65 | — | — | — | — | — | |
| GPT-5Category=General VLMs2026.07 | 64.8 | 69.8 | 57.95 | 82.69 | 53.7 | 63.1 | 63.78 | 65.09 | |
| CameraModelCategory=Camera Movement Specialized VLMs, Parameters=7B2026.07 | 58.1 | 62.68 | 51.63 | 71.15 | 88.89 | 98.81 | 61.83 | 68.45 | |
| Random Guess2026.07 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | |
| G2VLMCategory=Spatial VLMs, Parameters=2B2026.07 | 4.44 | 2.91 | 6.99 | 1.92 | 5.45 | 13.1 | 5.54 | 4.51 |