Spatio-temporal reasoning on V-STAR (test)
64.1What AccuracyOpen-o3 + MCoT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Open-o3 + MCoTGrounded reasoning component (r_ground)=true, Multi-Chain-of-Thought (MCoT)=true2026.03 | 64.1 | 27.3 | 26.8 | 33.6 | 38.1 | 36.6 | 50.5 | |
| Open-o3 + MCoTGrounded reasoning component (r_ground)=false, Multi-Chain-of-Thought (MCoT)=true2026.03 | 63.4 | 26.3 | 25.6 | 30.5 | 31.4 | 35.2 | 49.1 | |
| Motion-oGrounded reasoning component (r_ground)=true2026.03 | 62.7 | 26.6 | 26.2 | 28.2 | 15.1 | 35.5 | 49.4 | |
| Motion-oGrounded reasoning component (r_ground)=false2026.03 | 62.2 | 26.2 | 25.8 | 26.6 | 12.3 | 35 | 48.8 | |
| SER-7BParameters=7B2026.06 | 61.6 | 27.9 | 27.3 | 30.7 | 5.1 | 35.7 | 49.6 | |
| Open-o3 Video2026.03 | 61 | 24.5 | 24 | 25.4 | 6 | 33.7 | 46.6 | |
| Open-o3-VideoReference=ICML262026.06 | 61 | 24.5 | 24 | 25.4 | 6 | 33.7 | 46.6 | |
| GPT-4o2026.03 | 60.8 | 16.7 | 12.8 | 6.5 | 3 | 26.8 | 38.2 | |
| GPT-4o2026.06 | 60.8 | 16.7 | 12.8 | 6.5 | 3 | 26.8 | 38.2 | |
| Gemini-2-Flash2026.03 | 53 | 24.5 | 23.8 | 4.6 | 2.2 | 26.9 | 35.6 | |
| Gemini-2-Flash2026.06 | 53 | 24.5 | 23.8 | 4.6 | 2.2 | 26.9 | 35.6 | |
| LLaVA-Video2026.03 | 49.5 | 10.5 | 12.2 | 1.9 | 1.3 | 20.8 | 27.3 | |
| LLaVA-Video2026.06 | 49.5 | 10.5 | 12.2 | 1.9 | 1.3 | 20.8 | 27.3 | |
| InternVL-2.5-8B2026.03 | 44.2 | 8.7 | 7.8 | 0.7 | 0.1 | 17.6 | 24.9 | |
| InternVL-2.5-8BParameters=8B2026.06 | 44.2 | 8.7 | 7.8 | 0.7 | 0.1 | 17.6 | 24.9 | |
| Video-LLaMA32026.03 | 41.9 | 23 | 23.1 | 0.9 | 0.2 | 21.7 | 27 | |
| Video-LLaMA32026.06 | 41.9 | 23 | 23.1 | 0.9 | 0.2 | 21.7 | 27 | |
| VideoChat22026.03 | 36.2 | 13.7 | 12.5 | 2.5 | 1 | 17 | 20.3 | |
| VideoChat22026.06 | 36.2 | 13.7 | 12.5 | 2.5 | 1 | 17 | 20.3 | |
| Qwen2.5-VL-7B (base)Evaluation framework=vLLM, Number of sampled frames=162026.03 | 33.5 | 15.4 | 13.8 | 17 | 2.5 | 19.3 | 22.4 | |
| Qwen2.5-VL-7BParameters=7B, Configuration=Base model2026.06 | 33.5 | 15.4 | 13.8 | 17 | 2.5 | 19.3 | 22.4 | |
| Oryx-1.5-7B2026.03 | 20.5 | 13.5 | 14.8 | 10.1 | 3.5 | 15.1 | 13.8 | |
| Oryx-1.5-7BParameters=1.5-7B2026.06 | 20.5 | 13.5 | 14.8 | 10.1 | 3.5 | 15.1 | 13.8 | |
| TRACE2026.03 | 17.6 | 19.1 | 17.1 | 0 | 0 | 12 | 13.3 | |
| TRACE2026.06 | 17.6 | 19.1 | 17.1 | 0 | 0 | 12 | 13.3 | |
| Sa2VA-8B2026.03 | 16.4 | 0.1 | 0 | 32.3 | 37.5 | 17.1 | 20.3 | |
| Sa2VA-8BParameters=8B2026.06 | 16.4 | 0.1 | 0 | 32.3 | 37.5 | 17.1 | 20.3 |