Video Physical Reasoning on MASS-Bench 1.0 (test)
65.82Spatial Understanding (SU) AccuracyGemini-2.5-Flash
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemini-2.5-FlashOpen/Closed-Source Status=Close-Source, Model Group=Larger / Close-Source2025.11 | 65.82 | 59.65 | 68.57 | 65.15 | 43.8 | 63.18 | |
| Qwen2.5-VL-7B + MASSModel Augmentation=MASS, Parameter Scale=7B, Model Group=VLMs with MASS2025.11 | 63.52 | 63.06 | 64.07 | 65.06 | 45.79 | 61.93 | |
| LLaVA-OneVision-7B + MASSModel Augmentation=MASS, Parameter Scale=7B, Model Group=VLMs with MASS2025.11 | 61.26 | 65.41 | 58.15 | 61.45 | 42.37 | 59.24 | |
| Qwen3-VL-32BModel Family=Qwen, Parameter Scale=32B, Model Group=Larger / Close-Source2025.11 | 58.87 | 50.59 | 48.05 | 59.84 | 38.63 | 53.44 | |
| Video-R1Fine-tuning Strategy=R1-finetuned, Model Group=R1-finetuned Reasoning VLMs2025.11 | 57.49 | 55.19 | 47.65 | 63.53 | 33.91 | 53.08 | |
| Qwen3-VL-8BModel Family=Qwen, Parameter Scale=8B, Model Group=Baselines2025.11 | 57.29 | 50.77 | 51.92 | 58.65 | 37.07 | 53.23 | |
| Qwen2.5-VL-7BModel Family=Qwen, Parameter Scale=7B, Model Group=Baselines2025.11 | 55.87 | 53.03 | 54.89 | 52.36 | 30.21 | 52.3 | |
| GPT-4oOpen/Closed-Source Status=Close-Source, Model Group=Larger / Close-Source2025.11 | 54.47 | 55.13 | 55.62 | 60.15 | 31.32 | 52.81 | |
| InternVL3.5-38B-FlashModel Family=InternVL, Parameter Scale=38B, Model Group=Larger / Close-Source2025.11 | 52.19 | 49.88 | 43.29 | 61.45 | 32.71 | 48.71 | |
| VideoChat-R1Fine-tuning Strategy=R1-finetuned, Model Group=R1-finetuned Reasoning VLMs2025.11 | 51.2 | 48.66 | 47.34 | 52.3 | 31.09 | 47.05 | |
| LLaVA-OneVision-7BModel Family=LLaVA, Parameter Scale=7B, Model Group=Baselines2025.11 | 49.18 | 51.99 | 38.12 | 56.02 | 30.46 | 45.79 | |
| InternVL3-8BModel Family=InternVL, Parameter Scale=8B, Model Group=Baselines2025.11 | 48.2 | 49.89 | 39.29 | 57.2 | 32.47 | 45.59 | |
| InternVL3.5-14B-FlashModel Family=InternVL, Parameter Scale=14B, Model Group=Baselines2025.11 | 47.68 | 49.23 | 41.96 | 57.52 | 31.03 | 45.71 | |
| InternVL3.5-8B-FlashModel Family=InternVL, Parameter Scale=8B, Model Group=Baselines2025.11 | 47.22 | 45.25 | 39.4 | 58.65 | 32.76 | 44.68 | |
| VideoLLaVAModel Family=VideoLLaVA, Model Group=Baselines2025.11 | 36.39 | 31.76 | 21.01 | 38.55 | 17.76 | 30.72 |