Spatio-temporal reasoning on VSTemporalI-Bench
36.1Object-Object Relative Position ErrorChance Level (Random)
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Chance Level (Random)Rank=-2025.05 | 36.1 | — | — | — | 36.1 | 50 | |
| VILA-1.5-8BRank=8, Model Type=Open-sourced VLMs2025.05 | 50.4 | 37.3 | 30.1 | 27.3 | 42.2 | 36.7 | |
| Chance Level (Frequency)Rank=-2025.05 | 52.2 | 27.4 | 5.4 | 6.2 | 40.7 | 32.4 | |
| LongVILA-8BRank=11, Model Type=Open-sourced VLMs2025.05 | 52.3 | 30.5 | 20 | 11.6 | 35.4 | 33.4 | |
| LongVA-7BRank=10, Model Type=Open-sourced VLMs2025.05 | 57.9 | 32.3 | 13.5 | 5.1 | 43.7 | 41.2 | |
| GPT-4oRank=2, Model Type=Proprietary Models (API)2025.05 | 58.1 | 38.2 | 29.5 | 23.4 | 37.3 | 42.5 | |
| LLaVA-OneVision-7BRank=4, Model Type=Open-sourced VLMs2025.05 | 62.1 | 41.7 | 29.9 | 19.3 | 47.5 | 49.8 | |
| LLaVA-NeXT-Video-7BRank=5, Model Type=Open-sourced VLMs2025.05 | 64.7 | 40 | 28.2 | 1.8 | 49.8 | 55.6 | |
| VILA-1.5-40BRank=6, Model Type=Open-sourced VLMs2025.05 | 65.4 | 38.2 | 28.2 | 15.7 | 28.8 | 53 | |
| InternVL2-8BRank=3, Model Type=Open-sourced VLMs2025.05 | 68 | 43.5 | 32.9 | 13.5 | 48 | 55 | |
| LLaVA-NeXT-Video-72BRank=2, Model Type=Open-sourced VLMs2025.05 | 78.3 | 44 | 32.3 | 10.5 | 48.1 | 50.9 | |
| Gemini-2.5 ProRank=1, Model Type=Proprietary Models (API)2025.05 | 84 | 42.4 | 29.2 | 8 | 30 | 60.7 | |
| VLM-3R (7B)Rank=1, Model Type=Open-sourced VLMs2025.05 | 86.5 | 58.8 | 39.4 | 39.6 | 60.6 | 68.6 | |
| †Human LevelRank=-2025.05 | 97.5 | 77 | 51.4 | 46.8 | 95.1 | 94.3 |